Skip to content

Instantly share code, notes, and snippets.

@pgwillia
Last active November 21, 2025 21:03
Show Gist options
  • Select an option

  • Save pgwillia/b9a05c548f9f94cbe60424f94a70e825 to your computer and use it in GitHub Desktop.

Select an option

Save pgwillia/b9a05c548f9f94cbe60424f94a70e825 to your computer and use it in GitHub Desktop.
Skill Building Uncoference
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"cell_type": "markdown",
"id": "767beb70",
"metadata": {},
"source": [
"# Skill Building Unconference\n",
"\n",
"## Goals\n",
"1. Responsible AI\n",
"2. Deeper dive into Data Pipelines - Apache Airflow\n",
"3. Project Management - practice with diagrams and risk analysis"
]
},
{
"cell_type": "markdown",
"id": "2e5e24c0",
"metadata": {},
"source": [
"# AI webinars\n",
"\n",
"\n",
"## Responsible AI\n",
"We look forward to seeing you at Responsible AI in 2025, a virtual research conference hosted by Good Systems — Ethical AI at UT Austin. View the full program and speaker bios here: responsibleai2025.splashthat.com\n",
"\n",
"Zoom webinar link: https://us06web.zoom.us/j/85813523512 Webinar ID: 858 1352 3512\n",
"\n",
"## The AI generation: Innovation powered by co-op students \n",
"Event: The AI generation: Innovation powered by co-op students \n",
"Time: Nov 19, 2025, 11:00 AM America/Edmonton\n",
"Host: Co-operative and Experiential Education\n",
"\n",
"View event detail\n",
"https://events.zoom.us/ev/AhXA0aAVONr_0CGRxV_zAgIz0Lud7gLVROhmYaZK6yQ-Yj7Gr9ki~AhHd0znMo0CWknKq1niJ0XwQx5w5RwlNudLpojP0pX_aBSKHgbO6XAgRng"
]
},
{
"cell_type": "markdown",
"id": "0f0f6827",
"metadata": {},
"source": [
"# Data Pipelines - Apache Airflow\n",
"`/mnt/aviary/dags/` is just: https://github.com/ualbertalib/airflow_dags_aviary_staging\n",
"\n",
"Reading the docs https://airflow.apache.org/docs/\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "66c1b006",
"metadata": {},
"source": [
"# Project Management\n",
"\n",
"## Diagrams\n",
"\n",
"### C4 \n",
"- https://c4model.com/introduction\n",
"- https://www.drawio.com/blog/c4-modelling -- this is money! And [Draw.io in VSCode](https://www.drawio.com/blog/embed-diagrams-vscode)\n",
"\n",
"![sample diagram](./diagram.drawio.png)\n",
"\n",
"### Mermaid \n",
"- https://github.blog/developer-skills/github/include-diagrams-markdown-files-mermaid/\n",
"- [Mermaid video LinkedIn Learning](https://www.linkedin.com/learning/mermaid-diagrams/null?trk=share_android_video_learning&shareId=N%2FqW9XlvQvyPG%2B00n9SjmQ%3D%3D)\n",
"\n",
"### Types\n",
"- sketching and drawing: MS Paint, Procreate, infographics\n",
"- whiteboard: Miro, Mind Meister, Canva, Figma\n",
"- venn diagram\n",
"- technical diagramming - flowcharts, UML, business hierarchies, relational models, activity diagram, mockup, architecture diagram, c4, state diagram, and user, data and control flows. Draw.io\n",
"- CAD and modeling tools: AutoCAD\n",
"- [Grafana Flowcharting](https://github.com/algenty/grafana-flowcharting) -- needs more investigation\n",
"- Matrix management diagrams show how people report to more than one supervisor in a cross-functional, cross-business model.\n",
"- Organigraphs represent the relationships between departments by connecting nodes based on processes and not just on reporting hierarchies.\n",
"- [Showing probabilities and risk in diagrams](https://www.drawio.com/blog/probability-risk-diagrams)\n",
"- [Threat Modeling](https://www.drawio.com/blog/threat-modelling): Data flow (DFD), Process flow (PFD), Attack trees "
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "88a0eff5",
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "fa0582d8",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Name</th>\n",
" <th>State</th>\n",
" <th>Status</th>\n",
" <th>Provisioned Space</th>\n",
" <th>Used Space</th>\n",
" <th>Host CPU</th>\n",
" <th>Host Mem</th>\n",
" <th>Guest OS</th>\n",
" <th>CPUs</th>\n",
" <th>DNS Name</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>acton</td>\n",
" <td>Powered On</td>\n",
" <td>Normal</td>\n",
" <td>22.09 GB</td>\n",
" <td>22.09 GB</td>\n",
" <td>104 MHz</td>\n",
" <td>1.56 GB</td>\n",
" <td>CentOS 7 (64-bit)</td>\n",
" <td>4</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>albion.library.ualberta.ca</td>\n",
" <td>Powered On</td>\n",
" <td>Normal</td>\n",
" <td>24.08 GB</td>\n",
" <td>24.08 GB</td>\n",
" <td>242 MHz</td>\n",
" <td>3.7 GB</td>\n",
" <td>Red Hat Enterprise Linux 9 (64-bit)</td>\n",
" <td>2</td>\n",
" <td>albion.library.ualberta.ca</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>ayr</td>\n",
" <td>Powered On</td>\n",
" <td>Normal</td>\n",
" <td>24.09 GB</td>\n",
" <td>24.09 GB</td>\n",
" <td>308 MHz</td>\n",
" <td>4.03 GB</td>\n",
" <td>CentOS 7 (64-bit)</td>\n",
" <td>4</td>\n",
" <td>ayr.library.ualberta.ca</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>bld-srv-prd-1</td>\n",
" <td>Powered On</td>\n",
" <td>Normal</td>\n",
" <td>168.09 GB</td>\n",
" <td>168.09 GB</td>\n",
" <td>264 MHz</td>\n",
" <td>3.14 GB</td>\n",
" <td>Red Hat Enterprise Linux 9 (64-bit)</td>\n",
" <td>4</td>\n",
" <td>bld-srv-prd-1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>bld-srv-prd-2.library.ualberta.ca</td>\n",
" <td>Powered On</td>\n",
" <td>Normal</td>\n",
" <td>93.08 GB</td>\n",
" <td>93.08 GB</td>\n",
" <td>167 MHz</td>\n",
" <td>4.36 GB</td>\n",
" <td>AlmaLinux (64-bit)</td>\n",
" <td>4</td>\n",
" <td>bld-srv-prd-2.library.ualberta.ca</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Name State Status Provisioned Space \\\n",
"0 acton Powered On Normal 22.09 GB \n",
"1 albion.library.ualberta.ca Powered On Normal 24.08 GB \n",
"2 ayr Powered On Normal 24.09 GB \n",
"3 bld-srv-prd-1 Powered On Normal 168.09 GB \n",
"4 bld-srv-prd-2.library.ualberta.ca Powered On Normal 93.08 GB \n",
"\n",
" Used Space Host CPU Host Mem Guest OS CPUs \\\n",
"0 22.09 GB 104 MHz 1.56 GB CentOS 7 (64-bit) 4 \n",
"1 24.08 GB 242 MHz 3.7 GB Red Hat Enterprise Linux 9 (64-bit) 2 \n",
"2 24.09 GB 308 MHz 4.03 GB CentOS 7 (64-bit) 4 \n",
"3 168.09 GB 264 MHz 3.14 GB Red Hat Enterprise Linux 9 (64-bit) 4 \n",
"4 93.08 GB 167 MHz 4.36 GB AlmaLinux (64-bit) 4 \n",
"\n",
" DNS Name \n",
"0 NaN \n",
"1 albion.library.ualberta.ca \n",
"2 ayr.library.ualberta.ca \n",
"3 bld-srv-prd-1 \n",
"4 bld-srv-prd-2.library.ualberta.ca "
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = pd.read_csv('vms.csv')\n",
"df.head()"
]
},
{
"cell_type": "markdown",
"id": "8ab370d9",
"metadata": {},
"source": [
"Wondering if there was an automated way to get diagrams from our [Ansible tree](https://github.com/ualbertalib/build-centos9)\n",
"\n",
"Discovered [ansible-playbook-grapher](https://networktocode.com/blog/visualizing-ansible-plays-and-tasks/). "
]
},
{
"cell_type": "markdown",
"id": "6cbcfc40",
"metadata": {},
"source": [
"## Software Architecture\n",
"\n",
"### Tools\n",
"\n",
"#### Mind Mapping\n",
"\n",
"#### Hype Cycle\n",
"\n",
"Maturity of emerging technologies\n",
"<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/jB1RDz9jaj0?si=qtCxltfebSwi0gYS\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen></iframe>\n",
"\n",
"Not available at UAL\n",
"Here's a list of similar\n",
"- [LSEG Insiders Data](https://guides.library.ualberta.ca/az/thomson-reuters-insiders-data-wrds)\n",
"- [BCC Researcher](https://guides.library.ualberta.ca/az/bcc-research) [State of the Digital World Industry](IFT324A-Overview.pdf)\n",
"- [MarketResearch](https://guides.library.ualberta.ca/az/marketresearchcom-academic)\n",
"- [Some Gartner Alternatives](https://libraries.emory.edu/media/10296)\n",
"\n",
"State of Industry - Information Technology (IFT) [State of the Digital World Industry](IFT324A-Overview.pdf)\n",
"\n",
"The IT landscape is resetting around three forces: cloud migration, AI acceleration and cybersecurity resilience. Demand is shifting from traditional infrastructure toward hyperscale compute, intelligent automation, and secure digital platforms that enable enterprise modernization, AI-native applications and real-time decisions. AI workloads remain the focal point, but the cycle is mixed. Microsoft is expanding AI copilots across Office and Azure ecosystems, while Google is scaling Gemini integrations into Workspace and cloud APIs, both signaling strong enterprise uptake. Yet, several venture-backed AI startups are pausing global expansion amid tighter funding and customer consolidation, highlighting near-term volatility in enterprise AI adoption.\n",
"\n",
"In the enterprise software sector, incumbents are increasingly adopting “platform-first” strategies. Salesforce is embedding AI copilots into CRM and workflow automation, and SAP is aligning capex and R&D to AI-augmented ERP roadmaps. Similarly, ServiceNow is advancing automation suites with generative and predictive capabilities. Meanwhile, low-code and automation players, UiPath, Mendix and OutSystems, are shifting from pilot to scaled deployments across regulated sectors such as finance, healthcare and government, underscoring mainstream adoption momentum.\n",
"\n",
"Hardware and semiconductor companies are also repositioning their capacity to meet AI-driven demand. Nvidia continues to lead in GPU architecture with new innovations and expanded supply agreements with TSMC. AMD is accelerating competitive CPU/GPU launches for data center workloads, and Intel is consolidating manufacturing assets and partnerships to regain share. At the device edge, Qualcomm, Apple and MediaTek are embedding AI accelerators into next-gen processors to capture mobile and IoT intelligence opportunities.\n",
"\n",
"Cybersecurity players are aligning portfolios with cloud-first and AI-enabled threat landscapes. Palo Alto Networks is broadening its platform coverage with integrated SASE and SOC automation, CrowdStrike is advancing endpoint-to-cloud security orchestration, while Fortinet is streamlining around unified threat management. Zero-trust frameworks are being reinforced by U.S. and EU regulatory mandates, accelerating the adoption curves of enterprises.\n",
"\n",
"[Predictions 2026: Cybersecurity And Risk Leaders Grapple With New Tech And Geopolitical Threats](https://www.forrester.com/blogs/predictions-2026-cybersecurity-and-risk/)\n",
"2025 was a tumultuous year for cybersecurity professionals. A change in political leadership in the US introduced instability within federal cybersecurity agencies and had a worldwide ripple effect; the focus on AI technology shifted from generative AI (genAI) to agent and agentic AI for productivity, cybersecurity, and malicious actors; and the variety of cyberattacks targeting critical infrastructure markets as well as average businesses, reaching all four corners of the globe, kept security and risk teams on their toes.\n",
"\n",
"In 2026, continued political instability coupled with technological advancements being used by cybercriminals will force security, risk, and privacy leaders to not just adapt their defensive technologies to respond but to also prepare their workforce for these shifts to reduce the risk to the business. To help business and security leaders understand where to focus their attention in the year ahead, here are three of Forrester’s 2026 cybersecurity and risk predictions:\n",
"\n",
"1. An agentic AI deployment will cause a public breach and lead to employee dismissals. Since its launch in 2022, genAI has caused several data breaches or affected the integrity or availability of sensitive data. As companies begin building agentic AI workflows, these issues will only become more prevalent. Without the right guardrails, systems of autonomous AI agents may sacrifice accuracy for speed of delivery, especially when interacting directly with customers. When these failures occur, some treat AI agents as their own entities while others point fingers at individual employees, but breaches like these are due to a cascade of failures, not a single individual. To prevent these failures, and scapegoating, security organizations must enable the business to develop agentic applications with minimum viable security. Follow the AEGIS framework, securing intent, ensuring appropriate identity and access management controls to track agent activity, and implementing data security controls to track data provenance.\n",
"2. Five governments will nationalize or place restrictions on critical telecom infrastructure. The Salt Typhoon cyberespionage campaign, attributed to nation-state actors, breached over 600 orgs across 80 countries, exposing the vulnerability of commercial telecom as hackers went undetected for years. Governments responded: Australia reinforced SOCI (Security of Critical Infrastructure) Act reforms, mandating direct oversight of telecom assets; Italy advanced a €22 billion restructuring of Telecom Italia’s network while planning its own satellites for encrypted comms; and the US banned Chinese and Russian ownership of subsea cables and bolstered cybersecurity standards. Telecom, however, relies on vast internet-of-things ecosystems (notoriously insecure and frequently exploited), while the rapid rise of space infrastructure such as low-Earth-orbit satellites adds new attack surfaces. To counter, governments will assert unprecedented control over telecom security. To stay ahead of new security regulations, CISOs must strengthen continuous monitoring of critical ecosystem risks while evolving to continuous control monitoring.\n",
"3. Quantum security spending will exceed 5% of the overall IT security budget. Forrester estimates that commercial quantum computers will break today’s asymmetric cryptography in less than 10 years and, given regular advances, sooner. Meanwhile, NIST guidance dictates that RSA and ECC support will be deprecated in 2030 and disallowed in 2035. In response, security teams will ramp up quantum security spending overnight in several areas. First, many will retain consulting services to help plan quantum security migrations. Second, product security teams will work with development counterparts to replace outdated cryptographic libraries and components. Third, security teams will work with risk and procurement colleagues to track vendor and partner quantum migration plans. Finally, teams will invest heavily in cryptographic discovery and inventory tools to prioritize high-impact systems for migration, and many will pilot cryptographic agility solutions. Quantum security is no longer just a concern for banking and critical infrastructure; all CISOs must consider similar spending.\n",
"\n",
"\n",
"Digitalization, especially AI, is reducing IT development cycles and reshaping go-to-market. Leaders are applying AI to code generation, testing and deployment: GitHub Copilot, Google’s Gemini Code Assist and Microsoft’s DevOps Copilots are reducing release times and boosting productivity, while platform players are deploying simulation-driven design to speed enterprise pilots. At the frontier, quantum computing remains experimental; IBM, IonQ and PsiQuantum are advancing pilots in pharma, logistics and advanced materials, yet commercialization timelines remain extended.\n",
"\n",
"It is relevant to say that the industry’s center of gravity is moving toward intelligent, secure and platformed IT, rewarding players that combine disciplined investment with ecosystem partnerships, regionalized cloud capacity and AI-enabled innovation engines.\n",
"\n",
"#### Innovation Funnel\n",
"\n",
"Collect ideas and filter them until they can be turned into a stable project"
]
},
{
"cell_type": "markdown",
"id": "b9d9840c",
"metadata": {},
"source": [
"@"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.14.0"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Display the source blob
Display the rendered blob
Raw
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment