brad/LLM_Engineering_OLD
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Launching refreshed version of LLM Engineering weeks 1-4 - see README

Browse Source
This commit is contained in:
Edward Donner
2025-10-11 15:58:39 -04:00
parent 3286cfb395
commit c7257b9ae6
68 changed files with 16583 additions and 3756 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.gitignore vendored
View File

@@ -194,3 +194,8 @@ challenge/
# WandB local sync data.
wandb/
week2/prices.db
week4/main.cpp
week4/main
week4/main.exe
week4/main.rs

1
.python-version Normal file
View File

@@ -0,0 +1 @@
3.12

62
README.md
View File

@@ -2,10 +2,27 @@
## Your 8 week journey to proficiency starts today
![Voyage](voyage.jpg)
![Voyage](assets/voyage.jpg)
I'm so happy you're joining me on this path. We'll be building immensely satisfying projects in the coming weeks. Some will be easy, some will be challenging, many will ASTOUND you! The projects build on each other so you develop deeper and deeper expertise each week. One thing's for sure: you're going to have a lot of fun along the way.
# IMPORTANT ANNOUNCEMENT - OCTOBER 2025 - PLEASE READ
I am phasing in new, updated versions of all the course videos, with new videos and new code. I realize this can be quite jarring for people already on the course, and I will do my best to minimize headache!
- Both video series will be available in Udemy and you can watch either. A new week should become available each week as we roll this out.
- You can follow the original videos or the new videos - either should work great. Switch between them at any time.
- The latest code is pushed to the repo. You can follow along with new code, or revert to original code.
The most significant change is that the new version uses the fabulous uv, instead of Anaconda! But there's also tons of new content, including new models, tools and techniques. Prompt caching, LiteLLM, inference techniques and so much more.
### To revert to the original version of code, consistent with the original videos (Anaconda + virtualenv)
If you'd prefer to stick with the code for the original videos, simply do this from your Anaconda Prompt or Terminal:
`git fetch`
`git checkout original`
And that's it! Any questions, please ask me on Udemy or at ed@edwarddonner.com. More details at the top of the course resources [here](https://edwarddonner.com/2024/11/13/llm-engineering-resources/).
### Before you begin
I'm here to help you be most successful with your learning! If you hit any snafus, or if you have any ideas on how I can improve the course, please do reach out in the platform or by emailing me direct (ed@edwarddonner.com). It's always great to connect with people on LinkedIn to build up the community - you'll find me here:
@@ -34,11 +51,12 @@ After we do the Ollama quick project, and after I introduce myself and the cours
Hopefully I've done a decent job of making these guides bulletproof - but please contact me right away if you hit roadblocks:
- PC people please follow the instructions in [SETUP-PC.md](SETUP-PC.md)
- Mac people please follow the instructions in [SETUP-mac.md](SETUP-mac.md)
- Linux people please follow the instructions in [SETUP-linux.md](SETUP-linux.md)
NEW INSTRUCTIONS for new version of the course (rolled out October 2025): [New Setup Instructions All Platforms](setup/SETUP-NEW.md)
The are also PDF versions of the setup instructions in this folder if you'd prefer.
ORIGINAL INSTRUCTIONS for people on the version prior to October 2025:
- PC people please follow the instructions here: [Original PC instructions](setup/SETUP-PC.md)
- Mac people please follow the instructions here: [Original Mac instructions](setup/SETUP-mac.md)
- Linux people please follow the instructions here: [Original Linux instructions](setup/SETUP-linux.md)
### An important point on API costs (which are optional! No need to spend if you don't wish)
@@ -48,31 +66,7 @@ Please do monitor your API usage to ensure you're comfortable with spend; I've i
### Free alternative to Paid APIs
Early in the course, I show you an alternative if you'd rather not spend anything on APIs:
Any time that we have code like:
`openai = OpenAI()`
You can use this as a direct replacement:
`openai = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')`
And also replace model names like **gpt-4o-mini** with **llama3.2**.
For week 1 day 1, you can find this in week1/solutions/day1_with_ollama.ipynb.
Below is a full example:
```
# You need to do this one time on your computer
!ollama pull llama3.2
from openai import OpenAI
MODEL = "llama3.2"
openai = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
response = openai.chat.completions.create(
model=MODEL,
messages=[{"role": "user", "content": "What is 2 + 2?"}]
)
print(response.choices[0].message.content)
```
See [Guide 9](guides/09_ai_apis_and_ollama.ipynb) in the guides directory for the detailed approach with exact code for Ollama, Gemini, OpenRouter and more!
### How this Repo is organized
@@ -81,7 +75,7 @@ Follow the setup instructions above, then open the Week 1 folder and prepare for
### The most important part
The mantra of the course is: the best way to learn is by **DOING**. I don't type all the code during the course; I execute it for you to see the results. You should work along with me or after each lecture, running each cell, inspecting the objects to get a detailed understanding of what's happening. Then tweak the code and make it your own. There are juicy challenges for you throughout the course. I'd love it if you wanted to submit a Pull Request for your code (instructions [here](https://chatgpt.com/share/677a9cb5-c64c-8012-99e0-e06e88afd293)) and I can make your solutions available to others so we share in your progress; as an added benefit, you'll be recognized in GitHub for your contribution to the repo. While the projects are enjoyable, they are first and foremost designed to be _educational_, teaching you business skills that can be put into practice in your work.
The mantra of the course is: the best way to learn is by **DOING**. I don't type all the code during the course; I execute it for you to see the results. You should work along with me or after each lecture, running each cell, inspecting the objects to get a detailed understanding of what's happening. Then tweak the code and make it your own. There are juicy challenges for you throughout the course. I'd love it if you wanted to submit a Pull Request for your code (see the Github guide in the guides folder) and I can make your solutions available to others so we share in your progress; as an added benefit, you'll be recognized in GitHub for your contribution to the repo. While the projects are enjoyable, they are first and foremost designed to be _educational_, teaching you business skills that can be put into practice in your work.
## Starting in Week 3, we'll also be using Google Colab for running with GPUs
@@ -99,10 +93,10 @@ The colab links are in the Week folders and also here:
### Monitoring API charges
You can keep your API spend very low throughout this course; you can monitor spend at the dashboards: [here](https://platform.openai.com/usage) for OpenAI, [here](https://console.anthropic.com/settings/cost) for Anthropic and [here](https://console.cloud.google.com/apis/api/generativelanguage.googleapis.com/cost) for Google Gemini.
You can keep your API spend very low throughout this course; you can monitor spend at the dashboards: [here](https://platform.openai.com/usage) for OpenAI, [here](https://console.anthropic.com/settings/cost) for Anthropic.
The charges for the exercsies in this course should always be quite low, but if you'd prefer to keep them minimal, then be sure to always choose the cheapest versions of models:
1. For OpenAI: Always use model `gpt-4o-mini` in the code instead of `gpt-4o`
1. For OpenAI: Always use model `gpt-4.1-nano` in the code
2. For Anthropic: Always use model `claude-3-haiku-20240307` in the code instead of the other Claude models
3. During week 7, look out for my instructions for using the cheaper dataset
@@ -111,7 +105,7 @@ Please do message me or email me at ed@edwarddonner.com if this doesn't work or
<table style="margin: 0; text-align: left;">
<tr>
<td style="width: 150px; height: 150px; vertical-align: middle;">
<img src="resources.jpg" width="150" height="150" style="display: block;" />
<img src="assets/resources.jpg" width="150" height="150" style="display: block;" />
</td>
<td>
<h2 style="color:#f71;">Other resources</h2>

BIN
SETUP-PC.pdf
View File

Binary file not shown.

BIN
SETUP-linux.pdf
View File

Binary file not shown.

BIN
SETUP-mac.pdf
View File

Binary file not shown.

0
business.jpg → assets/business.jpg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 367 KiB

After

Width:  |  Height:  |  Size: 367 KiB

0
important.jpg → assets/important.jpg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 356 KiB

After

Width:  |  Height:  |  Size: 356 KiB

0
resources.jpg → assets/resources.jpg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 439 KiB

After

Width:  |  Height:  |  Size: 439 KiB

0
thankyou.jpg → assets/thankyou.jpg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 432 KiB

After

Width:  |  Height:  |  Size: 432 KiB

0
voyage.jpg → assets/voyage.jpg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 761 KiB

After

Width:  |  Height:  |  Size: 761 KiB

68
guides/01_intro.ipynb Normal file
View File

@@ -0,0 +1,68 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Welcome to the Guides!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Introduction\n",
"\n",
"I've designed this course to be suitable for a wide range of backgrounds. For those of you relatively new to this, I've prepared some technical briefings to build your expertise.\n",
"\n",
"These are designed to be self-study; you work through them at your own pace, investigating and experimenting.\n",
"\n",
"I've heavily taken advantage of our AI friends to write some guides, and I've tried to frame them so they're as useful as possible and relevant to the course.\n",
"\n",
"There's only one requirement for the course: plenty of patience! Keep in mind that one of the best ways to learn is by solving problems - if you feel frustrated with a challenging puzzle, remember that this is where the learning happens! And, get in touch if I can help."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Contents (select from the sub-directory in the explorer on the left)\n",
"\n",
"1. Intro - this contents\n",
"2. The command line\n",
"3. Git and Github\n",
"4. Technical foundations (environment variables, networks, APIs, uv)\n",
"5. Notebooks\n",
"6. Python foundations\n",
"7. \"Vibe coding\" - successfully coding with the help of LLMs\n",
"8. Debugging techniques\n",
"9. APIs and Ollama\n",
"10. Intermediate level python, including decorators and async \n",
"11. Asynchronous Python\n",
"12. Starting your project - 3 crucial pieces of advice\n",
"13. Frontend Crash Course\n",
"14. Briefings on Docker and Terraform\n",
"\n",
"### Also see the community_contributions directory for some awesome Python cookbooks contributed by students! (Thank you!)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.12.9"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

63
guides/02_command_line.ipynb Normal file
View File

@@ -0,0 +1,63 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# The command line"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Welcome to the guide to build your confidence working at the command line.\n",
"\n",
"By the end of this guide, you should be able to confidently work at the command line, creating directories, moving files and more!\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## For Windows PC users\n",
"\n",
"Please see this briefing:\n",
"\n",
"https://chatgpt.com/share/68064acf-4d3c-8012-86a1-fb09a7c6f923"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## For Mac users\n",
"\n",
"Please see this briefing:\n",
"\n",
"https://chatgpt.com/share/68064bfe-662c-8012-8073-479c32595459"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Please do experiment to build your confidence.\n",
"\n",
"ChatGPT, Claude, Gemini, DeepSeek and others are your friends! They are excellent at explaining any command line commands in detail. This will quickly become second nature."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

81
guides/03_git_and_github.ipynb Normal file
View File

@@ -0,0 +1,81 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Git and Github"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This guide is all about using source code control: Git and Github.\n",
"\n",
"By the end of this, you should be confident with every day code control processes, including fetching the latest code and submitting a PR to merge your own changes."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Foundational briefing\n",
"\n",
"Here is Git and Github for a PC or Mac audience:\n",
"\n",
"https://chatgpt.com/share/68061486-08b8-8012-97bc-3264ad5ebcd4"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Pulling latest code\n",
"\n",
"I regularly add improvements to the course with new examples, exercises and materials.\n",
"\n",
"Here are instructions for how to bring in the latest - the easy way, and the rigorous way!\n",
"\n",
"https://chatgpt.com/share/6806178b-0700-8012-836f-7e87b2670b7b"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Contributing your changes to the repo to share your contributions with others\n",
"\n",
"Here are step by step instructions for submitting a PR.\n",
"\n",
"I'd be so grateful to include your contributions. It adds value for all other students, and I love to see it myself! As an added benefit, you get recognition in Github as a contributor to the repo. As this course involves building entire repos, I'd suggest that you create a Markdown file or Jupyter Notebook that links to your repo, and include it in community_contributions with a PR.\n",
"\n",
"Here are detailed instructions and explanations:\n",
"\n",
"https://chatgpt.com/share/6873c22b-2a1c-8012-bc9a-debdcf7c835b\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"vscode": {
"languageId": "plaintext"
}
},
"source": [
"### If you'd like to become a Git pro\n",
"\n",
"If you want to go deep on using Git, here is a brilliant guide. Read this and you will know much more than me!\n",
"\n",
"https://beej.us/guide/bggit/\n"
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

117
guides/04_technical_foundations.ipynb Normal file
View File

@@ -0,0 +1,117 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Technical Foundations\n",
"\n",
"It's crucial that you feel comfortable with the basic technical concepts that we work with. This will make your experience of the entire course so much better - it can be very frustrating if you're not sure what's gong on.\n",
"\n",
"These guides should build confidence in the underlying technologies we work with."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Topic 1: ChatGPT versus OpenAI API\n",
"\n",
"### What’s the difference between ChatGPT and the GPT API, both offered by OpenAI?\n",
"\n",
"#### ChatGPT is an end-user tool. It’s a Chat product designed for consumers who are AI users.\n",
"- It has a free plan, and it also has paid subscription plans with more features.\n",
"- The subscription plans give the user near-unlimited access to use the Chat product.\n",
"\n",
"#### The API is a service provided for AI engineers - software engineers and data scientists - working on other commercial products.\n",
"- It allows technical people, like you and me, to access the underlying models (like “GPT4.1” and “o3”) so that we can build our own products.\n",
"- If we wanted to, we could build our own version of ChatGPT using the API, and charge our end-users for it.\n",
"- Like most APIs, OpenAI charges a small amount based on API usage. For most examples on the course using gpt-4o-mini, it’s of the order of $0.001 per API call.\n",
"\n",
"### I’m paying $20/month for ChatGPT - why do I need to pay more for the API?\n",
"\n",
"- Hopefully this is now clear. The API is not for consumers; it’s for engineers to build their own platforms that they can charge for.\n",
"- If you were to have access to the API based on your subscription, then you could offer ChatGPT tools to others at a cheaper price, and put OpenAI out of business!\n",
"- Keep in mind: each API call may require 10,000,000,000,000 floating point calculations - that compute uses electricity!\n",
"\n",
"Instead of calling the API, you can run open source models locally, but typically they have 1,000 times fewer calculations — and even though it’s tiny, that processing still hits your electricity bill.."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Topic 2: Taking a Screenshot\n",
"\n",
"You may already be familiar with \"taking a screenshot\" on your computer, but if not (or if you think this means taking a photo with your camera..), please review this tutorial:\n",
"\n",
"https://chatgpt.com/share/681f691b-6644-8012-b07d-207c68f259d5"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Topic 3: Environment Variables and the `.env` file\n",
"\n",
"This tutorial walks you through everything you need to know about .env files!\n",
"\n",
"Obiously you don't need to add the .env file to .gitignore, as I've already done that for you. But it hopefully explains the point well.\n",
"\n",
"https://chatgpt.com/share/68061e89-dd84-8012-829d-9f4506c7baaa"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Topic 4: Networking basics\n",
"\n",
"This tutorial covers networking and typical issues with certificates, VPNs, DNS and the like.\n",
"\n",
"The sections give a summary; you should ask ChatGPT to expand on any section if it's relevant to your situation.\n",
"\n",
"https://chatgpt.com/share/680620ec-3b30-8012-8c26-ca86693d0e3d\n",
"\n",
"This is a more in-depth guide to tackling SSL / certificate issues, which is common in corporate environments:\n",
"\n",
"https://chatgpt.com/share/68c2efc4-0280-8012-933b-5e89d7db6b58"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Topic 5: APIs and Client Libraries - foundational briefing\n",
"\n",
"We use APIs a lot in this course!\n",
"\n",
"It's essential to understand the fundamentals of what's going on when we make a call to an API, and to be comfortable with words like \"endpoint\" and \"client library\".\n",
"\n",
"Please review this guide:\n",
"\n",
"https://chatgpt.com/share/68062432-43c8-8012-ad91-6311d4ad5858"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Topic 6: uv, package management, environment management\n",
"\n",
"This lays out the Dependency Management situation and why we love uv! And a crash course in how to use it.\n",
"\n",
"https://chatgpt.com/share/68c34d46-18a0-8012-8d65-0a0cce615912\n",
"\n",
"Note that this guide suggests `uv run python xxx` which works fine, but simply `uv run xxx` works too and is more common."
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

83
week1/Guide to Jupyter.ipynb → guides/05_notebooks.ipynb
View File

@@ -5,26 +5,30 @@
"id": "5c291475-8c7c-461c-9b12-545a887b2432",
"metadata": {},
"source": [
"# Jupyter Lab\n",
"# Notebooks in Cursor\n",
"\n",
"## A Quick Start Guide\n",
"This course makes heavy use of a brilliant thing called Notebooks (also known as Jupyter Notebooks or Labs.) Those from a traditional software engineering background may feel discomfort with the \"hacky\" nature of Notebooks, but I must assure you: part of working with AI is being comfortable being a Scientist. As a Scientist, there's a lot of exploration and experimentation. And Notebooks are ideal for this kind of activity.\n",
"\n",
"Welcome to the wonderful world of Jupyter lab! \n",
"This is a Data Science playground where you can easily write code and investigate the results. It's an ideal environment for: \n",
"A notebook is a file with the extension \".ipynb\" which stands for IPython Notebook, an early name for these.\n",
"\n",
"## Briefing on Notebooks in Cursor\n",
"\n",
"First, here's a briefing on how this fits together, and how to create and run a notebook in Cursor:\n",
"\n",
"https://chatgpt.com/share/6806291a-25f0-8012-a08b-057acb5045ae\n",
"\n",
"\n",
"## A broader guide to Notebooks with examples\n",
"\n",
"The Notebook is a Data Science playground where you can easily write code and investigate the results. It's an ideal environment for: \n",
"- Research & Development\n",
"- Prototyping\n",
"- Learning (that's us!)\n",
"\n",
"It's not typically used for shipping production code, and in Week 8 we'll explore the bridge between Jupyter and python code.\n",
"\n",
"A file in Jupyter Lab, like this one, is called a **Notebook**.\n",
"\n",
"A long time ago, Jupyter used to be called \"IPython\", and so the extensions of notebooks are \".ipynb\" which stands for \"IPython Notebook\".\n",
"\n",
"On the left is a File Browser that lets you navigate around the directories and choose different notebooks. But you probably know that already, or you wouldn't have got here!\n",
"\n",
"The notebook consists of a series of square boxes called \"cells\". Some of them contain text, like this cell, and some of them contain code, like the cell below.\n",
"\n",
"First, you may need to click the `Select Kernel` button on the top right, and then pick `venv (Python 3.12.x)` or similar.\n",
"\n",
"Click in a cell with code and press `Shift + Return` (or `Shift + Enter`) to run the code and print the output.\n",
"\n",
"Do that now for the cell below this:"
@@ -54,7 +58,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 3,
"id": "585eb9c1-85ee-4c27-8dc2-b4d8d022eda0",
"metadata": {},
"outputs": [],
@@ -90,7 +94,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 6,
"id": "4c5a4e60-b7f4-4953-9e80-6d84ba4664ad",
"metadata": {},
"outputs": [],
@@ -111,7 +115,7 @@
"\n",
"The order that code appears in the notebook doesn't matter. What matters is the order that the code is **executed**. There's a python process sitting behind this notebook in which the variables are being changed.\n",
"\n",
"This catches some people out when they first use Jupyter."
"This catches some people out when they first use notebooks."
]
},
{
@@ -137,7 +141,7 @@
"\n",
"Sitting behind this notebook is a Python process which executes each cell when you run it. That Python process is known as the Kernel. Each notebook has its own separate Kernel.\n",
"\n",
"You can go to the Kernel menu and select \"Restart Kernel\".\n",
"You can click the button above \"Restart Kernel\".\n",
"\n",
"If you then try to run the next cell, you'll get an error, because favorite_fruit is no longer defined. You'll need to run the cells from the top of the notebook again. Then the next cell should run fine."
]
@@ -157,9 +161,9 @@
"id": "4d4188fc-d9cc-42be-8b4e-ae8630456764",
"metadata": {},
"source": [
"# Adding and moving cells\n",
"# Adding and removing cells\n",
"\n",
"Click in this cell, then click the \\[+\\] button in the toolbar above to create a new cell immediately below this one. Copy and paste in the code in the prior cell, then run it! There are also icons in the top right of the selected cell to delete it (bin), duplicate it, and move it up and down.\n"
"Click in this cell, then click the \\[+ Code\\] button in the toolbar above to create a new cell immediately below this one. Copy and paste in the code in the prior cell, then run it! There are also icons in the top right of the selected cell to delete it (bin).\n"
]
},
{
@@ -177,9 +181,9 @@
"source": [
"# Cell output\n",
"\n",
"When you execute a cell, the standard output and the result of the last statement is written to the area immediately under the code, known as the 'cell output'. When you save a Notebook from the file menu (or command+S), the output is also saved, making it a useful record of what happened.\n",
"When you execute a cell, the standard output and the result of the last statement is written to the area immediately under the code, known as the 'cell output'. When you save a Notebook from the file menu (or ctrl+S or command+S), the output is also saved, making it a useful record of what happened.\n",
"\n",
"You can clean this up by going to Edit menu >> Clear Outputs of All Cells, or Kernel menu >> Restart Kernel and Clear Outputs of All Cells."
"You can clean this up by clicking \"Clear All Outputs\" in the toolbar. It's a good idea to clear outputs before you push code to a repo like GitHub, otherwise the files can be large and harder to read."
]
},
{
@@ -202,7 +206,7 @@
"source": [
"# Using markdown\n",
"\n",
"So what's going on with these areas with writing in them, like this one? Well, there's actually a different kind of cell called a 'Markdown' cell for adding explanations like this. Click the + button to add a cell. Then in the toolbar, click where it says 'Code' and change it to 'Markdown'.\n",
"So what's going on with these areas with writing in them, like this one? Well, there's actually a different kind of cell called a 'Markdown' cell for adding explanations like this. Click the [+ Markdown] button to add a new markdown cell.\n",
"\n",
"Add some comments using Markdown format, perhaps copying and pasting from here:\n",
"\n",
@@ -238,7 +242,8 @@
"\n",
"There's a super useful feature of jupyter labs; you can type a command with a ! in front of it in a code cell, like:\n",
"\n",
"!pip install \\[some_package\\]\n",
"!ls \n",
"!pwd\n",
"\n",
"And it will run it at the command line (as if in Windows Powershell or Mac Terminal) and print the result"
]
@@ -262,25 +267,11 @@
"metadata": {},
"outputs": [],
"source": [
"# ping cnn.com - press the stop button in the toolbar when you're bored\n",
"# ping cnn.com - press the stop / interrupt button in the toolbar when you're bored\n",
"\n",
"!ping cnn.com"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a58e9462-89a2-4b4f-b4aa-51c4bd9f796b",
"metadata": {},
"outputs": [],
"source": [
"# This is a useful command that ensures your Anaconda environment \n",
"# is up to date with any new upgrades to packages;\n",
"# But it might take a minute and will print a lot to output\n",
"\n",
"!conda env update -f ../environment.yml"
]
},
{
"cell_type": "markdown",
"id": "4688baaf-a72c-41b5-90b6-474cb24790a7",
@@ -288,14 +279,14 @@
"source": [
"# Minor things we encounter on the course\n",
"\n",
"This isn't necessarily a feature of Jupyter, but it's a nice package to know about that is useful in Jupyter Labs, and I use it in the course.\n",
"This isn't necessarily a feature of notebooks, but it's a nice package to know about that is useful in notebooks.\n",
"\n",
"The package `tqdm` will print a nice progress bar if you wrap any iterable."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 12,
"id": "2646a4e5-3c23-4aee-a34d-d623815187d2",
"metadata": {},
"outputs": [],
@@ -348,17 +339,23 @@
"id": "9d14c1fb-3321-4387-b6ca-9af27676f980",
"metadata": {},
"source": [
"# That's it! You're up to speed on Jupyter Lab.\n",
"# That's it! You're up to speed on Notebooks / Labs in Cursor.\n",
"\n",
"## Want to be even more advanced?\n",
"\n",
"If you want to become a pro at Jupyter Lab, you can read their tutorial [here](https://jupyterlab.readthedocs.io/en/latest/). But this isn't required for our course; just a good technique for hitting Shift + Return and enjoying the result!"
"If you want to become a pro at Jupyter Lab (the technology behind this), you can read their tutorial [here](https://jupyterlab.readthedocs.io/en/latest/). But this isn't required for our course; just a good technique for hitting Shift + Return and enjoying the result!"
]
},
{
"cell_type": "markdown",
"id": "db6e47b6",
"metadata": {},
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": ".venv",
"language": "python",
"name": "python3"
},
@@ -372,7 +369,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.11"
"version": "3.12.9"
}
},
"nbformat": 4,

97
guides/06_python_foundations.ipynb Normal file
View File

@@ -0,0 +1,97 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "5c291475-8c7c-461c-9b12-545a887b2432",
"metadata": {},
"source": [
"# Foundations of Python and fixing NameErrors\n",
"\n",
"## First, for a complete beginner\n",
"\n",
"This introductory tutorial gets you started with the basics.\n",
"\n",
"https://chatgpt.com/share/68063082-c2d8-8012-8d45-fa674aa1c1ed\n"
]
},
{
"cell_type": "markdown",
"id": "542f0577-a826-4613-a5d7-4170e9666d04",
"metadata": {},
"source": [
"## Next: a comprehensive briefing on Python\n",
"\n",
"I'm going to defer to an AI friend for this, because these explanations are so well written with great examples. Copy and paste the code examples into a new cell to give them a try. Pick whichever section(s) you'd like to brush up on.\n",
"\n",
"**Python imports:** \n",
"https://chatgpt.com/share/672f9f31-8114-8012-be09-29ef0d0140fb\n",
"\n",
"**Python functions** including default arguments: \n",
"https://chatgpt.com/share/672f9f99-7060-8012-bfec-46d4cf77d672\n",
"\n",
"**Python strings**, including slicing, split/join, replace and literals: \n",
"https://chatgpt.com/share/672fb526-0aa0-8012-9e00-ad1687c04518\n",
"\n",
"**Python f-strings** including number and date formatting: \n",
"https://chatgpt.com/share/672fa125-0de0-8012-8e35-27918cbb481c\n",
"\n",
"**Python lists, dicts and sets**, including the `get()` method: \n",
"https://chatgpt.com/share/672fa225-3f04-8012-91af-f9c95287da8d\n",
"\n",
"**Python files** including modes, encoding, context managers, Path, glob.glob: \n",
"https://chatgpt.com/share/673b53b2-6d5c-8012-a344-221056c2f960\n",
"\n",
"**Python classes:** \n",
"https://chatgpt.com/share/672fa07a-1014-8012-b2ea-6dc679552715\n",
"\n",
"**Pickling Python objects and converting to JSON:** \n",
"https://chatgpt.com/share/673b553e-9d0c-8012-9919-f3bb5aa23e31"
]
},
{
"cell_type": "markdown",
"id": "f9e0f8e1-09b3-478b-ada7-c8c35003929b",
"metadata": {},
"source": [
"## With this in mind - understanding NameErrors in Python\n",
"\n",
"It's quite common to hit a NameError in python. With foundational knowledge, you should always feel equipped to debug a NameError and get to the bottom of it.\n",
"\n",
"The most common reason if you're working in a Notebook is that you haven't executed all the cells in order, so the Kernel does not have something defined.\n",
"\n",
"If you're unsure how to fix a NameError, please see this [initial guide](https://chatgpt.com/share/67958312-ada0-8012-a1d3-62b3a5fcbbfc) and this [second guide with exercises](https://chatgpt.com/share/67a57e0b-0194-8012-bb50-8ea76c5995b8), and work through them both until you have high confidence.\n",
"\n",
"There's some repetition here, so feel free to skip it if you're already confident.\n",
"\n",
"After this, a NameError should never give you problems again!"
]
},
{
"cell_type": "markdown",
"id": "f465c23b",
"metadata": {},
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

65
guides/07_vibe_coding_and_debugging.ipynb Normal file
View File

@@ -0,0 +1,65 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Vibe coding and debugging\n",
"\n",
"\"Vibe coding\" is the affectionate term for coding with the assistance of LLMs. It's particularly easy and wonderful to do with Cursor! But there are some good techniques to make it work well, which I cover on the course. Here are a few essential tips:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. If you're prompting an LLM like ChatGPT or Claude to write code, include the current date in your prompt, and instruct the LLM to generate code that uses the latest versions of APIs. This is particularly important with the OpenAI API! ChatGPT frequently tries to use outdated versions of its own API...\n",
"\n",
"2. Also in your prompts, ask LLMs to keep their code as short and simple as possible. LLMs seem to love to add lots of extra, unnecessary checks that clutters code and makes it hard to find problems.\n",
"\n",
"3. Ask the same question to multiple LLMs, and pick the answer that is clearest and simplest.\n",
"\n",
"4. Similar: take the answer from 1 LLM, and ask another LLM to verify it for correctness and whether it could be simpler or clearer.\n",
"\n",
"5. Ask an LLM to give you 3 variations for ways to solve a problem, and pick the one that is clearest."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Vibe Coding for a larger project"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Try to avoid having an LLM generate 100-200 lines of code or more; it will be so hard for you to debug and figure out what's going wrong (unless you're already knowledgable).\n",
"\n",
"Instead: start by breaking down your problem into small, independently testable steps, that are each relatively small. If you're not sure how to break down your problem - this is something you can ask the LLM to do!\n",
"\n",
"Then for each of these building blocks:\n",
"- Use the tips above to have the LLM build the code\n",
"- Also have the LLM write tests to test and verify the code\n",
"- Test it yourself and satisfy yourself that it's working correctly\n",
"\n",
"This will allow you to build a larger project with confidence.\n",
"\n",
"## The golden rule: start small, work incrementally!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

72
guides/08_debugging.ipynb Normal file
View File

@@ -0,0 +1,72 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Debugging Survival Guide\n",
"\n",
"## The key is to consistently reproduce your problem in 5-10 lines of code\n",
"\n",
"Hitting an exception can sometimes feel quite frustrating, particularly if you're not sure how to approach it. But this is where the best learning happens! Getting to the bottom of hard problems is a great way to learn, and can be quite satisfying.\n",
"\n",
"But I'm here to help if you can't!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Preliminaries\n",
"\n",
"Here is a briefing on Exceptions, Stack Traces, Exception Handling and more:\n",
"\n",
"https://chatgpt.com/share/6806383c-ab38-8012-b21f-61af665bb900\n",
"\n",
"Also: see [this guide](https://chatgpt.com/share/681f691b-6644-8012-b07d-207c68f259d5) if you're unsure how to take screenshots."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Formula for debugging: recreate in 10 lines of code\n",
"\n",
"There is an art and a science to great debugging. The best way to pick it up is by practice! But here are the essentials:\n",
"\n",
"1. Recreate \n",
"Reproduce your problem, consistently\n",
"\n",
"2. Simplify \n",
"Reduce your problem down to the simplest possible way to reproduce it. I can't tell you how many people have sent me 100+ lines of code and said \"Ed, this isn't working, please fix it\". That's not how it works! The first thing I would try is to reduce this to the core problem - ideally in 10-20 lines of code or less. In one case, I showed a student how to reproduce it with 2 lines of code!\n",
"\n",
"_The thing is, it's much easier for **you** to do this than anybody else._ So while it can be tedious, this is almost always the best way to start. Reduce down to a few lines of code. And by the way, when you do that, you often figure out the problem yourself anyway!\n",
"\n",
"3. Get help \n",
"Once you've recreated the problem with just a few lines of code, if you can't see what's happening - you should get help! It's worth asking ChatGPT and friends of course; give them the short example and the full stack trace.\n",
"\n",
"And reach out to me! I'm here to make your learning experience as awesome as possible and if you are stuck, I will help you get unstuck.\n",
"\n",
"### The best way to work with me for fastest fixes...\n",
"\n",
"1. Simplify the problem as much as possible, with an easy way to reproduce\n",
"2. Include the full stack trace, and a screenshot not a photo (see [this guide](https://chatgpt.com/share/681f691b-6644-8012-b07d-207c68f259d5) if you're unsure how to take screenshots)\n",
"3. Email me at ed@edwarddonner.com. Or best of all: if you've used Google Colab before, this is a really terrific way to share an issue, because it will be identical for me to reproduce, and I can fix it and share the fix with you directly.\n",
"\n",
"I look forward to helping you!\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

466
guides/09_ai_apis_and_ollama.ipynb Normal file
View File

@@ -0,0 +1,466 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# LLM APIs and Ollama - beyond OpenAI\n",
"\n",
"_IMPORTANT: If you're not as familiar with APIs in general, and with Environment Variables on your PC or Mac, please review the APIs section in Guide 4 Technical Foundations before proceeding with this guide (topics 3 and 5 in Guide 4)._\n",
"\n",
"## Crucial context for using models other than OpenAI - please read this first!\n",
"\n",
"Throughout the course, we use APIs for connecting with the strongest LLMs on the planet.\n",
"\n",
"The companies behind these LLMs, such as OpenAI, Anthropic, Google and DeepSeek, have built web endpoints. You call their models by making an HTTP request to a Web Address and passing in all the information about your prompts.\n",
"\n",
"But it would be painful if we needed to build HTTP requests every time we wanted to call an API.\n",
"\n",
"To make this simple, the team at OpenAI wrote a python utility known as a \"Python Client Library\" which wraps the HTTP call. So you write python code and it calls the web.\n",
"\n",
"And THAT is what the library `openai` is.\n",
"\n",
"### What is the `openai` python client library\n",
"\n",
"It is:\n",
"- A lightweight python utility\n",
"- Turns your python requests into an HTTP call\n",
"- Converts the results coming back from the HTTP call into python objects\n",
"\n",
"### What it is NOT\n",
"\n",
"- It's not got any code to actually run a Large Language Model! No GPT code! It just makes a web request\n",
"- There's no scientific computing code, and nothing particularly specialized for OpenAI\n",
"\n",
"### How to use it:\n",
"\n",
"```python\n",
"# Create an OpenAI python client for making web calls to OpenAI\n",
"openai = OpenAI()\n",
"\n",
"# Make the call\n",
"response = openai.chat.completions.create(model=\"gpt-4.1-mini\", messages=[{\"role\":\"user\", \"content\": \"what is 2+2?\"}])\n",
"\n",
"# Print the result\n",
"print(response.choices[0].message.content)\n",
"```\n",
"\n",
"### What does this do\n",
"\n",
"When you make the python call: `openai.chat.completions.create()` \n",
"It simply makes a web request to this url: `https://api.openai.com/v1/chat/completions` \n",
"And it converts the response to python objects.\n",
"\n",
"That's it.\n",
"\n",
"Here's the API documentation if you make [direct web HTTP calls](https://platform.openai.com/docs/guides/text?api-mode=chat&lang=curl) \n",
"And here's the same API documentation if you use the [Python Client Library](https://platform.openai.com/docs/guides/text?api-mode=chat&lang=python)\n",
"\n",
"## With that context - how do I use other LLMs?\n",
"\n",
"It turns out - it's super easy!\n",
"\n",
"All the other major LLMs have API endpoints that are compatible with OpenAI.\n",
"\n",
"And so OpenAI did everyone a favor: they said, hey look - you can all use our utility for converting python to web requests. We'll allow you to change the utility from calling `https://api.openai/com/v1` to calling any web address that you specify.\n",
"\n",
"And so you can use the OpenAI utility even for calling models that are NOT OpenAI, like this:\n",
"\n",
"`not_actually_openai = OpenAI(base_url=\"https://somewhere.completely.different/\", api_key=\"another_providers_key\")`\n",
"\n",
"It's important to appreciate that this OpenAI code is just a utility for making HTTP calls to endpoints. So even though we're using code from the OpenAI team, we can use it to call models other than OpenAI.\n",
"\n",
"Here are all the OpenAI-compatible endpoints from the major providers. It even includes using Ollama, locally. Ollama provides an endpoint on your local machine, and they made it OpenAI compatible too - very convenient.\n",
"\n",
"```python\n",
"ANTHROPIC_BASE_URL = \"https://api.anthropic.com/v1/\"\n",
"DEEPSEEK_BASE_URL = \"https://api.deepseek.com/v1\"\n",
"GEMINI_BASE_URL = \"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
"GROK_BASE_URL = \"https://api.x.ai/v1\"\n",
"GROQ_BASE_URL = \"https://api.groq.com/openai/v1\"\n",
"OPENROUTER_BASE_URL = \"https://openrouter.ai/api/v1\"\n",
"OLLAMA_BASE_URL = \"http://localhost:11434/v1\"\n",
"```\n",
"\n",
"## Here are examples for Gemini, DeepSeek, Ollama and OpenRouter\n",
"\n",
"### Example 1: Using Gemini instead of OpenAI\n",
"\n",
"1. Visit Google Studio to set up an account: https://aistudio.google.com/ \n",
"2. Add your key as GOOGLE_API_KEY to your `.env` \n",
"3. Also add it a second time as GEMINI_API_KEY to your `.env` - this will be helpful later.\n",
"\n",
"Then:\n",
"\n",
"```python\n",
"import os\n",
"from openai import OpenAI\n",
"from dotenv import load_dotenv\n",
"load_dotenv(override=True)\n",
"\n",
"GEMINI_BASE_URL = \"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
"google_api_key = os.getenv(\"GOOGLE_API_KEY\")\n",
"gemini = OpenAI(base_url=GEMINI_BASE_URL, api_key=google_api_key)\n",
"response = gemini.chat.completions.create(model=\"gemini-2.5-flash-preview-05-20\", messages=[{\"role\":\"user\", \"content\": \"what is 2+2?\"}])\n",
"print(response.choices[0].message.content)\n",
"```\n",
"\n",
"### Example 2: Using DeepSeek API instead of OpenAI (cheap, and only $2 upfront)\n",
"\n",
"1. Visit DeepSeek API to set up an account: https://platform.deepseek.com/ \n",
"2. You will need to add an initial $2 minimum balance. \n",
"3. Add your key as DEEPSEEK_API_KEY to your `.env` \n",
"\n",
"Then:\n",
"\n",
"```python\n",
"import os\n",
"from openai import OpenAI\n",
"from dotenv import load_dotenv\n",
"load_dotenv(override=True)\n",
"\n",
"DEEPSEEK_BASE_URL = \"https://api.deepseek.com/v1\"\n",
"deepseek_api_key = os.getenv(\"DEEPSEEK_API_KEY\")\n",
"deepseek = OpenAI(base_url=DEEPSEEK_BASE_URL, api_key=deepseek_api_key)\n",
"response = deepseek.chat.completions.create(model=\"deepseek-chat\", messages=[{\"role\":\"user\", \"content\": \"what is 2+2?\"}])\n",
"print(response.choices[0].message.content)\n",
"```\n",
"\n",
"### Example 3: Using Ollama to be free and local instead of OpenAI\n",
"\n",
"Ollama allows you to run models locally; it provides an OpenAI compatible API on your machine. \n",
"There's no API key for Ollama; there's no third party with your credit card, so no need for any kind of key.\n",
"\n",
"1. If you're new to Ollama, install it by following the instructions here: https://ollama.com \n",
"2. Then in a Cursor Terminal, do `ollama run llama3.2` to chat with Llama 3.2 \n",
"BEWARE: do not use llama3.3 or llama4 - these are massive models not designed for home computing! They will fill up your disk. \n",
"\n",
"Then:\n",
"\n",
"```python\n",
"!ollama pull llama3.2\n",
"\n",
"from openai import OpenAI\n",
"\n",
"OLLAMA_BASE_URL = \"http://localhost:11434/v1\"\n",
"ollama = OpenAI(base_url=OLLAMA_BASE_URL, api_key=\"anything\")\n",
"response = ollama.chat.completions.create(model=\"llama3.2\", messages=[{\"role\":\"user\", \"content\": \"what is 2+2?\"}])\n",
"print(response.choices[0].message.content)\n",
"```\n",
"\n",
"### Example 4: Using the popular service [OpenRouter](https://openrouter.ai) which has an easier billing process instead of OpenAI\n",
"\n",
"OpenRouter is very convenient: it gives you free access to many models, and easy access with small upfront to paid models.\n",
"\n",
"1. Sign up at https://openrouter.ai\n",
"2. Add the minimum upfront balance as needed\n",
"3. Add your key as OPENROUTER_API_KEY to your `.env` file\n",
"\n",
"Then:\n",
"\n",
"```python\n",
"import os\n",
"from openai import OpenAI\n",
"from dotenv import load_dotenv\n",
"load_dotenv(override=True)\n",
"\n",
"OPENROUTER_BASE_URL = \"https://openrouter.ai/api/v1\"\n",
"openrouter_api_key = os.getenv(\"OPENROUTER_API_KEY\")\n",
"openrouter = OpenAI(base_url=OPENROUTER_BASE_URL, api_key=openrouter_api_key)\n",
"response = openrouter.chat.completions.create(model=\"openai/gpt-4.1-nano\", messages=[{\"role\":\"user\", \"content\": \"what is 2+2?\"}])\n",
"print(response.choices[0].message.content)\n",
"```\n",
"\n",
"\n",
"### Using different API providers with Agent Frameworks\n",
"\n",
"The Agent Frameworks make it easy to switch between these providers. You can switch LLMs and pick different ones at any point in the course. There are more notes below on each of them. For OpenAI Agents SDK, see a section later in this notebook. For CrewAI, we cover it on the course, but it's easy: just use the full path to the model that LiteLLM expects."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Costs of APIs\n",
"\n",
"The cost of each API call is very low indeed - most calls to models we use on this course are fractions of cents.\n",
"\n",
"But it's extremely important to note:\n",
"\n",
"1. A complex Agentic project could involve many LLM calls - perhaps 20-30 - and so it can add up. It's important to set limits and monitor usage.\n",
"\n",
"2. With Agentic AI, there is a risk of Agents getting into a loop or carrying out more processing than intended. You should monitor your API usage, and never put more budget than you are comfortable with. Some APIs have an \"auto-refill\" setting that can charge automatically to your card - I strongly recommend you keep this off.\n",
"\n",
"3. You should only spend what you are comfortable with. There is a free alternative in Ollama that you can use as a replacement if you wish. DeepSeek, Gemini 2.5 Flash and gpt-4.1-nano are significantly cheaper.\n",
"\n",
"Keep in mind that these LLM calls typically involve trillions of floating point calculations - someone has to pay the electricity bills!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Ollama: Free alternative to Paid APIs (but please see Warning about llama version)\n",
"\n",
"Ollama is a product that runs locally on your machine. It can run open-source models, and it provides an API endpoint on your computer that is compatible with OpenAI.\n",
"\n",
"First, download Ollama by visiting:\n",
"https://ollama.com\n",
"\n",
"Then from your Terminal in Cursor (View menu >> Terminal), run this command to download a model:\n",
"\n",
"```shell\n",
"ollama pull llama3.2\n",
"```\n",
"\n",
"WARNING: Be careful not to use llama3.3 or llama4 - these are much larger models that are not suitable for home computers.\n",
"\n",
"And now, any time that we have code like: \n",
"`openai = OpenAI()` \n",
"You can use this as a direct replacement: \n",
"`openai = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')` \n",
"And also replace model names like **gpt-4o-mini** with **llama3.2**. \n",
"\n",
"You don't need to put anything in your .env file for this; with Ollama, everything is running on your computer. You're not calling out to a third party on the cloud, nobody has your credit card details, so there's no need for a secret key! The code `api_key='ollama'` above is only required because the OpenAI client library expects an api_key to be passed in, but the value is ignored by Ollama.\n",
"\n",
"Below is a full example:\n",
"\n",
"```python\n",
"# You need to do this one time on your computer\n",
"!ollama pull llama3.2\n",
"\n",
"from openai import OpenAI\n",
"MODEL = \"llama3.2\"\n",
"openai = OpenAI(base_url=\"http://localhost:11434/v1\", api_key=\"ollama\")\n",
"\n",
"response = openai.chat.completions.create(\n",
" model=MODEL,\n",
" messages=[{\"role\": \"user\", \"content\": \"What is 2 + 2?\"}]\n",
")\n",
"\n",
"print(response.choices[0].message.content)\n",
"```\n",
"\n",
"You will need to make similar changes to use Ollama within any of the Agent Frameworks - you should be able to google for an exact example, or ask me."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### OpenRouter: Convenient gateway platform for OpenAI and others\n",
"\n",
"OpenRouter is a third party service that allows you to connect to a wide range of LLMs, including OpenAI.\n",
"\n",
"It's known for having a simpler billing process that may be easier for some countries outside the US.\n",
"\n",
"First, check out their website: \n",
"https://openrouter.ai/\n",
"\n",
"Then, take a peak at their quickstart: \n",
"https://openrouter.ai/docs/quickstart\n",
"\n",
"And add your key to your .env file: \n",
"```shell\n",
"OPENROUTER_API_KEY=sk-or....\n",
"```\n",
"\n",
"And now, any time you have code like this: \n",
"```python\n",
"MODEL = \"gpt-4o-mini\"\n",
"openai = OpenAI()\n",
"```\n",
"\n",
"You can replace it with code like this:\n",
"\n",
"```python\n",
"MODEL = \"openai/gpt-4o-mini\"\n",
"openrouter_api_key = os.getenv(\"OPENROUTER_API_KEY\")\n",
"openai = OpenAI(base_url=\"https://openrouter.ai/api/v1\", api_key=openrouter_api_key)\n",
"\n",
"response = openai.chat.completions.create(\n",
" model=MODEL,\n",
" messages=[{\"role\": \"user\", \"content\": \"What is 2 + 2?\"}]\n",
")\n",
"\n",
"print(response.choices[0].message.content)\n",
"```\n",
"\n",
"You will need to make similar changes to use OpenRouter within any of the Agent Frameworks - you should be able to google for an exact example, or ask me."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## OpenAI Agents SDK - specific instructions\n",
"\n",
"With OpenAI Agents SDK (weeks 2 and 6), it's particularly easy to use any model provided by OpenAI themselves. Simply pass in the model name:\n",
"\n",
"`agent = Agent(name=\"Jokester\", instructions=\"You are a joke teller\", model=\"gpt-4o-mini\")`\n",
"\n",
"You can also substitute in any other provider with an OpenAI compatible API. You do it in 3 steps like this:\n",
"\n",
"```python\n",
"DEEPSEEK_BASE_URL = \"https://api.deepseek.com/v1\"\n",
"deepseek_client = AsyncOpenAI(base_url=DEEPSEEK_BASE_URL, api_key=deepseek_api_key)\n",
"deepseek_model = OpenAIChatCompletionsModel(model=\"deepseek-chat\", openai_client=deepseek_client)\n",
"```\n",
"\n",
"And then you simply provide this model when you create an Agent.\n",
"\n",
"`agent = Agent(name=\"Jokester\", instructions=\"You are a joke teller\", model=deepseek_model)`\n",
"\n",
"And you can use a similar approach for any other OpenAI compatible API, with the same 3 steps:\n",
"\n",
"```python\n",
"# extra imports\n",
"from agents import OpenAIChatCompletionsModel\n",
"from openai import AsyncOpenAI\n",
"\n",
"# Step 1: specify the base URL endpoints where the provider offers an OpenAI compatible API\n",
"GEMINI_BASE_URL = \"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
"GROK_BASE_URL = \"https://api.x.ai/v1\"\n",
"GROQ_BASE_URL = \"https://api.groq.com/openai/v1\"\n",
"OPENROUTER_BASE_URL = \"https://openrouter.ai/api/v1\"\n",
"OLLAMA_BASE_URL = \"http://localhost:11434/v1\"\n",
"\n",
"# Step 2: Create an AsyncOpenAI object for that endpoint\n",
"gemini_client = AsyncOpenAI(base_url=GEMINI_BASE_URL, api_key=google_api_key)\n",
"grok_client = AsyncOpenAI(base_url=GROK_BASE_URL, api_key=grok_api_key)\n",
"groq_client = AsyncOpenAI(base_url=GROQ_BASE_URL, api_key=groq_api_key)\n",
"openrouter_client = AsyncOpenAI(base_url=OPENROUTER_BASE_URL, api_key=openrouter_api_key)\n",
"ollama_client = AsyncOpenAI(base_url=OLLAMA_BASE_URL, api_key=\"ollama\")\n",
"\n",
"# Step 3: Create a model object to provide when creating an Agent\n",
"gemini_model = OpenAIChatCompletionsModel(model=\"gemini-2.5-flash\", openai_client=gemini_client)\n",
"grok_3_model = OpenAIChatCompletionsModel(model=\"grok-3-mini-beta\", openai_client=openrouter_client)\n",
"llama3_3_model = OpenAIChatCompletionsModel(model=\"llama-3.3-70b-versatile\", openai_client=groq_client)\n",
"grok_3_via_openrouter_model = OpenAIChatCompletionsModel(model=\"x-ai/grok-3-mini-beta\", openai_client=openrouter_client)\n",
"llama_3_2_local_model = OpenAIChatCompletionsModel(model=\"llama3.2\", openai_client=ollama_client)\n",
"```\n",
"\n",
"### To use Azure with OpenAI Agents SDK\n",
"\n",
"See instructions here: \n",
"https://techcommunity.microsoft.com/blog/azure-ai-services-blog/use-azure-openai-and-apim-with-the-openai-agents-sdk/4392537\n",
"\n",
"Such as this:\n",
"```python\n",
"from openai import AsyncAzureOpenAI\n",
"from agents import set_default_openai_client\n",
"from dotenv import load_dotenv\n",
"import os\n",
" \n",
"# Load environment variables\n",
"load_dotenv()\n",
" \n",
"# Create OpenAI client using Azure OpenAI\n",
"openai_client = AsyncAzureOpenAI(\n",
" api_key=os.getenv(\"AZURE_OPENAI_API_KEY\"),\n",
" api_version=os.getenv(\"AZURE_OPENAI_API_VERSION\"),\n",
" azure_endpoint=os.getenv(\"AZURE_OPENAI_ENDPOINT\"),\n",
" azure_deployment=os.getenv(\"AZURE_OPENAI_DEPLOYMENT\")\n",
")\n",
" \n",
"# Set the default OpenAI client for the Agents SDK\n",
"set_default_openai_client(openai_client)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## CrewAI setup\n",
"\n",
"Here's Crew's docs for LLM connections with the model names to use for all models. As student Sadan S. pointed out (thank you!), it's worth knowing that for Google you need to use the environment variable `GEMINI_API_KEY` instead of `GOOGLE_API_KEY`:\n",
"\n",
"https://docs.crewai.com/concepts/llms\n",
"\n",
"And here's their tutorial with some more info:\n",
"\n",
"https://docs.crewai.com/how-to/llm-connections"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## LangGraph setup\n",
"\n",
"To use LangGraph with Ollama (and follow similar for other models): \n",
"https://python.langchain.com/docs/integrations/chat/ollama/#installation\n",
"\n",
"First add the package: \n",
"`uv add langchain-ollama`\n",
"\n",
"Then in the lab, make this replacement: \n",
"```python\n",
"from langchain_ollama import ChatOllama\n",
"# llm = ChatOpenAI(model=\"gpt-4o-mini\")\n",
"llm = ChatOllama(model=\"gemma3:4b\")\n",
"```\n",
"\n",
"And obviously run `!ollama pull gemma3:4b` (or whichever model) beforehand.\n",
"\n",
"Many thanks to Miroslav P. for adding this, and to Arvin F. for the question!\n",
"\n",
"## LangGraph with other models\n",
"\n",
"Just follow the same recipe as above, but use any of the models from here: \n",
"https://python.langchain.com/docs/integrations/chat/\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## AutoGen with other models\n",
"\n",
"Here's another contribution from Miroslav P. (thank you!) for using Ollama + local models with AutoGen, and Miroslav has a great example showing gemma3 performing well.\n",
"\n",
"```python\n",
"# model_client = OpenAIChatCompletionClient(model=\"gpt-4o-mini\")\n",
" \n",
"from autogen_ext.models.ollama import OllamaChatCompletionClient\n",
" \n",
"model_client = OllamaChatCompletionClient(\n",
" model=\"gemma3:4b\",\n",
" model_info={\n",
" \"vision\": True,\n",
" \"function_calling\": False,\n",
" \"json_output\": True,\n",
" \"family\": \"unknown\",\n",
" },\n",
")\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Worth keeping in mind\n",
"\n",
"1. If you wish to use Ollama to run models locally, you may find that smaller models struggle with the more advanced projects. You'll need to experiment with different model sizes and capabilities, and plenty of patience may be needed to find something that works well. I expect several of our projects are too challenging for llama3.2. As an alternative, consider the free models on openrouter.ai, or the very cheap models that are almost free - like DeepSeek.\n",
"\n",
"2. Chat models often do better than Reasoning models because Reasoning models can \"over-think\" some assignments. It's important to experiment. Bigger isn't always better...\n",
"\n",
"3. It's confusing, but there are 2 different providers that sound similar! \n",
"- Grok is the LLM from Elon Musk's X\n",
"- Groq is a platform for fast inference of open source models\n",
"\n",
"A student pointed out to me that \"Groq\" came first!\n"
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

114
week1/Intermediate Python.ipynb → guides/10_intermediate_python.ipynb
View File

@@ -7,63 +7,15 @@
"source": [
"# Intermediate Level Python\n",
"\n",
"## Getting you up to speed\n",
"## A briefing on more advanced features of Python\n",
"\n",
"This course assumes that you're at an intermediate level of python. For example, you should have a decent idea what something like this might do:\n",
"This section assumes you're up to speed on the foundations - and now we cover some important features of python that we use on the course.\n",
"\n",
"`yield from {book.get(\"author\") for book in books if book.get(\"author\")}`\n",
"\n",
"If not - then you've come to the right place! Welcome to the crash course in intermediate level python. The best way to learn is by doing!\n"
]
},
{
"cell_type": "markdown",
"id": "542f0577-a826-4613-a5d7-4170e9666d04",
"metadata": {},
"source": [
"## First: if you need a refresher on the foundations\n",
"\n",
"I'm going to defer to an AI friend for this, because these explanations are so well written with great examples. Copy and paste the code examples into a new cell to give them a try. Pick whichever section(s) you'd like to brush up on.\n",
"\n",
"**Python imports:** \n",
"https://chatgpt.com/share/672f9f31-8114-8012-be09-29ef0d0140fb\n",
"\n",
"**Python functions** including default arguments: \n",
"https://chatgpt.com/share/672f9f99-7060-8012-bfec-46d4cf77d672\n",
"\n",
"**Python strings**, including slicing, split/join, replace and literals: \n",
"https://chatgpt.com/share/672fb526-0aa0-8012-9e00-ad1687c04518\n",
"\n",
"**Python f-strings** including number and date formatting: \n",
"https://chatgpt.com/share/672fa125-0de0-8012-8e35-27918cbb481c\n",
"\n",
"**Python lists, dicts and sets**, including the `get()` method: \n",
"https://chatgpt.com/share/672fa225-3f04-8012-91af-f9c95287da8d\n",
"\n",
"**Python files** including modes, encoding, context managers, Path, glob.glob: \n",
"https://chatgpt.com/share/673b53b2-6d5c-8012-a344-221056c2f960\n",
"\n",
"**Python classes:** \n",
"https://chatgpt.com/share/672fa07a-1014-8012-b2ea-6dc679552715\n",
"\n",
"**Pickling Python objects and converting to JSON:** \n",
"https://chatgpt.com/share/673b553e-9d0c-8012-9919-f3bb5aa23e31"
]
},
{
"cell_type": "markdown",
"id": "f9e0f8e1-09b3-478b-ada7-c8c35003929b",
"metadata": {},
"source": [
"## With this in mind - understanding NameErrors in Python\n",
"\n",
"It's quite common to hit a NameError in python. With foundational knowledge, you should always feel equipped to debug a NameError and get to the bottom of it.\n",
"\n",
"If you're unsure how to fix a NameError, please see this [initial guide](https://chatgpt.com/share/67958312-ada0-8012-a1d3-62b3a5fcbbfc) and this [second guide with exercises](https://chatgpt.com/share/67a57e0b-0194-8012-bb50-8ea76c5995b8), and work through them both until you have high confidence.\n",
"\n",
"There's some repetition here, so feel free to skip it if you're already confident.\n",
"\n",
"## And now, on to the code!"
"1. Comprehensions \n",
"2. Generators \n",
"3. Sub-classes, Type Hints, Pydantic \n",
"4. Decorators\n",
"5. Docker (not really python, but we use it to run python code!)\n"
]
},
{
@@ -454,17 +406,61 @@
"id": "35760406-fe6c-41f9-b0c0-3e8cf73aafd0",
"metadata": {},
"source": [
"# Finally\n",
"# Part 3: Sub-classes, Type Hints, Pydantic\n",
"\n",
"Here are some intermediate level details of Classes from our AI friend, including use of type hints, inheritance and class methods. This includes a Book example.\n",
"\n",
"https://chatgpt.com/share/67348aca-65fc-8012-a4a9-fd1b8f04ba59"
"https://chatgpt.com/share/67348aca-65fc-8012-a4a9-fd1b8f04ba59\n",
"\n",
"And here is a comprehensive tutorial on Pydantic classes covering everything you need to know about Pydantic.\n",
"\n",
"https://chatgpt.com/share/68064537-6cfc-8012-93e1-f7dd0932f321"
]
},
{
"cell_type": "markdown",
"id": "6bbc9c63",
"metadata": {},
"source": [
"## Part 4: Decorators\n",
"\n",
"Here is a briefing, with an example from OpenAI Agents SDK:\n",
"\n",
"https://chatgpt.com/share/6806474d-3880-8012-b2a2-87b3ee4489da"
]
},
{
"cell_type": "markdown",
"id": "0beef7e9",
"metadata": {},
"source": [
"## Part 5: Docker\n",
"\n",
"Here is a convenient tutorial to introduce Docker.\n",
"\n",
"In the last section, this also covers an answer to a question in Week 6 - what does it mean to run an MCP server in Docker? But you can ignore this question if you're not on week 6 yet.\n",
"\n",
"https://chatgpt.com/share/6814bc1d-2f3c-8012-9b18-dddc82ea421b"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "73e215b2",
"metadata": {},
"outputs": [],
"source": [
"# You need to install docker to run this example\n",
"# This will download the Docker image for python 3.12, create a container,\n",
"# Run some python code and print the result\n",
"\n",
"!docker run --rm python:3.12 python -c \"print(2 + 2)\""
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": ".venv",
"language": "python",
"name": "python3"
},
@@ -478,7 +474,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.11"
"version": "3.12.9"
}
},
"nbformat": 4,

156
guides/11_async_python.ipynb Normal file
View File

@@ -0,0 +1,156 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "5c291475-8c7c-461c-9b12-545a887b2432",
"metadata": {},
"source": [
"# Async Python\n",
"\n",
"## A briefing on asynchronous python coding, essential in Agent engineering"
]
},
{
"cell_type": "markdown",
"id": "538fa044",
"metadata": {},
"source": [
"Here is a masterful tutorial by you-know-who with exercises and comparisons.\n",
"\n",
"https://chatgpt.com/share/680648b1-b0a0-8012-8449-4f90b540886c\n",
"\n",
"This includes how to run async code from a python module.\n",
"\n",
"### And now some examples:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "09f5662a",
"metadata": {},
"outputs": [],
"source": [
"# Let's define an async function\n",
"\n",
"import asyncio\n",
"\n",
"async def do_some_work():\n",
" print(\"Starting work\")\n",
" await asyncio.sleep(1)\n",
" print(\"Work complete\")\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "07ab3abf",
"metadata": {},
"outputs": [],
"source": [
"# What will this do?\n",
"\n",
"do_some_work()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6d681b6d",
"metadata": {},
"outputs": [],
"source": [
"# OK let's try that again!\n",
"\n",
"await do_some_work()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ea867090",
"metadata": {},
"outputs": [],
"source": [
"# What's wrong with this?\n",
"\n",
"async def do_a_lot_of_work():\n",
" do_some_work()\n",
" do_some_work()\n",
" do_some_work()\n",
"\n",
"await do_a_lot_of_work()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e9c75c3f",
"metadata": {},
"outputs": [],
"source": [
"# Interesting warning! Let's fix it\n",
"\n",
"async def do_a_lot_of_work():\n",
" await do_some_work()\n",
" await do_some_work()\n",
" await do_some_work()\n",
"\n",
"await do_a_lot_of_work()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "720cf3f5",
"metadata": {},
"outputs": [],
"source": [
"# And now let's do it in parallel\n",
"# It's important to recognize that this is not \"multi-threading\" in the way that you may be used to\n",
"# The asyncio library is running on a single thread, but it's using a loop to switch between tasks while one is waiting\n",
"\n",
"async def do_a_lot_of_work_in_parallel():\n",
" await asyncio.gather(do_some_work(), do_some_work(), do_some_work())\n",
"\n",
"await do_a_lot_of_work_in_parallel()"
]
},
{
"cell_type": "markdown",
"id": "230f85de",
"metadata": {},
"source": [
"### Finally - try writing a python module that calls do_a_lot_of_work_in_parallel\n",
"\n",
"See the link at the top; you'll need something like this in your module:\n",
"\n",
"```python\n",
"if __name__ == \"__main__\":\n",
" asyncio.run(do_a_lot_of_work_in_parallel())\n",
"```"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

97
guides/12_starting_your_project.ipynb Normal file
View File

@@ -0,0 +1,97 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## How to start working on your idea\n",
"\n",
"So you've had an idea for something you'd like to build.\n",
"\n",
"CONGRATULATIONS! That's the most important part already done. The idea is everything!\n",
"\n",
"But now you're unsure where to begin. I get a lot of questions about this.\n",
"\n",
"I want to give you three essential pieces of advice.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Advice 1: Be a Scientist\n",
"\n",
"Being an AI Engineer involves wearing 2 hats: being a Software Engineer, and a Data Scientist.\n",
"\n",
"At the start of your project, it's crucial to take off your Software Engineering hat and firmly put on your Data Science hat.\n",
"\n",
"A lot of people from a software engineering background (myself included!) struggle with this. It's common for people to fall into their comfort zone, asking questions about architecture, design, data pipes, scalability, deployment.\n",
"\n",
"These are all great questions, but they are not the critical ones at this early stage. They are not the hard questions. They won't make or break your idea.\n",
"\n",
"For the critical questions, you need to be a Scientist.\n",
"\n",
"Ask yourself: \n",
"1. How will you evaluate success of your model? What will you measure? \n",
"2. What data do you have, and what data do you need? \n",
"3. How can you build small prototypes to test different techniques and understand model performance?\n",
"\n",
"Address the science questions first - what you can achieve with LLMs and how.\n",
"\n",
"And I recommend starting in a Notebook before you work in python modules, to force yourself to operate with this scientific mindset.\n",
"\n",
"This is even more crucial with Agent projects. It can be tempting to draw up a big Agent architecture diagram on how your agents will collaborate, like a technical diagram. But this is thinking like a Software Engineer. I recommend approaching it differently; experiment with different approaches, investigate what works well, test hypotheses and iterate."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Advice 2: Embrace R&D\n",
"\n",
"OK so this is pretty similar to Advice 1, but worth repeating!\n",
"\n",
"I often get asked questions like: \"Ed, I want to build the following solution. Should I use model A, B, or C? Should I use Agents, RAG or fine-tuning?\"\n",
"\n",
"My answer is almost always: \n",
"- You should do all of the above! The key is to experiment; (a) come up with an evaluation criteria, (b) develop a curated dataset, (c) test different ideas and see how they perform\n",
"- I often have an instinct for which will work best, but my instinct is often wrong! Don't trust me: try it yourself \n",
"- There's simply no substitute for experimentation."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Advice 3: Dream big but start SMALL!\n",
"\n",
"I also often have students send me 200-300 lines of code saying, \"Ed, this isn't performing, please fix it\". 😂\n",
"\n",
"As I say in the debugging guide - that's not how it works!\n",
"\n",
"When starting projects, it's crucial to start small and simple. Perfect every prompt; work in detail on every step. Satisfy yourself that each LLM call is performing as you wish, and iterate on the inputs until responses are consistent and reliable.\n",
"\n",
"If you always work incrementally and with small, testable building blocks, you should have complete clarity on what's going on.\n",
"\n",
"And of course, I love helping people with projects and it's great to help fix problems. But if you approach your project incrementally, you should always have clarity on exactly what isn't performing - and then it's super efficient for me to help you."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# In summary\n",
"\n",
"You probably noticed the common theme through these 3 pieces of advice. It can feel jarring for people from a non-Data Science background to approach problems in such an ad-hoc way; it feels \"hacky\" and unsatisfying. But in my opinion, this is the single most important skill to acquire to be a successful AI engineer: being comfortable with uncertainty, enjoying the experiments, and embracing being a Scientist."
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

55
guides/13_frontend.ipynb Normal file
View File

@@ -0,0 +1,55 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "695d0c99",
"metadata": {},
"source": [
"# Your frontend crash course\n",
"\n",
"Here is a seven part series developed by you-know-who under my direction. It's very nicely done! You should take this in any direction you like.\n",
"\n",
"ChatGPT has written these in note form for someone who wants to experiment; you should ask it to rewrite in a more verbose way if you'd prefer deeper explanations.\n",
"\n",
"1. Refresher on modern HTML, CSS and JavaScript\n",
"\n",
"https://chatgpt.com/share/68c345ee-ae34-8012-9a05-0ce43a816f9c\n",
"\n",
"2. Modern JavaScript\n",
"\n",
"https://chatgpt.com/share/68c3495f-0c94-8012-b048-db1670aefa47\n",
"\n",
"3. From JavaScript to TypeScript\n",
"\n",
"https://chatgpt.com/share/68c34974-b768-8012-9926-5a9ed67d59c3\n",
"\n",
"4. React (TypeScript) basics\n",
"\n",
"https://chatgpt.com/share/68c3498d-e340-8012-b3a3-37a28802db6a\n",
"\n",
"5. React - state, props, hooks\n",
"\n",
"https://chatgpt.com/share/68c349c7-e75c-8012-b1e5-b4f51bb0e27c\n",
"\n",
"6. NextJS introduction\n",
"\n",
"https://chatgpt.com/share/68c349e7-e740-8012-8f0c-ba40ad7e148d\n",
"\n",
"7. NextJS rendering models and Tailwind\n",
"\n",
"https://chatgpt.com/share/68c34a13-3120-8012-9a67-f53d7a0cfc99\n",
"\n",
"\n",
"\n",
"\n"
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

33
guides/14_docker_terraform.ipynb Normal file
View File

@@ -0,0 +1,33 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "303a5c4f",
"metadata": {},
"source": [
"# Briefings on Docker and Terraform\n",
"\n",
"Here's a mutual friend giving an excellent self-study tutorial on Docker essentials:\n",
"\n",
"https://chatgpt.com/share/68c34f21-07bc-8012-b411-35cc97aff6ab\n",
"\n",
"And here is an equally excellent overview of Terraform to get you building infrastructure with code like a pro. No more AWS console!\n",
"\n",
"https://chatgpt.com/share/68c3513a-c194-8012-85c3-c0ea9143af5a"
]
},
{
"cell_type": "markdown",
"id": "6fe933fd",
"metadata": {},
"source": []
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

46
pyproject.toml Normal file
View File

@@ -0,0 +1,46 @@
[project]
name = "llm-engineering"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"anthropic>=0.69.0",
"beautifulsoup4>=4.14.2",
"chromadb>=1.1.0",
"datasets==3.6.0",
"feedparser>=6.0.12",
"google-genai>=1.41.0",
"google-generativeai>=0.8.5",
"gradio>=5.47.2",
"ipykernel>=6.30.1",
"ipywidgets>=8.1.7",
"jupyter-dash>=0.4.2",
"langchain>=0.3.27",
"langchain-chroma>=0.2.6",
"langchain-community>=0.3.30",
"langchain-core>=0.3.76",
"langchain-openai>=0.3.33",
"langchain-text-splitters>=0.3.11",
"litellm>=1.77.5",
"matplotlib>=3.10.6",
"modal>=1.1.4",
"numpy>=2.3.3",
"ollama>=0.6.0",
"openai>=1.109.1",
"pandas>=2.3.3",
"plotly>=6.3.0",
"protobuf==3.20.2",
"psutil>=7.1.0",
"pydub>=0.25.1",
"python-dotenv>=1.1.1",
"requests>=2.32.5",
"scikit-learn>=1.7.2",
"scipy>=1.16.2",
"sentence-transformers>=5.1.1",
"setuptools>=80.9.0",
"speedtest-cli>=2.1.3",
"tiktoken>=0.11.0",
"torch>=2.8.0",
"tqdm>=4.67.1",
"transformers>=4.56.2",
"wandb>=0.22.1",
]

3
requirements.txt
View File

@@ -1,5 +1,5 @@
python-dotenv
jupyterlab
ipykernel
ipywidgets
requests
numpy
@@ -7,7 +7,6 @@ pandas
scipy
scikit-learn
matplotlib
gensim
torch
transformers
tqdm

2
SETUP-PC.md → setup/SETUP-PC.md
View File

@@ -2,6 +2,8 @@
## Setup instructions for Windows
**These are the original instructions for the original version of the videos from before October 2025. For the new version, see [SETUP-new.md](SETUP-new.md).**
Welcome, PC people!
I should confess up-front: setting up a powerful environment to work at the forefront of AI is not as simple as I'd like. For most people these instructions will go great; but in some cases, for whatever reason, you'll hit a problem. Please don't hesitate to reach out - I am here to get you up and running quickly. There's nothing worse than feeling _stuck_. Message me, email me or LinkedIn message me and I will unstick you quickly!

4
SETUP-linux.md → setup/SETUP-linux.md
View File

@@ -1,6 +1,8 @@
# LLM Engineering - Master AI and LLMs
## Setup instructions for Linux
## Original Setup instructions for Linux
**These are the original instructions for the original version of the videos from before October 2025. For the new version, see [SETUP-new.md](SETUP-new.md).**
Welcome, Linux people!

4
SETUP-mac.md → setup/SETUP-mac.md
View File

@@ -1,6 +1,8 @@
# LLM Engineering - Master AI and LLMs
## Setup instructions for Mac
## Original Setup instructions for Mac
**These are the original instructions for the original version of the videos from before October 2025. For the new version, see [SETUP-new.md](SETUP-new.md).**
Welcome, Mac people!

213
setup/SETUP-new.md Normal file
View File

@@ -0,0 +1,213 @@
# LLM Engineering - Master AI and LLMs
## New Setup instructions for PC, Mac and Linux
**These are the Setup instructions for the new version of the course as of October 2025. For the original versions (Anaconda) please see the other files in this directory for your platform.**
_If you're looking at this in Cursor, please right click on the filename in the Explorer on the left, and select "Open preview", to view the formatted version._
Welcome, LLM engineers in the making!
I should confess up-front: setting up a powerful environment to work at the forefront of AI is not as simple as I'd like. For most people these instructions will go great; but in some cases, for whatever reason, you'll hit a problem. Please don't hesitate to reach out - I am here to get you up and running quickly. There's nothing worse than feeling _stuck_. Message me in Udemy or email me and I will unstick you quickly!
Email: ed@edwarddonner.com
LinkedIn: https://www.linkedin.com/in/eddonner/
## Step 0 - Before we begin - addressing the "GOTCHAS" which trip up many people:
Ignore this section at your peril! 80% of the questions I get about setup are solved by these very common system issues.
1. PC people: Permissions. Please take a look at this [tutorial](https://chatgpt.com/share/67b0ae58-d1a8-8012-82ca-74762b0408b0) on permissions on Windows. If you ever have an error that you don't have the rights / permissions / ability to run a script or install software, then read this first. ChatGPT can tell you everything you need to know about Permissions on Windows.
2. Anti-virus, Firewall, VPN. These can interfere with installations and network access; try temporarily disabling them as needed. Use a hotspot on your phone to prove whether it's a network issue.
3. PC people: The evil Windows 260 character limit to filenames - here is a full [explanation and fix](https://chatgpt.com/share/67b0afb9-1b60-8012-a9f7-f968a5a910c7)!
4. PC people: If you've not worked with Data Science packages on your computer before, you might need to install Microsoft Build Tools. Here are [instructions](https://chatgpt.com/share/67b0b762-327c-8012-b809-b4ec3b9e7be0). A student also mentioned that [these instructions](https://github.com/bycloudai/InstallVSBuildToolsWindows) might be helpful for people on Windows 11.
5. Mac people: If you're new to developing on your Mac, you may need to install XCode developer tools. Here are [instructions](https://chatgpt.com/share/67b0b8d7-8eec-8012-9a37-6973b9db11f5).
6. SSL and other network issues due to Corporate Security: if you ever get issues with SSL, such as an API Connection issue or any certificate issue, or an error trying to download files from Ollama (a Cloudflare error) then see Q15 [here](https://edwarddonner.com/faq)
## STEP 1 - installing git, projects directory, Cursor
This is the only section with separate steps for PC people and Mac/Linux people! Please pick your section below, then reconvene for Step 2...
___
**STEP 1 FOR PC people:**
1. **Install Git** (if not already installed):
- Open a new Powershell Prompt (start menu >> Powershell). If you ever have permissions errors, try opening the Powershell by right clicking and selecting "run as Administrator"
- Run the command `git` and see if it responds with details of the command or an error
- If you get an error, download Git from https://git-scm.com/download/win
- Run the installer and follow the prompts, using default options (press OK lots of times!)
2. **Create projects directory as needed**
- Open a new Powershell prompt, as in prior step. You should be in your home directory, like `C:\Users\YourUserName`
- Do you have a projects directory? Find out by typing `cd projects`
- If that has an error, then create a projects directory: `mkdir projects` then `cd projects`
- Now you should be in `C:\Users\YourUserName\projects`
- You can locate this anywhere convenient for you, but avoid any directories that are on your OneDrive
3. **Do a git clone:**
Enter this in the command prompt in the Projects folder:
`git clone https://github.com/ed-donner/llm_engineering.git`
This creates a new directory `llm_engineering` within your projects folder and downloads the code for the class.
Do `cd llm_engineering` to go into it. This `llm_engineering` directory is known as the "project root directory".
4. **Cursor** Install Cursor if needed and open the project:
Visit https://cursor.com
Click Download for Windows. Then run the installer. Accept and pick defaults for everything..
Then go to Start menu, enter cursor. Cursor will come up, and you might need to answer questions. Then you should see the 'new window' screen where you can click "Open Project". If not, go to File menu >> New Window. Then click "Open Project".
Find your llm_engineering directory within your projects directory. Double click on llm_engineering so you're looking at the contents of llm_engineering. Then click Open or Open Folder.
Cursor should then open up llm_engineering. You know you're in good shape if you see LLM_ENGINEERING in block caps on the top left.
___
**STEP 1 FOR MAC/LINUX PEOPLE**
1. **Install Git** (if not already installed):
Open a Terminal: on Mac, open a Finder window, go to Applications >> Utilities >> Terminal. On Linux, you people live in Terminals.. you hardly need instructions from me!
- Run `git --version` and you should see a git version number. If not, you should get instructions on how to install it, or follow gotcha #5 at the top of this doc.
2. **Create projects directory as needed**
- Open a new Terminal window, as in prior step. Type `pwd` to see where you are. You should be in your home directory, like `/Users/username`
- Do you have a projects directory? Find out by typing `cd projects`
- If that has an error, then create a projects directory: `mkdir projects` then `cd projects`
- If you now do `pwd` you should be in `/Users/username/projects`
- You can locate this anywhere convenient for you, but avoid any directories that are on your icloud
3. **Do a git clone:**
Enter this in the command prompt in the Projects folder:
`git clone https://github.com/ed-donner/llm_engineering.git`
This creates a new directory `llm_engineering` within your projects folder and downloads the code for the class.
Do `cd llm_engineering` to go into it. This `llm_engineering` directory is known as the "project root directory".
4. **Cursor** Install Cursor if needed and open the project:
Visit https://cursor.com
Click Download for Mac OS. Or Linux. Then run the installer. Accept and pick defaults for everything..
Then go to Start menu, enter cursor. Cursor will come up, and you might need to answer questions. Then you should see the 'new window' screen where you can click "Open Project". If not, go to File menu >> New Window. Then click "Open Project".
Find your llm_engineering directory within your projects directory. Double click on llm_engineering so you're looking at the contents of llm_engineering. Then click Open.
Cursor should then open up llm_engineering. You know you're in good shape if you see LLM_ENGINEERING in block caps on the top left.
___
## STEP 2: Installing the fabulous **uv** then doing a `uv sync`
For this course, we're using uv, the blazingly fast package manager. It's really taken off in the Data Science world -- and for good reason.
It's fast and reliable. You're going to love it!
First, within Cursor, select View >> Terminal, to see a Terminal window within Cursor. Type `pwd` to check you're in the project root directory.
Now type `uv --version` to see if uv is installed. If you get a version number, then terrific! But if you get an error, then follow the instructions here to install uv - I recommend using the Standalone Installer approach at the very top, but you can use any approach. Run commands in the Cursor installer. If one approach doesn't work for you, then try another.
https://docs.astral.sh/uv/getting-started/installation/
Once you've installed uv, you need to open a new terminal window in Cursor (the plus sign or Ctrl+shift+backtick) for `uv --version` to work. Please check!
Mac people: if you hit a problem with not having permissions to edit your profile files, please see Q14 [here](https://edwarddonner.com/faq) for the fix to this common Mac configuration issue.
### Now that it's installed:
Run `uv self update` to make sure you're on the latest version of uv.
One thing to watch for: if you've used Anaconda before, make sure that your Anaconda environment is deactivated
`conda deactivate`
And if you still have any problems with conda and python versions, it's possible that you will need to run this too:
`conda config --set auto_activate_base false`
And now simply run:
`uv sync`
And marvel at the speed and reliability! If necessary, uv should install python 3.12, and then it should install all the packages.
If you get an error about "invalid certificate" while running `uv sync`, then please check Gotcha 6 above, and try this instead:
`uv --native-tls sync`
And also try this instead:
`uv --allow-insecure-host github.com sync`
And you now have a full spec environment!!
___
## STEP 3 - OPTIONAL - Set up OpenAI Account
Alternative: see Guide 9 in the guides folder for free alternatives!
Go to https://platform.openai.com
- Click Sign Up to create an account if you don't have one. You might need to click some buttons to create an Organization first - just put in sensible defaults. See Guide 4 in the Guides folder if you're unsure about the differences between ChatGPT and the OpenAI API.
- Click on the Settings icon on the top right, and then Billing on the left sidebar navigation.
- Ensure Auto-Recharge is off. As needed, "Add to Credit Balance" and pick the $5 up-front payment amount, be sure that you add a proper payment method
- Still in Settings, select API keys in the left sidebar (near the top)
- Press "Create new secret key" - select "Owned by you", give it any name you want, for the project select "Default project", leave Permissions at All
- Press "Create secret key" and you should see your new key. Press Copy to copy it into your clipboard.
___
## STEP 4 - needed for any model like OpenAI or Gemini, but not needed if you only use Ollama - create (and SAVE) your .env file
**Be dilligent with this section!** - any mistakes with your key will be very hard to track down!! I get a huge volume of questions from students who make a mistake with one of these steps... Above all, remember to save the file after you change it.
1. Create your `.env` file
- Go back to Cursor
- In the File Explorer on the left, right click in the blank space at the bottom of all your files, select "New File" and call your file `.env`
- I can't stress this enough: the file needs to be called PRECISELY `.env` - those four characters, no more and no less. Not ".env.txt" and not "john.env" and not "openai.env" and not anything else! And it needs to be within the project root.
If you're wondering why I rant about this: I get many, many frustrated people contacting me who (despite all my appeals) have named the file something else and think it's OK. It's not OK! It needs to be called `.env` in the llm_engineering directory 😂
2. Populate your `.env` file and then Save it:
Select the file on the left. You should see an empty blank file on the right. And type this into the contents of the file in the right:
`OPENAI_API_KEY=`
And then press paste! You should now see something like this:
`OPENAI_API_KEY=sk-proj-lots-and-lots-of-digits`
But obviously with your actual key there, not the words "sk-proj-lots-and-lots-of-digits"..
Now be sure to SAVE the file! File >> Save or Ctrl+S (PC) or Command+S (Mac). Many people forget to Save. You need to Save the file!
You should see a Stop sign by the .env file - don't be concerned.. this is a good thing! See Q7 [here](https://edwarddonner.com/faq) if you want to understand why.
__
## STEP 5 - Install Cursor Extensions, Open Day 1, Set the Kernel and GO!
(If you are prompted in Cursor to install recommended extensions, just say yes! That is a nice shortcut for this step.)
- Go to view menu and select Extensions.
- Search for "python" to bring up the Python extensions. Select the Python extension made by "ms-python" or by "anysphere" and install it if not already installed.
- Search for "jupyter" and select the extension made by "ms-toolsai" and install it if not already installed.
Now go to View >> Explorer. Open the week1 folder, and click on `day1.ipynb`.
- See where it says "Select Kernel" near the top right? Click that.. then pick "Python Environments"
- Pick the top option with a star that should say something like `.venv (Python 3.12.x) .venv/bin/python Recommended`
- If this doesn't come up, please go to the troubleshooting lab in the Setup folder.
# CONGRATULATIONS!! You made it! The rest of the course is easy 😂

0
diagnostics.ipynb → setup/diagnostics.ipynb
View File

0
diagnostics.py → setup/diagnostics.py
View File

88
week1/troubleshooting.ipynb → setup/troubleshooting.ipynb
View File

@@ -14,12 +14,61 @@
"And please remember - I'm standing by to help out. Message me or email ed@edwarddonner.com and I'll get on the case. The very last cell in this notebook has some diagnostics that will help me figure out what's happening.\n"
]
},
{
"cell_type": "markdown",
"id": "8ebcd34f",
"metadata": {},
"source": [
"# If you are taking the new version of the course (using uv and Cursor, rather than Anaconda) then check this section first, otherwise jump ahead to the next section headed \"Starting with the basics\"\n",
"\n",
"## 1. Check Cursor extensions\n",
"\n",
"Just to confirm that the extensions are installed:\n",
"- Open extensions (View >> extensions)\n",
"- Search for python, and when the results show, click on the ms-python one, and Install it if not already installed\n",
"- Search for jupyter, and when the results show, click on the Microsoft one, and Install it if not already installed \n",
"Then View >> Explorer to bring back the File Explorer.\n",
"\n",
"## 2. Connect this Kernel:\n",
"\n",
"If you see the words `Select Kernel` in a button near the top right of this Window, then press the button!\n",
"\n",
"You should see a drop down titled \"Select kernel for..\" or you might need to pick \"Python environment\" first.\n",
"\n",
"Pick the one that begins `.venv python 3.12` - it should be the top choice. You might need to click \"Python Environments\" first.\n",
"\n",
"It should now say `.venv (Python 3.12.x)` where it used to say `Select Kernel`.\n",
"\n",
"After you click \"Select Kernel\", if there is no option like `.venv (Python 3.12.x)` then please do the following: \n",
"1. On Mac: From the Cursor menu, choose Settings >> VS Code Settings (NOTE: be sure to select `VSCode Settings` not `Cursor Settings`); \n",
"Or on Windows PC: From the File menu, choose Preferences >> VS Code Settings (NOTE: be sure to select `VSCode Settings` not `Cursor Settings`) \n",
"2. In the Settings search bar, type \"venv\" \n",
"3. In the field \"Path to folder with a list of Virtual Environments\" put the path to the project root, like C:\\Users\\username\\projects\\llm_engineering (on a Windows PC) or /Users/username/projects/llm_engineering (on Mac or Linux). \n",
"And then try again.\n",
"\n",
"## 3. Anaconda interference and Python version horrors\n",
"\n",
"Having problems with missing Python versions? Have you ever used Anaconda before? It might be interferring. Quit Cursor, bring up a new command line, and make sure that your Anaconda environment is deactivated: \n",
"`conda deactivate` \n",
"And if you still have any problems with conda and python versions, it's possible that you will need to run this too: \n",
"`conda config --set auto_activate_base false` \n",
"And then start a new Terminal / Powershell. \n",
"and then from within the agents directory, you should be able to run `uv python list` and see the Python 3.12 version. \n",
"And this shouldn't be required, but if you want to be absolutely careful, then now: \n",
"1. Delete the folder called '.venv' in the project root directory \n",
"2. `uv python uninstall 3.12`\n",
"3. `uv python install 3.12`\n",
"4. `uv python list`\n",
"5. `uv sync` \n",
"And surely that is bulletproof!"
]
},
{
"cell_type": "markdown",
"id": "98787335-346f-4ee4-9cb7-6181b0e1b964",
"metadata": {},
"source": [
"# Before we begin\n",
"# Starting with the basics\n",
"\n",
"## Checking your internet connection\n",
"\n",
@@ -126,8 +175,8 @@
" venv_name = os.path.basename(virtual_env)\n",
" print(f\"Environment Name: {venv_name}\")\n",
"\n",
"if conda_name != \"llms\" and venv_name != \"llms\" and venv_name != \"venv\":\n",
" print(\"Neither Anaconda nor Virtualenv seem to be activated with the expected name 'llms' or 'venv'\")\n",
"if conda_name != \"llms\" and venv_name != \"llms\" and venv_name != \"venv\" and venv_name != \".venv\":\n",
" print(\"Neither Anaconda nor Virtualenv seem to be activated with the expected name 'llms' or 'venv' or '.venv'\")\n",
" print(\"Did you run 'jupyter lab' from an activated environment with (llms) showing on the command line?\")\n",
" print(\"If in doubt, close down all jupyter lab, and follow Part 5 in the SETUP-PC or SETUP-mac guide.\")"
]
@@ -143,23 +192,22 @@
"\n",
"And now, this next cell should run with no output - no import errors. \n",
"\n",
"Import errors might indicate that you started jupyter lab without your environment activated? See SETUP Part 5. \n",
"For people on the new version of the course (October 2025 on) - an error would suggest that you don't have the right kernel.\n",
"\n",
"Or you might need to restart your Kernel and Jupyter Lab. \n",
"For people on the original course:\n",
"\n",
"Or it's possible that something is wrong with Anaconda. \n",
"If so, here are some recovery instructions: \n",
"First, close everything down and restart your computer. \n",
"Then in an Anaconda Prompt (PC) or Terminal (Mac), from an activated environment, with **(llms)** showing in the prompt, from the llm_engineering directory, run this: \n",
"`python -m pip install --upgrade pip` \n",
"`pip install --retries 5 --timeout 15 --no-cache-dir --force-reinstall -r requirements.txt` \n",
"Watch carefully for any errors, and let me know. \n",
"If you see instructions to install Microsoft Build Tools, or Apple XCode tools, then follow the instructions. \n",
"Then try again!\n",
"\n",
"Finally, if that doesn't work, please try SETUP Part 2B, the alternative to Part 2 (with Python 3.11 or Python 3.12). \n",
"\n",
"If you're unsure, please run the diagnostics (last cell in this notebook) and then email me at ed@edwarddonner.com"
"> Import errors might indicate that you started jupyter lab without your environment activated? See SETUP Part 5. \n",
"> Or you might need to restart your Kernel and Jupyter Lab. \n",
"> Or it's possible that something is wrong with Anaconda. If so, here are some recovery instructions: \n",
"> First, close everything down and restart your computer. \n",
"> Then in an Anaconda Prompt (PC) or Terminal (Mac), from an activated environment, with **(llms)** showing in the prompt, from the llm_engineering directory, run this: \n",
"> `python -m pip install --upgrade pip` \n",
"> `pip install --retries 5 --timeout 15 --no-cache-dir --force-reinstall -r requirements.txt` \n",
"> Watch carefully for any errors, and let me know. \n",
"> If you see instructions to install Microsoft Build Tools, or Apple XCode tools, then follow the instructions. \n",
"> Then try again!\n",
"> Finally, if that doesn't work, please try SETUP Part 2B, the alternative to Part 2 (with Python 3.11 or Python 3.12). \n",
"> If you're unsure, please run the diagnostics (last cell in this notebook) and then email me at ed@edwarddonner.com"
]
},
{
@@ -487,7 +535,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": ".venv",
"language": "python",
"name": "python3"
},
@@ -501,7 +549,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.11"
"version": "3.12.9"
}
},
"nbformat": 4,

4891
uv.lock generated Normal file
View File

File diff suppressed because it is too large Load Diff

235
week1/day1.ipynb
View File

@@ -8,64 +8,31 @@
"# YOUR FIRST LAB\n",
"### Please read this section. This is valuable to get you prepared, even if it's a long read -- it's important stuff.\n",
"\n",
"## Your first Frontier LLM Project\n",
"Be sure to read the README.md first!\n",
"\n",
"Let's build a useful LLM solution - in a matter of minutes.\n",
"## Your first Frontier LLM Project\n",
"\n",
"By the end of this course, you will have built an autonomous Agentic AI solution with 7 agents that collaborate to solve a business problem. All in good time! We will start with something smaller...\n",
"\n",
"Our goal is to code a new kind of Web Browser. Give it a URL, and it will respond with a summary. The Reader's Digest of the internet!!\n",
"\n",
"Before starting, you should have completed the setup for [PC](../SETUP-PC.md) or [Mac](../SETUP-mac.md) and you hopefully launched this jupyter lab from within the project root directory, with your environment activated.\n",
"Before starting, you should have completed the setup linked in the README.\n",
"\n",
"## If you're new to Jupyter Lab\n",
"### If you're new to working in \"Notebooks\" (also known as Labs or Jupyter Lab)\n",
"\n",
"Welcome to the wonderful world of Data Science experimentation! Once you've used Jupyter Lab, you'll wonder how you ever lived without it. Simply click in each \"cell\" with code in it, such as the cell immediately below this text, and hit Shift+Return to execute that cell. As you wish, you can add a cell with the + button in the toolbar, and print values of variables, or try out variations. \n",
"Welcome to the wonderful world of Data Science experimentation! Simply click in each \"cell\" with code in it, such as the cell immediately below this text, and hit Shift+Return to execute that cell. Be sure to run every cell, starting at the top, in order.\n",
"\n",
"I've written a notebook called [Guide to Jupyter](Guide%20to%20Jupyter.ipynb) to help you get more familiar with Jupyter Labs, including adding Markdown comments, using `!` to run shell commands, and `tqdm` to show progress.\n",
"\n",
"## If you're new to the Command Line\n",
"\n",
"Please see these excellent guides: [Command line on PC](https://chatgpt.com/share/67b0acea-ba38-8012-9c34-7a2541052665) and [Command line on Mac](https://chatgpt.com/canvas/shared/67b0b10c93a081918210723867525d2b). \n",
"\n",
"## If you'd prefer to work in IDEs\n",
"\n",
"If you're more comfortable in IDEs like VSCode, Cursor or PyCharm, they both work great with these lab notebooks too. \n",
"If you'd prefer to work in VSCode, [here](https://chatgpt.com/share/676f2e19-c228-8012-9911-6ca42f8ed766) are instructions from an AI friend on how to configure it for the course.\n",
"\n",
"## If you'd like to brush up your Python\n",
"\n",
"I've added a notebook called [Intermediate Python](Intermediate%20Python.ipynb) to get you up to speed. But you should give it a miss if you already have a good idea what this code does: \n",
"`yield from {book.get(\"author\") for book in books if book.get(\"author\")}`\n",
"Please look in the [Guides folder](../guides/01_intro.ipynb) for all the guides.\n",
"\n",
"## I am here to help\n",
"\n",
"If you have any problems at all, please do reach out. \n",
"I'm available through the platform, or at ed@edwarddonner.com, or at https://www.linkedin.com/in/eddonner/ if you'd like to connect (and I love connecting!) \n",
"And this is new to me, but I'm also trying out X/Twitter at [@edwarddonner](https://x.com/edwarddonner) - if you're on X, please show me how it's done 😂 \n",
"And this is new to me, but I'm also trying out X at [@edwarddonner](https://x.com/edwarddonner) - if you're on X, please show me how it's done 😂 \n",
"\n",
"## More troubleshooting\n",
"\n",
"Please see the [troubleshooting](troubleshooting.ipynb) notebook in this folder to diagnose and fix common problems. At the very end of it is a diagnostics script with some useful debug info.\n",
"\n",
"## For foundational technical knowledge (eg Git, APIs, debugging) \n",
"\n",
"If you're relatively new to programming -- I've got your back! While it's ideal to have some programming experience for this course, there's only one mandatory prerequisite: plenty of patience. 😁 I've put together a set of self-study guides that cover Git and GitHub, APIs and endpoints, beginner python and more.\n",
"\n",
"This covers Git and GitHub; what they are, the difference, and how to use them: \n",
"https://github.com/ed-donner/agents/blob/main/guides/03_git_and_github.ipynb\n",
"\n",
"This covers technical foundations: \n",
"ChatGPT vs API; taking screenshots; Environment Variables; Networking basics; APIs and endpoints: \n",
"https://github.com/ed-donner/agents/blob/main/guides/04_technical_foundations.ipynb\n",
"\n",
"This covers Python for beginners, and making sure that a `NameError` never trips you up: \n",
"https://github.com/ed-donner/agents/blob/main/guides/06_python_foundations.ipynb\n",
"\n",
"This covers the essential techniques for figuring out errors: \n",
"https://github.com/ed-donner/agents/blob/main/guides/08_debugging.ipynb\n",
"\n",
"And you'll find other useful guides in the same folder in GitHub. Some information applies to my other Udemy course (eg Async Python) but most of it is very relevant for LLM engineering.\n",
"Please see the [troubleshooting](../setup/troubleshooting.ipynb) notebook in the setup folder to diagnose and fix common problems. At the very end of it is a diagnostics script with some useful debug info.\n",
"\n",
"## If this is old hat!\n",
"\n",
@@ -74,7 +41,7 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#900;\">Please read - important note</h2>\n",
@@ -85,7 +52,7 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#f71;\">This code is a live resource - keep an eye out for my emails</h2>\n",
@@ -98,7 +65,7 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../business.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/business.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#181;\">Business value of these exercises</h2>\n",
@@ -108,6 +75,33 @@
"</table>"
]
},
{
"cell_type": "markdown",
"id": "83f28feb",
"metadata": {},
"source": [
"### If necessary, install Cursor Extensions\n",
"\n",
"1. From the View menu, select Extensions\n",
"2. Search for Python\n",
"3. Click on \"Python\" made by \"ms-python\" and select Install if not already installed\n",
"4. Search for Jupyter\n",
"5. Click on \"Jupyter\" made by \"ms-toolsai\" and select Install of not already installed\n",
"\n",
"\n",
"### Next Select the Kernel\n",
"\n",
"Click on \"Select Kernel\" on the Top Right\n",
"\n",
"Choose \"Python Environments...\"\n",
"\n",
"Then choose the one that looks like `.venv (Python 3.12.x) .venv/bin/python` - it should be marked as \"Recommended\" and have a big star next to it.\n",
"\n",
"Any problems with this? Head over to the troubleshooting.\n",
"\n",
"### Note: you'll need to set the Kernel with every notebook.."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -118,9 +112,8 @@
"# imports\n",
"\n",
"import os\n",
"import requests\n",
"from dotenv import load_dotenv\n",
"from bs4 import BeautifulSoup\n",
"from scraper import fetch_website_contents\n",
"from IPython.display import Markdown, display\n",
"from openai import OpenAI\n",
"\n",
@@ -140,9 +133,9 @@
"\n",
"## Troubleshooting if you have problems:\n",
"\n",
"Head over to the [troubleshooting](troubleshooting.ipynb) notebook in this folder for step by step code to identify the root cause and fix it!\n",
"If you get a \"Name Error\" - have you run all cells from the top down? Head over to the Python Foundations guide for a bulletproof way to find and fix all Name Errors.\n",
"\n",
"If you make a change, try restarting the \"Kernel\" (the python process sitting behind this notebook) by Kernel menu >> Restart Kernel and Clear Outputs of All Cells. Then try this notebook again, starting at the top.\n",
"If that doesn't fix it, head over to the [troubleshooting](../setup/troubleshooting.ipynb) notebook for step by step code to identify the root cause and fix it!\n",
"\n",
"Or, contact me! Message me or email ed@edwarddonner.com and we will get this to work.\n",
"\n",
@@ -173,19 +166,6 @@
" print(\"API key found and looks good so far!\")\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "019974d9-f3ad-4a8a-b5f9-0a3719aea2d3",
"metadata": {},
"outputs": [],
"source": [
"openai = OpenAI()\n",
"\n",
"# If this doesn't work, try Kernel menu >> Restart Kernel and Clear Outputs Of All Cells, then run the cells from the top of this notebook down.\n",
"# If it STILL doesn't work (horrors!) then please see the Troubleshooting notebook in this folder for full instructions"
]
},
{
"cell_type": "markdown",
"id": "442fc84b-0815-4f40-99ab-d9a5da6bda91",
@@ -204,8 +184,23 @@
"# To give you a preview -- calling OpenAI with these messages is this easy. Any problems, head over to the Troubleshooting notebook.\n",
"\n",
"message = \"Hello, GPT! This is my first ever message to you! Hi!\"\n",
"response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=[{\"role\":\"user\", \"content\":message}])\n",
"print(response.choices[0].message.content)"
"\n",
"messages = [{\"role\": \"user\", \"content\": message}]\n",
"\n",
"messages\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "08330159",
"metadata": {},
"outputs": [],
"source": [
"openai = OpenAI()\n",
"\n",
"response = openai.chat.completions.create(model=\"gpt-5-nano\", messages=messages)\n",
"response.choices[0].message.content"
]
},
{
@@ -216,36 +211,6 @@
"## OK onwards with our first project"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c5e793b2-6775-426a-a139-4848291d0463",
"metadata": {},
"outputs": [],
"source": [
"# A class to represent a Webpage\n",
"# If you're not familiar with Classes, check out the \"Intermediate Python\" notebook\n",
"\n",
"# Some websites need you to use proper headers when fetching them:\n",
"headers = {\n",
" \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36\"\n",
"}\n",
"\n",
"class Website:\n",
"\n",
" def __init__(self, url):\n",
" \"\"\"\n",
" Create this Website object from the given url using the BeautifulSoup library\n",
" \"\"\"\n",
" self.url = url\n",
" response = requests.get(url, headers=headers)\n",
" soup = BeautifulSoup(response.content, 'html.parser')\n",
" self.title = soup.title.string if soup.title else \"No title found\"\n",
" for irrelevant in soup.body([\"script\", \"style\", \"img\", \"input\"]):\n",
" irrelevant.decompose()\n",
" self.text = soup.body.get_text(separator=\"\\n\", strip=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -253,11 +218,10 @@
"metadata": {},
"outputs": [],
"source": [
"# Let's try one out. Change the website and add print statements to follow along.\n",
"# Let's try out this utility\n",
"\n",
"ed = Website(\"https://edwarddonner.com\")\n",
"print(ed.title)\n",
"print(ed.text)"
"ed = fetch_website_contents(\"https://edwarddonner.com\")\n",
"print(ed)"
]
},
{
@@ -269,7 +233,7 @@
"\n",
"You may know this already - but if not, you will get very familiar with it!\n",
"\n",
"Models like GPT4o have been trained to receive instructions in a particular way.\n",
"Models like GPT have been trained to receive instructions in a particular way.\n",
"\n",
"They expect to receive:\n",
"\n",
@@ -287,9 +251,11 @@
"source": [
"# Define our system prompt - you can experiment with this later, changing the last sentence to 'Respond in markdown in Spanish.\"\n",
"\n",
"system_prompt = \"You are an assistant that analyzes the contents of a website \\\n",
"and provides a short summary, ignoring text that might be navigation related. \\\n",
"Respond in markdown.\""
"system_prompt = \"\"\"\n",
"You are a snarkyassistant that analyzes the contents of a website,\n",
"and provides a short, snarky, humorous summary, ignoring text that might be navigation related.\n",
"Respond in markdown. Do not wrap the markdown in a code block - respond just with the markdown.\n",
"\"\"\""
]
},
{
@@ -299,25 +265,14 @@
"metadata": {},
"outputs": [],
"source": [
"# A function that writes a User Prompt that asks for summaries of websites:\n",
"# Define our user prompt\n",
"\n",
"def user_prompt_for(website):\n",
" user_prompt = f\"You are looking at a website titled {website.title}\"\n",
" user_prompt += \"\\nThe contents of this website is as follows; \\\n",
"please provide a short summary of this website in markdown. \\\n",
"If it includes news or announcements, then summarize these too.\\n\\n\"\n",
" user_prompt += website.text\n",
" return user_prompt"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "26448ec4-5c00-4204-baec-7df91d11ff2e",
"metadata": {},
"outputs": [],
"source": [
"print(user_prompt_for(ed))"
"user_prompt_prefix = \"\"\"\n",
"Here are the contents of a website.\n",
"Provide a short summary of this website.\n",
"If it includes news or announcements, then summarize these too.\n",
"\n",
"\"\"\""
]
},
{
@@ -347,22 +302,12 @@
"outputs": [],
"source": [
"messages = [\n",
" {\"role\": \"system\", \"content\": \"You are a snarky assistant\"},\n",
" {\"role\": \"system\", \"content\": \"You are a helpful assistant\"},\n",
" {\"role\": \"user\", \"content\": \"What is 2 + 2?\"}\n",
"]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "21ed95c5-7001-47de-a36d-1d6673b403ce",
"metadata": {},
"outputs": [],
"source": [
"# To give you a preview -- calling OpenAI with system and user messages:\n",
"]\n",
"\n",
"response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
"print(response.choices[0].message.content)"
"response = openai.chat.completions.create(model=\"gpt-4.1-nano\", messages=messages)\n",
"response.choices[0].message.content"
]
},
{
@@ -370,7 +315,7 @@
"id": "d06e8d78-ce4c-4b05-aa8e-17050c82bb47",
"metadata": {},
"source": [
"## And now let's build useful messages for GPT-4o-mini, using a function"
"## And now let's build useful messages for GPT-4.1-mini, using a function"
]
},
{
@@ -385,7 +330,7 @@
"def messages_for(website):\n",
" return [\n",
" {\"role\": \"system\", \"content\": system_prompt},\n",
" {\"role\": \"user\", \"content\": user_prompt_for(website)}\n",
" {\"role\": \"user\", \"content\": user_prompt_prefix + website}\n",
" ]"
]
},
@@ -419,9 +364,9 @@
"# And now: call the OpenAI API. You will get very familiar with this!\n",
"\n",
"def summarize(url):\n",
" website = Website(url)\n",
" website = fetch_website_contents(url)\n",
" response = openai.chat.completions.create(\n",
" model = \"gpt-4o-mini\",\n",
" model = \"gpt-4.1-mini\",\n",
" messages = messages_for(website)\n",
" )\n",
" return response.choices[0].message.content"
@@ -444,7 +389,7 @@
"metadata": {},
"outputs": [],
"source": [
"# A function to display this nicely in the Jupyter output, using markdown\n",
"# A function to display this nicely in the output, using markdown\n",
"\n",
"def display_summary(url):\n",
" summary = summarize(url)\n",
@@ -505,7 +450,7 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../business.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/business.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#181;\">Business applications</h2>\n",
@@ -519,7 +464,7 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#900;\">Before you continue - now try yourself</h2>\n",
@@ -549,12 +494,10 @@
"messages = [] # fill this in\n",
"\n",
"# Step 3: Call OpenAI\n",
"\n",
"response =\n",
"# response =\n",
"\n",
"# Step 4: print the result\n",
"\n",
"print("
"# print("
]
},
{
@@ -593,7 +536,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": ".venv",
"language": "python",
"name": "python3"
},
@@ -607,7 +550,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.12"
"version": "3.12.9"
}
},
"nbformat": 4,

316
week1/day2 EXERCISE.ipynb
View File

@@ -1,316 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "d15d8294-3328-4e07-ad16-8a03e9bbfdb9",
"metadata": {},
"source": [
"# Welcome to your first assignment!\n",
"\n",
"Instructions are below. Please give this a try, and look in the solutions folder if you get stuck (or feel free to ask me!)"
]
},
{
"cell_type": "markdown",
"id": "ada885d9-4d42-4d9b-97f0-74fbbbfe93a9",
"metadata": {},
"source": [
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#f71;\">Just before we get to the assignment --</h2>\n",
" <span style=\"color:#f71;\">I thought I'd take a second to point you at this page of useful resources for the course. This includes links to all the slides.<br/>\n",
" <a href=\"https://edwarddonner.com/2024/11/13/llm-engineering-resources/\">https://edwarddonner.com/2024/11/13/llm-engineering-resources/</a><br/>\n",
" Please keep this bookmarked, and I'll continue to add more useful links there over time.\n",
" </span>\n",
" </td>\n",
" </tr>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"id": "6e9fa1fc-eac5-4d1d-9be4-541b3f2b3458",
"metadata": {},
"source": [
"# HOMEWORK EXERCISE ASSIGNMENT\n",
"\n",
"Upgrade the day 1 project to summarize a webpage to use an Open Source model running locally via Ollama rather than OpenAI\n",
"\n",
"You'll be able to use this technique for all subsequent projects if you'd prefer not to use paid APIs.\n",
"\n",
"**Benefits:**\n",
"1. No API charges - open-source\n",
"2. Data doesn't leave your box\n",
"\n",
"**Disadvantages:**\n",
"1. Significantly less power than Frontier Model\n",
"\n",
"## Recap on installation of Ollama\n",
"\n",
"Simply visit [ollama.com](https://ollama.com) and install!\n",
"\n",
"Once complete, the ollama server should already be running locally. \n",
"If you visit: \n",
"[http://localhost:11434/](http://localhost:11434/)\n",
"\n",
"You should see the message `Ollama is running`. \n",
"\n",
"If not, bring up a new Terminal (Mac) or Powershell (Windows) and enter `ollama serve` \n",
"And in another Terminal (Mac) or Powershell (Windows), enter `ollama pull llama3.2` \n",
"Then try [http://localhost:11434/](http://localhost:11434/) again.\n",
"\n",
"If Ollama is slow on your machine, try using `llama3.2:1b` as an alternative. Run `ollama pull llama3.2:1b` from a Terminal or Powershell, and change the code below from `MODEL = \"llama3.2\"` to `MODEL = \"llama3.2:1b\"`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4e2a9393-7767-488e-a8bf-27c12dca35bd",
"metadata": {},
"outputs": [],
"source": [
"# imports\n",
"\n",
"import requests\n",
"from bs4 import BeautifulSoup\n",
"from IPython.display import Markdown, display"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "29ddd15d-a3c5-4f4e-a678-873f56162724",
"metadata": {},
"outputs": [],
"source": [
"# Constants\n",
"\n",
"OLLAMA_API = \"http://localhost:11434/api/chat\"\n",
"HEADERS = {\"Content-Type\": \"application/json\"}\n",
"MODEL = \"llama3.2\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dac0a679-599c-441f-9bf2-ddc73d35b940",
"metadata": {},
"outputs": [],
"source": [
"# Create a messages list using the same format that we used for OpenAI\n",
"\n",
"messages = [\n",
" {\"role\": \"user\", \"content\": \"Describe some of the business applications of Generative AI\"}\n",
"]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7bb9c624-14f0-4945-a719-8ddb64f66f47",
"metadata": {},
"outputs": [],
"source": [
"payload = {\n",
" \"model\": MODEL,\n",
" \"messages\": messages,\n",
" \"stream\": False\n",
" }"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "479ff514-e8bd-4985-a572-2ea28bb4fa40",
"metadata": {},
"outputs": [],
"source": [
"# Let's just make sure the model is loaded\n",
"\n",
"!ollama pull llama3.2"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "42b9f644-522d-4e05-a691-56e7658c0ea9",
"metadata": {},
"outputs": [],
"source": [
"# If this doesn't work for any reason, try the 2 versions in the following cells\n",
"# And double check the instructions in the 'Recap on installation of Ollama' at the top of this lab\n",
"# And if none of that works - contact me!\n",
"\n",
"response = requests.post(OLLAMA_API, json=payload, headers=HEADERS)\n",
"print(response.json()['message']['content'])"
]
},
{
"cell_type": "markdown",
"id": "6a021f13-d6a1-4b96-8e18-4eae49d876fe",
"metadata": {},
"source": [
"# Introducing the ollama package\n",
"\n",
"And now we'll do the same thing, but using the elegant ollama python package instead of a direct HTTP call.\n",
"\n",
"Under the hood, it's making the same call as above to the ollama server running at localhost:11434"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7745b9c4-57dc-4867-9180-61fa5db55eb8",
"metadata": {},
"outputs": [],
"source": [
"import ollama\n",
"\n",
"response = ollama.chat(model=MODEL, messages=messages)\n",
"print(response['message']['content'])"
]
},
{
"cell_type": "markdown",
"id": "a4704e10-f5fb-4c15-a935-f046c06fb13d",
"metadata": {},
"source": [
"## Alternative approach - using OpenAI python library to connect to Ollama"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "23057e00-b6fc-4678-93a9-6b31cb704bff",
"metadata": {},
"outputs": [],
"source": [
"# There's actually an alternative approach that some people might prefer\n",
"# You can use the OpenAI client python library to call Ollama:\n",
"\n",
"from openai import OpenAI\n",
"ollama_via_openai = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n",
"\n",
"response = ollama_via_openai.chat.completions.create(\n",
" model=MODEL,\n",
" messages=messages\n",
")\n",
"\n",
"print(response.choices[0].message.content)"
]
},
{
"cell_type": "markdown",
"id": "9f9e22da-b891-41f6-9ac9-bd0c0a5f4f44",
"metadata": {},
"source": [
"## Are you confused about why that works?\n",
"\n",
"It seems strange, right? We just used OpenAI code to call Ollama?? What's going on?!\n",
"\n",
"Here's the scoop:\n",
"\n",
"The python class `OpenAI` is simply code written by OpenAI engineers that makes calls over the internet to an endpoint. \n",
"\n",
"When you call `openai.chat.completions.create()`, this python code just makes a web request to the following url: \"https://api.openai.com/v1/chat/completions\"\n",
"\n",
"Code like this is known as a \"client library\" - it's just wrapper code that runs on your machine to make web requests. The actual power of GPT is running on OpenAI's cloud behind this API, not on your computer!\n",
"\n",
"OpenAI was so popular, that lots of other AI providers provided identical web endpoints, so you could use the same approach.\n",
"\n",
"So Ollama has an endpoint running on your local box at http://localhost:11434/v1/chat/completions \n",
"And in week 2 we'll discover that lots of other providers do this too, including Gemini and DeepSeek.\n",
"\n",
"And then the team at OpenAI had a great idea: they can extend their client library so you can specify a different 'base url', and use their library to call any compatible API.\n",
"\n",
"That's it!\n",
"\n",
"So when you say: `ollama_via_openai = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')` \n",
"Then this will make the same endpoint calls, but to Ollama instead of OpenAI."
]
},
{
"cell_type": "markdown",
"id": "bc7d1de3-e2ac-46ff-a302-3b4ba38c4c90",
"metadata": {},
"source": [
"## Also trying the amazing reasoning model DeepSeek\n",
"\n",
"Here we use the version of DeepSeek-reasoner that's been distilled to 1.5B. \n",
"This is actually a 1.5B variant of Qwen that has been fine-tuned using synethic data generated by Deepseek R1.\n",
"\n",
"Other sizes of DeepSeek are [here](https://ollama.com/library/deepseek-r1) all the way up to the full 671B parameter version, which would use up 404GB of your drive and is far too large for most!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cf9eb44e-fe5b-47aa-b719-0bb63669ab3d",
"metadata": {},
"outputs": [],
"source": [
"!ollama pull deepseek-r1:1.5b"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1d3d554b-e00d-4c08-9300-45e073950a76",
"metadata": {},
"outputs": [],
"source": [
"# This may take a few minutes to run! You should then see a fascinating \"thinking\" trace inside <think> tags, followed by some decent definitions\n",
"\n",
"response = ollama_via_openai.chat.completions.create(\n",
" model=\"deepseek-r1:1.5b\",\n",
" messages=[{\"role\": \"user\", \"content\": \"Please give definitions of some core concepts behind LLMs: a neural network, attention and the transformer\"}]\n",
")\n",
"\n",
"print(response.choices[0].message.content)"
]
},
{
"cell_type": "markdown",
"id": "1622d9bb-5c68-4d4e-9ca4-b492c751f898",
"metadata": {},
"source": [
"# NOW the exercise for you\n",
"\n",
"Take the code from day1 and incorporate it here, to build a website summarizer that uses Llama 3.2 running locally instead of OpenAI; use either of the above approaches."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6de38216-6d1c-48c4-877b-86d403f4e0f8",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

389
week1/day2.ipynb Normal file
View File

@@ -0,0 +1,389 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "d15d8294-3328-4e07-ad16-8a03e9bbfdb9",
"metadata": {},
"source": [
"# Welcome to the Day 2 Lab!\n"
]
},
{
"cell_type": "markdown",
"id": "ada885d9-4d42-4d9b-97f0-74fbbbfe93a9",
"metadata": {},
"source": [
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../assets/resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#f71;\">Just before we get started --</h2>\n",
" <span style=\"color:#f71;\">I thought I'd take a second to point you at this page of useful resources for the course. This includes links to all the slides.<br/>\n",
" <a href=\"https://edwarddonner.com/2024/11/13/llm-engineering-resources/\">https://edwarddonner.com/2024/11/13/llm-engineering-resources/</a><br/>\n",
" Please keep this bookmarked, and I'll continue to add more useful links there over time.\n",
" </span>\n",
" </td>\n",
" </tr>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"id": "79ffe36f",
"metadata": {},
"source": [
"## First - let's talk about the Chat Completions API\n",
"\n",
"1. The simplest way to call an LLM\n",
"2. It's called Chat Completions because it's saying: \"here is a conversation, please predict what should come next\"\n",
"3. The Chat Completions API was invented by OpenAI, but it's so popular that everybody uses it!\n",
"\n",
"### We will start by calling OpenAI again - but don't worry non-OpenAI people, your time is coming!\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e38f17a0",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from dotenv import load_dotenv\n",
"\n",
"load_dotenv(override=True)\n",
"api_key = os.getenv('OPENAI_API_KEY')\n",
"\n",
"if not api_key:\n",
" print(\"No API key was found - please head over to the troubleshooting notebook in this folder to identify & fix!\")\n",
"elif not api_key.startswith(\"sk-proj-\"):\n",
" print(\"An API key was found, but it doesn't start sk-proj-; please check you're using the right key - see troubleshooting notebook\")\n",
"else:\n",
" print(\"API key found and looks good so far!\")\n"
]
},
{
"cell_type": "markdown",
"id": "97846274",
"metadata": {},
"source": [
"## Do you know what an Endpoint is?\n",
"\n",
"If not, please review the Technical Foundations guide in the guides folder\n",
"\n",
"And, here is an endpoint that might interest you..."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5af5c188",
"metadata": {},
"outputs": [],
"source": [
"import requests\n",
"\n",
"headers = {\"Authorization\": f\"Bearer {api_key}\", \"Content-Type\": \"application/json\"}\n",
"\n",
"payload = {\n",
" \"model\": \"gpt-5-nano\",\n",
" \"messages\": [\n",
" {\"role\": \"user\", \"content\": \"Tell me a fun fact\"}]\n",
"}\n",
"\n",
"payload"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2d0ab242",
"metadata": {},
"outputs": [],
"source": [
"response = requests.post(\n",
" \"https://api.openai.com/v1/chat/completions\",\n",
" headers=headers,\n",
" json=payload\n",
")\n",
"\n",
"response.json()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cb11a9f6",
"metadata": {},
"outputs": [],
"source": [
"response.json()[\"choices\"][0][\"message\"][\"content\"]"
]
},
{
"cell_type": "markdown",
"id": "cea3026a",
"metadata": {},
"source": [
"# What is the openai package?\n",
"\n",
"It's known as a Python Client Library.\n",
"\n",
"It's nothing more than a wrapper around making this exact call to the http endpoint.\n",
"\n",
"It just allows you to work with nice Python code instead of messing around with janky json objects.\n",
"\n",
"But that's it. It's open-source and lightweight. Some people think it contains OpenAI model code - it doesn't!\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "490fdf09",
"metadata": {},
"outputs": [],
"source": [
"# Create OpenAI client\n",
"\n",
"from openai import OpenAI\n",
"openai = OpenAI()\n",
"\n",
"response = openai.chat.completions.create(model=\"gpt-5-nano\", messages=[{\"role\": \"user\", \"content\": \"Tell me a fun fact\"}])\n",
"\n",
"response.choices[0].message.content\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "c7739cda",
"metadata": {},
"source": [
"## And then this great thing happened:\n",
"\n",
"OpenAI's Chat Completions API was so popular, that the other model providers created endpoints that are identical.\n",
"\n",
"They are known as the \"OpenAI Compatible Endpoints\".\n",
"\n",
"For example, google made one here: https://generativelanguage.googleapis.com/v1beta/openai/\n",
"\n",
"And OpenAI decided to be kind: they said, hey, you can just use the same client library that we made for GPT. We'll allow you to specify a different endpoint URL and a different key, to use another provider.\n",
"\n",
"So you can use:\n",
"\n",
"```python\n",
"gemini = OpenAI(base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\", api_key=\"AIz....\")\n",
"gemini.chat.completions.create(...)\n",
"```\n",
"\n",
"And to be clear - even though OpenAI is in the code, we're only using this lightweight python client library to call the endpoint - there's no OpenAI model involved here.\n",
"\n",
"If you're confused, please review Guide 9 in the Guides folder!\n",
"\n",
"And now let's try it!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f74293bc",
"metadata": {},
"outputs": [],
"source": [
"GEMINI_BASE_URL = \"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
"\n",
"google_api_key = os.getenv(\"GOOGLE_API_KEY\")\n",
"\n",
"if not google_api_key:\n",
" print(\"No API key was found - please head over to the troubleshooting notebook in this folder to identify & fix!\")\n",
"elif not google_api_key.startswith(\"AIz\"):\n",
" print(\"An API key was found, but it doesn't start AIz\")\n",
"else:\n",
" print(\"API key found and looks good so far!\")\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d060f484",
"metadata": {},
"outputs": [],
"source": [
"gemini = OpenAI(base_url=GEMINI_BASE_URL, api_key=google_api_key)\n",
"\n",
"response = gemini.chat.completions.create(model=\"gemini-2.5-pro\", messages=[{\"role\": \"user\", \"content\": \"Tell me a fun fact\"}])\n",
"\n",
"response.choices[0].message.content"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a5b069be",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "65272432",
"metadata": {},
"source": [
"## And Ollama also gives an OpenAI compatible endpoint\n",
"\n",
"...and it's on your local machine!\n",
"\n",
"If the next cell doesn't print \"Ollama is running\" then please open a terminal and run `ollama serve`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f06280ad",
"metadata": {},
"outputs": [],
"source": [
"requests.get(\"http://localhost:11434\").content"
]
},
{
"cell_type": "markdown",
"id": "c6ef3807",
"metadata": {},
"source": [
"### Download llama3.2 from meta\n",
"\n",
"Change this to llama3.2:1b if your computer is smaller.\n",
"\n",
"Don't use llama3.3 or llama4! They are too big for your computer.."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e633481d",
"metadata": {},
"outputs": [],
"source": [
"!ollama pull llama3.2"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d9419762",
"metadata": {},
"outputs": [],
"source": [
"OLLAMA_BASE_URL = \"http://localhost:11434/v1\"\n",
"\n",
"ollama = OpenAI(base_url=OLLAMA_BASE_URL, api_key='ollama')\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e2456cdf",
"metadata": {},
"outputs": [],
"source": [
"# Get a fun fact\n",
"\n",
"response = ollama.chat.completions.create(model=\"llama3.2\", messages=[{\"role\": \"user\", \"content\": \"Tell me a fun fact\"}])\n",
"\n",
"response.choices[0].message.content"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1e6cae7f",
"metadata": {},
"outputs": [],
"source": [
"# Now let's try deepseek-r1:1.5b - this is DeepSeek \"distilled\" into Qwen from Alibaba Cloud\n",
"\n",
"!ollama pull deepseek-r1:1.5b"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "25002f25",
"metadata": {},
"outputs": [],
"source": [
"response = ollama.chat.completions.create(model=\"deepseek-r1:1.5b\", messages=[{\"role\": \"user\", \"content\": \"Tell me a fun fact\"}])\n",
"\n",
"response.choices[0].message.content"
]
},
{
"cell_type": "markdown",
"id": "6e9fa1fc-eac5-4d1d-9be4-541b3f2b3458",
"metadata": {},
"source": [
"# HOMEWORK EXERCISE ASSIGNMENT\n",
"\n",
"Upgrade the day 1 project to summarize a webpage to use an Open Source model running locally via Ollama rather than OpenAI\n",
"\n",
"You'll be able to use this technique for all subsequent projects if you'd prefer not to use paid APIs.\n",
"\n",
"**Benefits:**\n",
"1. No API charges - open-source\n",
"2. Data doesn't leave your box\n",
"\n",
"**Disadvantages:**\n",
"1. Significantly less power than Frontier Model\n",
"\n",
"## Recap on installation of Ollama\n",
"\n",
"Simply visit [ollama.com](https://ollama.com) and install!\n",
"\n",
"Once complete, the ollama server should already be running locally. \n",
"If you visit: \n",
"[http://localhost:11434/](http://localhost:11434/)\n",
"\n",
"You should see the message `Ollama is running`. \n",
"\n",
"If not, bring up a new Terminal (Mac) or Powershell (Windows) and enter `ollama serve` \n",
"And in another Terminal (Mac) or Powershell (Windows), enter `ollama pull llama3.2` \n",
"Then try [http://localhost:11434/](http://localhost:11434/) again.\n",
"\n",
"If Ollama is slow on your machine, try using `llama3.2:1b` as an alternative. Run `ollama pull llama3.2:1b` from a Terminal or Powershell, and change the code from `MODEL = \"llama3.2\"` to `MODEL = \"llama3.2:1b\"`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6de38216-6d1c-48c4-877b-86d403f4e0f8",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

303
week1/day4.ipynb Normal file
View File

@@ -0,0 +1,303 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "d9e61417",
"metadata": {},
"source": [
"# Day 4\n",
"\n",
"## Tokenizing with code"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "7dc1c1d9",
"metadata": {},
"outputs": [],
"source": [
"import tiktoken\n",
"\n",
"encoding = tiktoken.encoding_for_model(\"gpt-4.1-mini\")\n",
"\n",
"tokens = encoding.encode(\"Hi my name is Ed and I like banoffee pie\")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "7632966c",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[12194, 922, 1308, 382, 6117, 326, 357, 1299, 9171, 26458, 5148]"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tokens"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "cce0c188",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"12194 = Hi\n",
"922 = my\n",
"1308 = name\n",
"382 = is\n",
"6117 = Ed\n",
"326 = and\n",
"357 = I\n",
"1299 = like\n",
"9171 = ban\n",
"26458 = offee\n",
"5148 = pie\n"
]
}
],
"source": [
"for token_id in tokens:\n",
" token_text = encoding.decode([token_id])\n",
" print(f\"{token_id} = {token_text}\")"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "98e3bbd2",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"' and'"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"encoding.decode([326])"
]
},
{
"cell_type": "markdown",
"id": "538efe61",
"metadata": {},
"source": [
"# And another topic!\n",
"\n",
"### The Illusion of \"memory\"\n",
"\n",
"Many of you will know this already. But for those that don't -- this might be an \"AHA\" moment!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "83a4b3eb",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from dotenv import load_dotenv\n",
"\n",
"load_dotenv(override=True)\n",
"api_key = os.getenv('OPENAI_API_KEY')\n",
"\n",
"if not api_key:\n",
" print(\"No API key was found - please head over to the troubleshooting notebook in this folder to identify & fix!\")\n",
"elif not api_key.startswith(\"sk-proj-\"):\n",
" print(\"An API key was found, but it doesn't start sk-proj-; please check you're using the right key - see troubleshooting notebook\")\n",
"else:\n",
" print(\"API key found and looks good so far!\")"
]
},
{
"cell_type": "markdown",
"id": "b618859b",
"metadata": {},
"source": [
"### You should be very comfortable with what the next cell is doing!\n",
"\n",
"_I'm creating a new instance of the OpenAI Python Client library, a lightweight wrapper around making HTTP calls to an endpoint for calling the GPT LLM, or other LLM providers_"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b959be3b",
"metadata": {},
"outputs": [],
"source": [
"from openai import OpenAI\n",
"\n",
"openai = OpenAI()"
]
},
{
"cell_type": "markdown",
"id": "aa889e80",
"metadata": {},
"source": [
"### A message to OpenAI is a list of dicts"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "97298fea",
"metadata": {},
"outputs": [],
"source": [
"messages = [\n",
" {\"role\": \"system\", \"content\": \"You are a helpful assistant\"},\n",
" {\"role\": \"user\", \"content\": \"Hi! I'm Ed!\"}\n",
" ]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3475a36d",
"metadata": {},
"outputs": [],
"source": [
"response = openai.chat.completions.create(model=\"gpt-4.1-mini\", messages=messages)\n",
"response.choices[0].message.content"
]
},
{
"cell_type": "markdown",
"id": "a5f45ed8",
"metadata": {},
"source": [
"### OK let's now ask a follow-up question"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6bce2208",
"metadata": {},
"outputs": [],
"source": [
"messages = [\n",
" {\"role\": \"system\", \"content\": \"You are a helpful assistant\"},\n",
" {\"role\": \"user\", \"content\": \"What's my name?\"}\n",
" ]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "404462f5",
"metadata": {},
"outputs": [],
"source": [
"response = openai.chat.completions.create(model=\"gpt-4.1-mini\", messages=messages)\n",
"response.choices[0].message.content"
]
},
{
"cell_type": "markdown",
"id": "098237ef",
"metadata": {},
"source": [
"### Wait, wha??\n",
"\n",
"We just told you!\n",
"\n",
"What's going on??\n",
"\n",
"Here's the thing: every call to an LLM is completely STATELESS. It's a totally new call, every single time. As AI engineers, it's OUR JOB to devise techniques to give the impression that the LLM has a \"memory\"."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b6d43f92",
"metadata": {},
"outputs": [],
"source": [
"messages = [\n",
" {\"role\": \"system\", \"content\": \"You are a helpful assistant\"},\n",
" {\"role\": \"user\", \"content\": \"Hi! I'm Ed!\"},\n",
" {\"role\": \"assistant\", \"content\": \"Hi Ed! How can I assist you today?\"},\n",
" {\"role\": \"user\", \"content\": \"What's my name?\"}\n",
" ]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e7ac742c",
"metadata": {},
"outputs": [],
"source": [
"response = openai.chat.completions.create(model=\"gpt-4.1-mini\", messages=messages)\n",
"response.choices[0].message.content"
]
},
{
"cell_type": "markdown",
"id": "96c49557",
"metadata": {},
"source": [
"## To recap\n",
"\n",
"With apologies if this is obvious to you - but it's still good to reinforce:\n",
"\n",
"1. Every call to an LLM is stateless\n",
"2. We pass in the entire conversation so far in the input prompt, every time\n",
"3. This gives the illusion that the LLM has memory - it apparently keeps the context of the conversation\n",
"4. But this is a trick; it's a by-product of providing the entire conversation, every time\n",
"5. An LLM just predicts the most likely next tokens in the sequence; if that sequence contains \"My name is Ed\" and later \"What's my name?\" then it will predict.. Ed!\n",
"\n",
"The ChatGPT product uses exactly this trick - every time you send a message, it's the entire conversation that gets passed in.\n",
"\n",
"\"Does that mean we have to pay extra each time for all the conversation so far\"\n",
"\n",
"For sure it does. And that's what we WANT. We want the LLM to predict the next tokens in the sequence, looking back on the entire conversation. We want that compute to happen, so we need to pay the electricity bill for it!\n",
"\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

1658
week1/day5.ipynb
View File

File diff suppressed because one or more lines are too long

419
week1/diagnostics.py
View File

@@ -1,419 +0,0 @@
import os
import sys
import platform
import subprocess
import shutil
import time
import ssl
import tempfile
from pathlib import Path
from datetime import datetime
class Diagnostics:
FILENAME = 'report.txt'
def __init__(self):
self.errors = []
self.warnings = []
if os.path.exists(self.FILENAME):
os.remove(self.FILENAME)
def log(self, message):
print(message)
with open(self.FILENAME, 'a', encoding='utf-8') as f:
f.write(message + "\n")
def start(self):
now = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
self.log(f"Starting diagnostics at {now}\n")
def end(self):
now = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
self.log(f"\n\nCompleted diagnostics at {now}\n")
print("\nPlease send these diagnostics to me at ed@edwarddonner.com")
print(f"Either copy & paste the above output into an email, or attach the file {self.FILENAME} that has been created in this directory.")
def _log_error(self, message):
self.log(f"ERROR: {message}")
self.errors.append(message)
def _log_warning(self, message):
self.log(f"WARNING: {message}")
self.warnings.append(message)
def run(self):
self.start()
self._step1_system_info()
self._step2_check_files()
self._step3_git_repo()
self._step4_check_env_file()
self._step5_anaconda_check()
self._step6_virtualenv_check()
self._step7_network_connectivity()
self._step8_environment_variables()
self._step9_additional_diagnostics()
if self.warnings:
self.log("\n===== Warnings Found =====")
self.log("The following warnings were detected. They might not prevent the program from running but could cause unexpected behavior:")
for warning in self.warnings:
self.log(f"- {warning}")
if self.errors:
self.log("\n===== Errors Found =====")
self.log("The following critical issues were detected. Please address them before proceeding:")
for error in self.errors:
self.log(f"- {error}")
if not self.errors and not self.warnings:
self.log("\n✅ All diagnostics passed successfully!")
self.end()
def _step1_system_info(self):
self.log("===== System Information =====")
try:
system = platform.system()
self.log(f"Operating System: {system}")
if system == "Windows":
release, version, csd, ptype = platform.win32_ver()
self.log(f"Windows Release: {release}")
self.log(f"Windows Version: {version}")
elif system == "Darwin":
release, version, machine = platform.mac_ver()
self.log(f"MacOS Version: {release}")
else:
self.log(f"Platform: {platform.platform()}")
self.log(f"Architecture: {platform.architecture()}")
self.log(f"Machine: {platform.machine()}")
self.log(f"Processor: {platform.processor()}")
try:
import psutil
ram = psutil.virtual_memory()
total_ram_gb = ram.total / (1024 ** 3)
available_ram_gb = ram.available / (1024 ** 3)
self.log(f"Total RAM: {total_ram_gb:.2f} GB")
self.log(f"Available RAM: {available_ram_gb:.2f} GB")
if available_ram_gb < 2:
self._log_warning(f"Low available RAM: {available_ram_gb:.2f} GB")
except ImportError:
self._log_warning("psutil module not found. Cannot determine RAM information.")
total, used, free = shutil.disk_usage(os.path.expanduser("~"))
free_gb = free / (1024 ** 3)
self.log(f"Free Disk Space: {free_gb:.2f} GB")
if free_gb < 5:
self._log_warning(f"Low disk space: {free_gb:.2f} GB free")
except Exception as e:
self._log_error(f"System information check failed: {e}")
def _step2_check_files(self):
self.log("\n===== File System Information =====")
try:
current_dir = os.getcwd()
self.log(f"Current Directory: {current_dir}")
# Check write permissions
test_file = Path(current_dir) / ".test_write_permission"
try:
test_file.touch(exist_ok=True)
test_file.unlink()
self.log("Write permission: OK")
except Exception as e:
self._log_error(f"No write permission in current directory: {e}")
self.log("\nFiles in Current Directory:")
try:
for item in sorted(os.listdir(current_dir)):
self.log(f" - {item}")
except Exception as e:
self._log_error(f"Cannot list directory contents: {e}")
except Exception as e:
self._log_error(f"File system check failed: {e}")
def _step3_git_repo(self):
self.log("\n===== Git Repository Information =====")
try:
result = subprocess.run(['git', 'rev-parse', '--show-toplevel'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
if result.returncode == 0:
git_root = result.stdout.strip()
self.log(f"Git Repository Root: {git_root}")
result = subprocess.run(['git', 'rev-parse', 'HEAD'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
if result.returncode == 0:
self.log(f"Current Commit: {result.stdout.strip()}")
else:
self._log_warning(f"Could not get current commit: {result.stderr.strip()}")
result = subprocess.run(['git', 'remote', 'get-url', 'origin'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
if result.returncode == 0:
self.log(f"Remote Origin: {result.stdout.strip()}")
else:
self._log_warning("No remote 'origin' configured")
else:
self._log_warning("Not a git repository")
except FileNotFoundError:
self._log_warning("Git is not installed or not in PATH")
except Exception as e:
self._log_error(f"Git check failed: {e}")
def _step4_check_env_file(self):
self.log("\n===== Environment File Check =====")
try:
result = subprocess.run(['git', 'rev-parse', '--show-toplevel'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
if result.returncode == 0:
git_root = result.stdout.strip()
env_path = os.path.join(git_root, '.env')
if os.path.isfile(env_path):
self.log(f".env file exists at: {env_path}")
try:
with open(env_path, 'r') as f:
has_api_key = any(line.strip().startswith('OPENAI_API_KEY=') for line in f)
if has_api_key:
self.log("OPENAI_API_KEY found in .env file")
else:
self._log_warning("OPENAI_API_KEY not found in .env file")
except Exception as e:
self._log_error(f"Cannot read .env file: {e}")
else:
self._log_warning(".env file not found in project root")
# Check for additional .env files
for root, _, files in os.walk(git_root):
if '.env' in files and os.path.join(root, '.env') != env_path:
self._log_warning(f"Additional .env file found at: {os.path.join(root, '.env')}")
else:
self._log_warning("Git root directory not found. Cannot perform .env file check.")
except FileNotFoundError:
self._log_warning("Git is not installed or not in PATH")
except Exception as e:
self._log_error(f"Environment file check failed: {e}")
def _step5_anaconda_check(self):
self.log("\n===== Anaconda Environment Check =====")
try:
conda_prefix = os.environ.get('CONDA_PREFIX')
if conda_prefix:
self.log("Anaconda environment is active:")
self.log(f"Environment Path: {conda_prefix}")
self.log(f"Environment Name: {os.path.basename(conda_prefix)}")
conda_exe = os.environ.get('CONDA_EXE', 'conda')
result = subprocess.run([conda_exe, '--version'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
if result.returncode == 0:
self.log(f"Conda Version: {result.stdout.strip()}")
else:
self._log_warning("Could not determine Conda version")
self._check_python_packages()
else:
self.log("No active Anaconda environment detected")
except Exception as e:
self._log_error(f"Anaconda environment check failed: {e}")
def _step6_virtualenv_check(self):
self.log("\n===== Virtualenv Check =====")
try:
virtual_env = os.environ.get('VIRTUAL_ENV')
if virtual_env:
self.log("Virtualenv is active:")
self.log(f"Environment Path: {virtual_env}")
self.log(f"Environment Name: {os.path.basename(virtual_env)}")
self._check_python_packages()
else:
self.log("No active virtualenv detected")
if not virtual_env and not os.environ.get('CONDA_PREFIX'):
self._log_warning("Neither virtualenv nor Anaconda environment is active")
except Exception as e:
self._log_error(f"Virtualenv check failed: {e}")
def _check_python_packages(self):
self.log("\nPython Environment:")
self.log(f"Python Version: {sys.version}")
self.log(f"Python Executable: {sys.executable}")
required_packages = ['openai', 'python-dotenv', 'requests', 'gradio', 'transformers']
try:
import pkg_resources
installed = {pkg.key: pkg.version for pkg in pkg_resources.working_set}
self.log("\nRequired Package Versions:")
for package in required_packages:
if package in installed:
self.log(f"{package}: {installed[package]}")
else:
self._log_error(f"Required package '{package}' is not installed")
# Check for potentially conflicting packages
problem_pairs = [
('openai', 'openai-python'),
('python-dotenv', 'dotenv')
]
for pkg1, pkg2 in problem_pairs:
if pkg1 in installed and pkg2 in installed:
self._log_warning(f"Potentially conflicting packages: {pkg1} and {pkg2}")
except ImportError:
self._log_error("Could not import 'pkg_resources' to check installed packages")
except Exception as e:
self._log_error(f"Package check failed: {e}")
def _step7_network_connectivity(self):
self.log("\n===== Network Connectivity Check =====")
try:
self.log(f"SSL Version: {ssl.OPENSSL_VERSION}")
import requests
import speedtest # Importing the speedtest-cli library
# Basic connectivity check
urls = [
'https://www.google.com',
'https://www.cloudflare.com'
]
connected = False
for url in urls:
try:
start_time = time.time()
response = requests.get(url, timeout=10)
elapsed_time = time.time() - start_time
response.raise_for_status()
self.log(f"✓ Connected to {url}")
self.log(f" Response time: {elapsed_time:.2f}s")
if elapsed_time > 2:
self._log_warning(f"Slow response from {url}: {elapsed_time:.2f}s")
connected = True
break
except requests.exceptions.RequestException as e:
self._log_warning(f"Failed to connect to {url}: {e}")
else:
self.log("Basic connectivity OK")
if not connected:
self._log_error("Failed to connect to any test URLs")
return
# Bandwidth test using speedtest-cli
self.log("\nPerforming bandwidth test using speedtest-cli...")
try:
st = speedtest.Speedtest()
st.get_best_server()
download_speed = st.download() # Bits per second
upload_speed = st.upload() # Bits per second
download_mbps = download_speed / 1e6 # Convert to Mbps
upload_mbps = upload_speed / 1e6
self.log(f"Download speed: {download_mbps:.2f} Mbps")
self.log(f"Upload speed: {upload_mbps:.2f} Mbps")
if download_mbps < 1:
self._log_warning("Download speed is low")
if upload_mbps < 0.5:
self._log_warning("Upload speed is low")
except speedtest.ConfigRetrievalError:
self._log_error("Failed to retrieve speedtest configuration")
except Exception as e:
self._log_warning(f"Bandwidth test failed: {e}")
except ImportError:
self._log_error("Required packages are not installed. Please install them using 'pip install requests speedtest-cli'")
except Exception as e:
self._log_error(f"Network connectivity check failed: {e}")
def _step8_environment_variables(self):
self.log("\n===== Environment Variables Check =====")
try:
# Check Python paths
pythonpath = os.environ.get('PYTHONPATH')
if pythonpath:
self.log("\nPYTHONPATH:")
for path in pythonpath.split(os.pathsep):
self.log(f" - {path}")
else:
self.log("\nPYTHONPATH is not set.")
self.log("\nPython sys.path:")
for path in sys.path:
self.log(f" - {path}")
# Check OPENAI_API_KEY
from dotenv import load_dotenv
load_dotenv()
api_key = os.environ.get('OPENAI_API_KEY')
if api_key:
self.log("OPENAI_API_KEY is set after calling load_dotenv()")
if not api_key.startswith('sk-proj-') or len(api_key)<12:
self._log_warning("OPENAI_API_KEY format looks incorrect after calling load_dotenv()")
else:
self._log_warning("OPENAI_API_KEY environment variable is not set after calling load_dotenv()")
except Exception as e:
self._log_error(f"Environment variables check failed: {e}")
def _step9_additional_diagnostics(self):
self.log("\n===== Additional Diagnostics =====")
try:
# Get the site-packages directory paths
import site
site_packages_paths = site.getsitepackages()
if hasattr(site, 'getusersitepackages'):
site_packages_paths.append(site.getusersitepackages())
# Function to check if a path is within site-packages
def is_in_site_packages(path):
return any(os.path.commonpath([path, sp]) == sp for sp in site_packages_paths)
# Check for potential name conflicts in the current directory and sys.path
conflict_names = ['openai.py', 'dotenv.py']
# Check current directory
current_dir = os.getcwd()
for name in conflict_names:
conflict_path = os.path.join(current_dir, name)
if os.path.isfile(conflict_path):
self._log_warning(f"Found '{name}' in the current directory, which may cause import conflicts: {conflict_path}")
# Check sys.path directories
for path in sys.path:
if not path or is_in_site_packages(path):
continue # Skip site-packages and empty paths
for name in conflict_names:
conflict_file = os.path.join(path, name)
if os.path.isfile(conflict_file):
self._log_warning(f"Potential naming conflict: {conflict_file}")
# Check temp directory
try:
with tempfile.NamedTemporaryFile() as tmp:
self.log(f"Temp directory is writable: {os.path.dirname(tmp.name)}")
except Exception as e:
self._log_error(f"Cannot write to temp directory: {e}")
except Exception as e:
self._log_error(f"Additional diagnostics failed: {e}")
if __name__ == "__main__":
diagnostics = Diagnostics()
diagnostics.run()

37
week1/scraper.py Normal file
View File

@@ -0,0 +1,37 @@
from bs4 import BeautifulSoup
import requests
# Standard headers to fetch a website
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36"
}
def fetch_website_contents(url):
"""
Return the title and contents of the website at the given url;
truncate to 2,000 characters as a sensible limit
"""
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, "html.parser")
title = soup.title.string if soup.title else "No title found"
if soup.body:
for irrelevant in soup.body(["script", "style", "img", "input"]):
irrelevant.decompose()
text = soup.body.get_text(separator="\n", strip=True)
else:
text = ""
return (title + "\n\n" + text)[:2_000]
def fetch_website_links(url):
"""
Return the links on the webiste at the given url
I realize this is inefficient as we're parsing twice! This is to keep the code in the lab simple.
Feel free to use a class and optimize it!
"""
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, "html.parser")
links = [link.get("href") for link in soup.find_all("a")]
return [link for link in links if link]

53
week1/solution.py Normal file
View File

@@ -0,0 +1,53 @@
"""
Website summarizer using Ollama instead of OpenAI.
"""
from openai import OpenAI
from scraper import fetch_website_contents
OLLAMA_BASE_URL = "http://localhost:11434/v1"
MODEL = "llama3.2"
system_prompt = """
You are a snarky assistant that analyzes the contents of a website,
and provides a short, snarky, humorous summary, ignoring text that might be navigation related.
Respond in markdown. Do not wrap the markdown in a code block - respond just with the markdown.
"""
user_prompt_prefix = """
Here are the contents of a website.
Provide a short summary of this website.
If it includes news or announcements, then summarize these too.
"""
def messages_for(website):
"""Create message list for the LLM."""
return [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt_prefix + website}
]
def summarize(url):
"""Fetch and summarize a website using Ollama."""
ollama = OpenAI(base_url=OLLAMA_BASE_URL, api_key='ollama')
website = fetch_website_contents(url)
response = ollama.chat.completions.create(
model=MODEL,
messages=messages_for(website)
)
return response.choices[0].message.content
def main():
"""Main entry point for testing."""
url = input("Enter a URL to summarize: ")
print("\nFetching and summarizing...\n")
summary = summarize(url)
print(summary)
if __name__ == "__main__":
main()

955
week2/day1.ipynb
View File

File diff suppressed because it is too large Load Diff

742
week2/day2.ipynb
View File

File diff suppressed because it is too large Load Diff

194
week2/day3.ipynb
View File

@@ -35,23 +35,11 @@
"\n",
"load_dotenv(override=True)\n",
"openai_api_key = os.getenv('OPENAI_API_KEY')\n",
"anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')\n",
"google_api_key = os.getenv('GOOGLE_API_KEY')\n",
"\n",
"if openai_api_key:\n",
" print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n",
"else:\n",
" print(\"OpenAI API Key not set\")\n",
" \n",
"if anthropic_api_key:\n",
" print(f\"Anthropic API Key exists and begins {anthropic_api_key[:7]}\")\n",
"else:\n",
" print(\"Anthropic API Key not set\")\n",
"\n",
"if google_api_key:\n",
" print(f\"Google API Key exists and begins {google_api_key[:8]}\")\n",
"else:\n",
" print(\"Google API Key not set\")"
" print(\"OpenAI API Key not set\")"
]
},
{
@@ -64,7 +52,7 @@
"# Initialize\n",
"\n",
"openai = OpenAI()\n",
"MODEL = 'gpt-4o-mini'"
"MODEL = 'gpt-4.1-mini'"
]
},
{
@@ -74,6 +62,8 @@
"metadata": {},
"outputs": [],
"source": [
"# Again, I'll be in scientist-mode and change this global during the lab\n",
"\n",
"system_message = \"You are a helpful assistant\""
]
},
@@ -82,32 +72,75 @@
"id": "98e97227-f162-4d1a-a0b2-345ff248cbe7",
"metadata": {},
"source": [
"# Please read this! A change from the video:\n",
"## And now, writing a new callback\n",
"\n",
"In the video, I explain how we now need to write a function called:\n",
"We now need to write a function called:\n",
"\n",
"`chat(message, history)`\n",
"\n",
"Which expects to receive `history` in a particular format, which we need to map to the OpenAI format before we call OpenAI:\n",
"Which will be a callback function we will give gradio.\n",
"\n",
"```\n",
"[\n",
" {\"role\": \"system\", \"content\": \"system message here\"},\n",
" {\"role\": \"user\", \"content\": \"first user prompt here\"},\n",
" {\"role\": \"assistant\", \"content\": \"the assistant's response\"},\n",
" {\"role\": \"user\", \"content\": \"the new user prompt\"},\n",
"]\n",
"```\n",
"### The job of this function\n",
"\n",
"But Gradio has been upgraded! Now it will pass in `history` in the exact OpenAI format, perfect for us to send straight to OpenAI.\n",
"\n",
"So our work just got easier!\n",
"\n",
"We will write a function `chat(message, history)` where: \n",
"**message** is the prompt to use \n",
"**history** is the past conversation, in OpenAI format \n",
"\n",
"We will combine the system message, history and latest message, then call OpenAI."
"Take a message, take the prior conversation, and return the response.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "354ce793",
"metadata": {},
"outputs": [],
"source": [
"def chat(message, history):\n",
" return \"bananas\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e87f3417",
"metadata": {},
"outputs": [],
"source": [
"gr.ChatInterface(fn=chat, type=\"messages\").launch()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5d4996e8",
"metadata": {},
"outputs": [],
"source": [
"def chat(message, history):\n",
" return f\"You said {message} and the history is {history} but I still say bananas\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "434a0417",
"metadata": {},
"outputs": [],
"source": [
"gr.ChatInterface(fn=chat, type=\"messages\").launch()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7890cac3",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "f7330d7f",
"metadata": {},
"source": [
"## OK! Let's write a slightly better chat callback!"
]
},
{
@@ -117,46 +150,61 @@
"metadata": {},
"outputs": [],
"source": [
"# Simpler than in my video - we can easily create this function that calls OpenAI\n",
"# It's now just 1 line of code to prepare the input to OpenAI!\n",
"\n",
"# Student Octavio O. has pointed out that this isn't quite as straightforward for Claude -\n",
"# see the excellent contribution in community-contributions \"Gradio_issue_with_Claude\" that handles Claude.\n",
"\n",
"def chat(message, history):\n",
" history = [{\"role\":h[\"role\"], \"content\":h[\"content\"]} for h in history]\n",
" messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages)\n",
" return response.choices[0].message.content\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0ab706f9",
"metadata": {},
"outputs": [],
"source": [
"gr.ChatInterface(fn=chat, type=\"messages\").launch()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3bce145a",
"metadata": {},
"outputs": [],
"source": [
"def chat(message, history):\n",
" history = [{\"role\":h[\"role\"], \"content\":h[\"content\"]} for h in history]\n",
" messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n",
"\n",
" print(\"History is:\")\n",
" print(history)\n",
" print(\"And messages is:\")\n",
" print(messages)\n",
"\n",
" stream = openai.chat.completions.create(model=MODEL, messages=messages, stream=True)\n",
"\n",
" response = \"\"\n",
" for chunk in stream:\n",
" response += chunk.choices[0].delta.content or ''\n",
" yield response"
]
},
{
"cell_type": "markdown",
"id": "1334422a-808f-4147-9c4c-57d63d9780d0",
"metadata": {},
"source": [
"## And then enter Gradio's magic!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0866ca56-100a-44ab-8bd0-1568feaf6bf2",
"id": "b8beeca6",
"metadata": {},
"outputs": [],
"source": [
"gr.ChatInterface(fn=chat, type=\"messages\").launch()"
]
},
{
"cell_type": "markdown",
"id": "1334422a-808f-4147-9c4c-57d63d9780d0",
"metadata": {},
"source": [
"## OK let's keep going!\n",
"\n",
"Using a system message to add context, and to give an example answer.. this is \"one shot prompting\" again"
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -171,24 +219,6 @@
"Encourage the customer to buy hats if they are unsure what to get.\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4e5be3ec-c26c-42bc-ac16-c39d369883f6",
"metadata": {},
"outputs": [],
"source": [
"def chat(message, history):\n",
" messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n",
"\n",
" stream = openai.chat.completions.create(model=MODEL, messages=messages, stream=True)\n",
"\n",
" response = \"\"\n",
" for chunk in stream:\n",
" response += chunk.choices[0].delta.content or ''\n",
" yield response"
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -227,13 +257,11 @@
"metadata": {},
"outputs": [],
"source": [
"# Fixed a bug in this function brilliantly identified by student Gabor M.!\n",
"# I've also improved the structure of this function\n",
"\n",
"def chat(message, history):\n",
"\n",
" history = [{\"role\":h[\"role\"], \"content\":h[\"content\"]} for h in history]\n",
" relevant_system_message = system_message\n",
" if 'belt' in message:\n",
" if 'belt' in message.lower():\n",
" relevant_system_message += \" The store does not sell belts; if you are asked for belts, be sure to point out other items on sale.\"\n",
" \n",
" messages = [{\"role\": \"system\", \"content\": relevant_system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n",
@@ -264,7 +292,7 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../business.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/business.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#181;\">Business Applications</h2>\n",
@@ -277,17 +305,15 @@
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6dfb9e21-df67-4c2b-b952-5e7e7961b03d",
"cell_type": "markdown",
"id": "acc0e5a9",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": ".venv",
"language": "python",
"name": "python3"
},
@@ -301,7 +327,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.13"
"version": "3.12.9"
}
},
"nbformat": 4,

274
week2/day4.ipynb
View File

@@ -17,8 +17,6 @@
"metadata": {},
"outputs": [],
"source": [
"# imports\n",
"\n",
"import os\n",
"import json\n",
"from dotenv import load_dotenv\n",
@@ -43,7 +41,7 @@
"else:\n",
" print(\"OpenAI API Key not set\")\n",
" \n",
"MODEL = \"gpt-4o-mini\"\n",
"MODEL = \"gpt-4.1-mini\"\n",
"openai = OpenAI()\n",
"\n",
"# As an alternative, if you'd like to use Ollama instead of OpenAI\n",
@@ -59,9 +57,11 @@
"metadata": {},
"outputs": [],
"source": [
"system_message = \"You are a helpful assistant for an Airline called FlightAI. \"\n",
"system_message += \"Give short, courteous answers, no more than 1 sentence. \"\n",
"system_message += \"Always be accurate. If you don't know the answer, say so.\""
"system_message = \"\"\"\n",
"You are a helpful assistant for an Airline called FlightAI.\n",
"Give short, courteous answers, no more than 1 sentence.\n",
"Always be accurate. If you don't know the answer, say so.\n",
"\"\"\""
]
},
{
@@ -71,9 +71,8 @@
"metadata": {},
"outputs": [],
"source": [
"# This function looks rather simpler than the one from my video, because we're taking advantage of the latest Gradio updates\n",
"\n",
"def chat(message, history):\n",
" history = [{\"role\":h[\"role\"], \"content\":h[\"content\"]} for h in history]\n",
" messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages)\n",
" return response.choices[0].message.content\n",
@@ -109,9 +108,9 @@
"ticket_prices = {\"london\": \"$799\", \"paris\": \"$899\", \"tokyo\": \"$1400\", \"berlin\": \"$499\"}\n",
"\n",
"def get_ticket_price(destination_city):\n",
" print(f\"Tool get_ticket_price called for {destination_city}\")\n",
" city = destination_city.lower()\n",
" return ticket_prices.get(city, \"Unknown\")"
" print(f\"Tool called for city {destination_city}\")\n",
" price = ticket_prices.get(destination_city.lower(), \"Unknown ticket price\")\n",
" return f\"The price of a ticket to {destination_city} is {price}\"\n"
]
},
{
@@ -121,7 +120,7 @@
"metadata": {},
"outputs": [],
"source": [
"get_ticket_price(\"Berlin\")"
"get_ticket_price(\"London\")"
]
},
{
@@ -135,7 +134,7 @@
"\n",
"price_function = {\n",
" \"name\": \"get_ticket_price\",\n",
" \"description\": \"Get the price of a return ticket to the destination city. Call this whenever you need to know the ticket price, for example when a customer asks 'How much is a ticket to this city'\",\n",
" \"description\": \"Get the price of a return ticket to the destination city.\",\n",
" \"parameters\": {\n",
" \"type\": \"object\",\n",
" \"properties\": {\n",
@@ -162,6 +161,16 @@
"tools = [{\"type\": \"function\", \"function\": price_function}]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "818b4b2b",
"metadata": {},
"outputs": [],
"source": [
"tools"
]
},
{
"cell_type": "markdown",
"id": "c3d3554f-b4e3-4ce7-af6f-68faa6dd2340",
@@ -184,12 +193,13 @@
"outputs": [],
"source": [
"def chat(message, history):\n",
" history = [{\"role\":h[\"role\"], \"content\":h[\"content\"]} for h in history]\n",
" messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n",
"\n",
" if response.choices[0].finish_reason==\"tool_calls\":\n",
" message = response.choices[0].message\n",
" response, city = handle_tool_call(message)\n",
" response = handle_tool_call(message)\n",
" messages.append(message)\n",
" messages.append(response)\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages)\n",
@@ -208,15 +218,16 @@
"\n",
"def handle_tool_call(message):\n",
" tool_call = message.tool_calls[0]\n",
" arguments = json.loads(tool_call.function.arguments)\n",
" city = arguments.get('destination_city')\n",
" price = get_ticket_price(city)\n",
" response = {\n",
" \"role\": \"tool\",\n",
" \"content\": json.dumps({\"destination_city\": city,\"price\": price}),\n",
" \"tool_call_id\": tool_call.id\n",
" }\n",
" return response, city"
" if tool_call.function.name == \"get_ticket_price\":\n",
" arguments = json.loads(tool_call.function.arguments)\n",
" city = arguments.get('destination_city')\n",
" price_details = get_ticket_price(city)\n",
" response = {\n",
" \"role\": \"tool\",\n",
" \"content\": price_details,\n",
" \"tool_call_id\": tool_call.id\n",
" }\n",
" return response"
]
},
{
@@ -228,11 +239,224 @@
"source": [
"gr.ChatInterface(fn=chat, type=\"messages\").launch()"
]
},
{
"cell_type": "markdown",
"id": "47f30fbe",
"metadata": {},
"source": [
"## Let's make a couple of improvements\n",
"\n",
"Handling multiple tool calls in 1 response\n",
"\n",
"Handling multiple tool calls 1 after another"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b6f5c860",
"metadata": {},
"outputs": [],
"source": [
"def chat(message, history):\n",
" history = [{\"role\":h[\"role\"], \"content\":h[\"content\"]} for h in history]\n",
" messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n",
"\n",
" if response.choices[0].finish_reason==\"tool_calls\":\n",
" message = response.choices[0].message\n",
" responses = handle_tool_calls(message)\n",
" messages.append(message)\n",
" messages.extend(responses)\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages)\n",
" \n",
" return response.choices[0].message.content"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9c46a861",
"metadata": {},
"outputs": [],
"source": [
"def handle_tool_calls(message):\n",
" responses = []\n",
" for tool_call in message.tool_calls:\n",
" if tool_call.function.name == \"get_ticket_price\":\n",
" arguments = json.loads(tool_call.function.arguments)\n",
" city = arguments.get('destination_city')\n",
" price_details = get_ticket_price(city)\n",
" responses.append({\n",
" \"role\": \"tool\",\n",
" \"content\": price_details,\n",
" \"tool_call_id\": tool_call.id\n",
" })\n",
" return responses"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "95f02a4d",
"metadata": {},
"outputs": [],
"source": [
"gr.ChatInterface(fn=chat, type=\"messages\").launch()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cf262abc",
"metadata": {},
"outputs": [],
"source": [
"def chat(message, history):\n",
" history = [{\"role\":h[\"role\"], \"content\":h[\"content\"]} for h in history]\n",
" messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n",
"\n",
" while response.choices[0].finish_reason==\"tool_calls\":\n",
" message = response.choices[0].message\n",
" responses = handle_tool_calls(message)\n",
" messages.append(message)\n",
" messages.extend(responses)\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n",
" \n",
" return response.choices[0].message.content"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "47d50e70",
"metadata": {},
"outputs": [],
"source": [
"import sqlite3\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bb61a45d",
"metadata": {},
"outputs": [],
"source": [
"DB = \"prices.db\"\n",
"\n",
"with sqlite3.connect(DB) as conn:\n",
" cursor = conn.cursor()\n",
" cursor.execute('CREATE TABLE IF NOT EXISTS prices (city TEXT PRIMARY KEY, price REAL)')\n",
" conn.commit()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "12c73b6a",
"metadata": {},
"outputs": [],
"source": [
"def get_ticket_price(city):\n",
" print(f\"DATABASE TOOL CALLED: Getting price for {city}\", flush=True)\n",
" with sqlite3.connect(DB) as conn:\n",
" cursor = conn.cursor()\n",
" cursor.execute('SELECT price FROM prices WHERE city = ?', (city.lower(),))\n",
" result = cursor.fetchone()\n",
" return f\"Ticket price to {city} is ${result[0]}\" if result else \"No price data available for this city\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7cb2e079",
"metadata": {},
"outputs": [],
"source": [
"get_ticket_price(\"London\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "46e43463",
"metadata": {},
"outputs": [],
"source": [
"def set_ticket_price(city, price):\n",
" with sqlite3.connect(DB) as conn:\n",
" cursor = conn.cursor()\n",
" cursor.execute('INSERT INTO prices (city, price) VALUES (?, ?) ON CONFLICT(city) DO UPDATE SET price = ?', (city.lower(), price, price))\n",
" conn.commit()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9185228e",
"metadata": {},
"outputs": [],
"source": [
"ticket_prices = {\"london\":799, \"paris\": 899, \"tokyo\": 1420, \"sydney\": 2999}\n",
"for city, price in ticket_prices.items():\n",
" set_ticket_price(city, price)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cda459b9",
"metadata": {},
"outputs": [],
"source": [
"get_ticket_price(\"Tokyo\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bfbfa251",
"metadata": {},
"outputs": [],
"source": [
"gr.ChatInterface(fn=chat, type=\"messages\").launch()"
]
},
{
"cell_type": "markdown",
"id": "d1a9e9c7",
"metadata": {},
"source": [
"## Exercise\n",
"\n",
"Add a tool to set the price of a ticket!"
]
},
{
"cell_type": "markdown",
"id": "6aeba34c",
"metadata": {},
"source": [
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../assets/business.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#181;\">Business Applications</h2>\n",
" <span style=\"color:#181;\">Hopefully this hardly needs to be stated! You now have the ability to give actions to your LLMs. This Airline Assistant can now do more than answer questions - it could interact with booking APIs to make bookings!</span>\n",
" </td>\n",
" </tr>\n",
"</table>"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": ".venv",
"language": "python",
"name": "python3"
},
@@ -246,7 +470,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.13"
"version": "3.12.9"
}
},
"nbformat": 4,

609
week2/day5.ipynb
View File

@@ -23,7 +23,8 @@
"import json\n",
"from dotenv import load_dotenv\n",
"from openai import OpenAI\n",
"import gradio as gr"
"import gradio as gr\n",
"import sqlite3"
]
},
{
@@ -43,8 +44,10 @@
"else:\n",
" print(\"OpenAI API Key not set\")\n",
" \n",
"MODEL = \"gpt-4o-mini\"\n",
"openai = OpenAI()"
"MODEL = \"gpt-4.1-mini\"\n",
"openai = OpenAI()\n",
"\n",
"DB = \"prices.db\""
]
},
{
@@ -54,83 +57,49 @@
"metadata": {},
"outputs": [],
"source": [
"system_message = \"You are a helpful assistant for an Airline called FlightAI. \"\n",
"system_message += \"Give short, courteous answers, no more than 1 sentence. \"\n",
"system_message += \"Always be accurate. If you don't know the answer, say so.\""
"system_message = \"\"\"\n",
"You are a helpful assistant for an Airline called FlightAI.\n",
"Give short, courteous answers, no more than 1 sentence.\n",
"Always be accurate. If you don't know the answer, say so.\n",
"\"\"\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "61a2a15d-b559-4844-b377-6bd5cb4949f6",
"id": "c3e8173c",
"metadata": {},
"outputs": [],
"source": [
"# This function looks rather simpler than the one from my video, because we're taking advantage of the latest Gradio updates\n",
"\n",
"def chat(message, history):\n",
" messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages)\n",
" return response.choices[0].message.content\n",
"\n",
"gr.ChatInterface(fn=chat, type=\"messages\").launch()"
]
},
{
"cell_type": "markdown",
"id": "36bedabf-a0a7-4985-ad8e-07ed6a55a3a4",
"metadata": {},
"source": [
"## Tools\n",
"\n",
"Tools are an incredibly powerful feature provided by the frontier LLMs.\n",
"\n",
"With tools, you can write a function, and have the LLM call that function as part of its response.\n",
"\n",
"Sounds almost spooky.. we're giving it the power to run code on our machine?\n",
"\n",
"Well, kinda."
"def get_ticket_price(city):\n",
" print(f\"DATABASE TOOL CALLED: Getting price for {city}\", flush=True)\n",
" with sqlite3.connect(DB) as conn:\n",
" cursor = conn.cursor()\n",
" cursor.execute('SELECT price FROM prices WHERE city = ?', (city.lower(),))\n",
" result = cursor.fetchone()\n",
" return f\"Ticket price to {city} is ${result[0]}\" if result else \"No price data available for this city\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0696acb1-0b05-4dc2-80d5-771be04f1fb2",
"id": "03f19289",
"metadata": {},
"outputs": [],
"source": [
"# Let's start by making a useful function\n",
"\n",
"ticket_prices = {\"london\": \"$799\", \"paris\": \"$899\", \"tokyo\": \"$1400\", \"berlin\": \"$499\"}\n",
"\n",
"def get_ticket_price(destination_city):\n",
" print(f\"Tool get_ticket_price called for {destination_city}\")\n",
" city = destination_city.lower()\n",
" return ticket_prices.get(city, \"Unknown\")"
"get_ticket_price(\"Paris\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "80ca4e09-6287-4d3f-997d-fa6afbcf6c85",
"id": "bcfb6523",
"metadata": {},
"outputs": [],
"source": [
"get_ticket_price(\"London\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4afceded-7178-4c05-8fa6-9f2085e6a344",
"metadata": {},
"outputs": [],
"source": [
"# There's a particular dictionary structure that's required to describe our function:\n",
"\n",
"price_function = {\n",
" \"name\": \"get_ticket_price\",\n",
" \"description\": \"Get the price of a return ticket to the destination city. Call this whenever you need to know the ticket price, for example when a customer asks 'How much is a ticket to this city'\",\n",
" \"description\": \"Get the price of a return ticket to the destination city.\",\n",
" \"parameters\": {\n",
" \"type\": \"object\",\n",
" \"properties\": {\n",
@@ -142,52 +111,46 @@
" \"required\": [\"destination_city\"],\n",
" \"additionalProperties\": False\n",
" }\n",
"}"
"}\n",
"tools = [{\"type\": \"function\", \"function\": price_function}]\n",
"tools"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bdca8679-935f-4e7f-97e6-e71a4d4f228c",
"id": "61a2a15d-b559-4844-b377-6bd5cb4949f6",
"metadata": {},
"outputs": [],
"source": [
"# And this is included in a list of tools:\n",
"\n",
"tools = [{\"type\": \"function\", \"function\": price_function}]"
]
},
{
"cell_type": "markdown",
"id": "c3d3554f-b4e3-4ce7-af6f-68faa6dd2340",
"metadata": {},
"source": [
"## Getting OpenAI to use our Tool\n",
"def chat(message, history):\n",
" history = [{\"role\": h[\"role\"], \"content\": h[\"content\"]} for h in history]\n",
" messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages)\n",
" return response.choices[0].message.content\n",
"\n",
"There's some fiddly stuff to allow OpenAI \"to call our tool\"\n",
"\n",
"What we actually do is give the LLM the opportunity to inform us that it wants us to run the tool.\n",
"\n",
"Here's how the new chat function looks:"
"gr.ChatInterface(fn=chat, type=\"messages\").launch()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ce9b0744-9c78-408d-b9df-9f6fd9ed78cf",
"id": "c91d012e",
"metadata": {},
"outputs": [],
"source": [
"def chat(message, history):\n",
" history = [{\"role\":h[\"role\"], \"content\":h[\"content\"]} for h in history]\n",
" messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n",
"\n",
" if response.choices[0].finish_reason==\"tool_calls\":\n",
" while response.choices[0].finish_reason==\"tool_calls\":\n",
" message = response.choices[0].message\n",
" response, city = handle_tool_call(message)\n",
" responses = handle_tool_calls(message)\n",
" messages.append(message)\n",
" messages.append(response)\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages)\n",
" messages.extend(responses)\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n",
" \n",
" return response.choices[0].message.content"
]
@@ -195,35 +158,59 @@
{
"cell_type": "code",
"execution_count": null,
"id": "b0992986-ea09-4912-a076-8e5603ee631f",
"id": "956c3b61",
"metadata": {},
"outputs": [],
"source": [
"# We have to write that function handle_tool_call:\n",
"\n",
"def handle_tool_call(message):\n",
" tool_call = message.tool_calls[0]\n",
" arguments = json.loads(tool_call.function.arguments)\n",
" city = arguments.get('destination_city')\n",
" price = get_ticket_price(city)\n",
" response = {\n",
" \"role\": \"tool\",\n",
" \"content\": json.dumps({\"destination_city\": city,\"price\": price}),\n",
" \"tool_call_id\": tool_call.id\n",
" }\n",
" return response, city"
"def handle_tool_calls(message):\n",
" responses = []\n",
" for tool_call in message.tool_calls:\n",
" if tool_call.function.name == \"get_ticket_price\":\n",
" arguments = json.loads(tool_call.function.arguments)\n",
" city = arguments.get('destination_city')\n",
" price_details = get_ticket_price(city)\n",
" responses.append({\n",
" \"role\": \"tool\",\n",
" \"content\": price_details,\n",
" \"tool_call_id\": tool_call.id\n",
" })\n",
" return responses"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f4be8a71-b19e-4c2f-80df-f59ff2661f14",
"id": "8eca803e",
"metadata": {},
"outputs": [],
"source": [
"gr.ChatInterface(fn=chat, type=\"messages\").launch()"
]
},
{
"cell_type": "markdown",
"id": "b369bf10",
"metadata": {},
"source": [
"## A bit more about what Gradio actually does:\n",
"\n",
"1. Gradio constructs a frontend Svelte app based on our Python description of the UI\n",
"2. Gradio starts a server built upon the Starlette web framework listening on a free port that serves this React app\n",
"3. Gradio creates backend routes for our callbacks, like chat(), which calls our functions\n",
"\n",
"And of course when Gradio generates the frontend app, it ensures that the the Submit button calls the right backend route.\n",
"\n",
"That's it!\n",
"\n",
"It's simple, and it has a resut that feels magical."
]
},
{
"cell_type": "markdown",
"id": "863aac34",
"metadata": {},
"source": []
},
{
"cell_type": "markdown",
"id": "473e5b39-da8f-4db1-83ae-dbaca2e9531e",
@@ -289,414 +276,128 @@
"id": "728a12c5-adc3-415d-bb05-82beb73b079b",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "f4975b87-19e9-4ade-a232-9b809ec75c9a",
"metadata": {},
"source": [
"## Audio (NOTE - Audio is optional for this course - feel free to skip Audio if it causes trouble!)\n",
"\n",
"And let's make a function talker that uses OpenAI's speech model to generate Audio\n",
"\n",
"### Troubleshooting Audio issues\n",
"\n",
"If you have any problems running this code below (like a FileNotFound error, or a warning of a missing package), you may need to install FFmpeg, a very popular audio utility.\n",
"\n",
"**For PC Users**\n",
"\n",
"Detailed instructions are [here](https://chatgpt.com/share/6724efee-6b0c-8012-ac5e-72e2e3885905) and summary instructions:\n",
"\n",
"1. Download FFmpeg from the official website: https://ffmpeg.org/download.html\n",
"\n",
"2. Extract the downloaded files to a location on your computer (e.g., `C:\\ffmpeg`)\n",
"\n",
"3. Add the FFmpeg bin folder to your system PATH:\n",
"- Right-click on 'This PC' or 'My Computer' and select 'Properties'\n",
"- Click on 'Advanced system settings'\n",
"- Click on 'Environment Variables'\n",
"- Under 'System variables', find and edit 'Path'\n",
"- Add a new entry with the path to your FFmpeg bin folder (e.g., `C:\\ffmpeg\\bin`)\n",
"- Restart your command prompt, and within Jupyter Lab do Kernel -> Restart kernel, to pick up the changes\n",
"\n",
"4. Open a new command prompt and run this to make sure it's installed OK\n",
"`ffmpeg -version`\n",
"\n",
"**For Mac Users**\n",
"\n",
"1. Install homebrew if you don't have it already by running this in a Terminal window and following any instructions: \n",
"`/bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\"`\n",
"\n",
"2. Then install FFmpeg with `brew install ffmpeg`\n",
"\n",
"3. Verify your installation with `ffmpeg -version` and if everything is good, within Jupyter Lab do Kernel -> Restart kernel to pick up the changes\n",
"\n",
"Message me or email me at ed@edwarddonner.com with any problems!"
]
},
{
"cell_type": "markdown",
"id": "4cc90e80-c96e-4dd4-b9d6-386fe2b7e797",
"metadata": {},
"source": [
"## To check you now have ffmpeg and can access it here\n",
"\n",
"Excecute the next cell to see if you get a version number. (Putting an exclamation mark before something in Jupyter Lab tells it to run it as a terminal command rather than python code).\n",
"\n",
"If this doesn't work, you may need to actually save and close down your Jupyter lab, and start it again from a new Terminal window (Mac) or Anaconda prompt (PC), remembering to activate the llms environment. This ensures you pick up ffmpeg.\n",
"\n",
"And if that doesn't work, please contact me!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7b3be0fb-1d34-4693-ab6f-dbff190afcd7",
"metadata": {},
"outputs": [],
"source": [
"!ffmpeg -version\n",
"!ffprobe -version\n",
"!ffplay -version"
]
},
{
"cell_type": "markdown",
"id": "d91d3f8f-e505-4e3c-a87c-9e42ed823db6",
"metadata": {},
"source": [
"# For Mac users - and possibly many PC users too\n",
"\n",
"This version should work fine for you. It might work for Windows users too, but you might get a Permissions error writing to a temp file. If so, see the next section!\n",
"\n",
"As always, if you have problems, please contact me! (You could also comment out the audio talker() in the later code if you're less interested in audio generation)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ffbfe93b-5e86-4e68-ba71-b301cd5230db",
"metadata": {},
"outputs": [],
"source": [
"from pydub import AudioSegment\n",
"from pydub.playback import play\n",
"\n",
"def talker(message):\n",
" response = openai.audio.speech.create(\n",
" model=\"tts-1\",\n",
" voice=\"onyx\", # Also, try replacing onyx with alloy\n",
" model=\"gpt-4o-mini-tts\",\n",
" voice=\"onyx\", # Also, try replacing onyx with alloy or coral\n",
" input=message\n",
" )\n",
" \n",
" audio_stream = BytesIO(response.content)\n",
" audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n",
" play(audio)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b88d775d-d357-4292-a1ad-5dc5ed567281",
"metadata": {},
"outputs": [],
"source": [
"talker(\"Well, hi there\")"
" return response.content"
]
},
{
"cell_type": "markdown",
"id": "ad89a9bd-bb1e-4bbb-a49a-83af5f500c24",
"id": "3bc7580b",
"metadata": {},
"source": [
"# For Windows users (or any Mac users with problems above)\n",
"## Let's bring this home:\n",
"\n",
"## First try the Mac version above, but if you get a permissions error writing to a temp file, then this code should work instead.\n",
"\n",
"A collaboration between students Mark M. and Patrick H. and Claude got this resolved!\n",
"\n",
"Below are 4 variations - hopefully one of them will work on your PC. If not, message me please!\n",
"\n",
"And for Mac people - all 3 of the below work on my Mac too - please try these if the Mac version gave you problems.\n",
"\n",
"## PC Variation 1"
"1. A multi-modal AI assistant with image and audio generation\n",
"2. Tool callling with database lookup\n",
"3. A step towards an Agentic workflow\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d104b96a-02ca-4159-82fe-88e0452aa479",
"metadata": {},
"outputs": [],
"source": [
"import base64\n",
"from io import BytesIO\n",
"from PIL import Image\n",
"from IPython.display import Audio, display\n",
"\n",
"def talker(message):\n",
" response = openai.audio.speech.create(\n",
" model=\"tts-1\",\n",
" voice=\"onyx\",\n",
" input=message)\n",
"\n",
" audio_stream = BytesIO(response.content)\n",
" output_filename = \"output_audio.mp3\"\n",
" with open(output_filename, \"wb\") as f:\n",
" f.write(audio_stream.read())\n",
"\n",
" # Play the generated audio\n",
" display(Audio(output_filename, autoplay=True))\n",
"\n",
"talker(\"Well, hi there\")"
]
},
{
"cell_type": "markdown",
"id": "3a5d11f4-bbd3-43a1-904d-f684eb5f3e3a",
"metadata": {},
"source": [
"## PC Variation 2"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d59c8ebd-79c5-498a-bdf2-3a1c50d91aa0",
"metadata": {},
"outputs": [],
"source": [
"import tempfile\n",
"import subprocess\n",
"from io import BytesIO\n",
"from pydub import AudioSegment\n",
"import time\n",
"\n",
"def play_audio(audio_segment):\n",
" temp_dir = tempfile.gettempdir()\n",
" temp_path = os.path.join(temp_dir, \"temp_audio.wav\")\n",
" try:\n",
" audio_segment.export(temp_path, format=\"wav\")\n",
" time.sleep(3) # Student Dominic found that this was needed. You could also try commenting out to see if not needed on your PC\n",
" subprocess.call([\n",
" \"ffplay\",\n",
" \"-nodisp\",\n",
" \"-autoexit\",\n",
" \"-hide_banner\",\n",
" temp_path\n",
" ], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n",
" finally:\n",
" try:\n",
" os.remove(temp_path)\n",
" except Exception:\n",
" pass\n",
" \n",
"def talker(message):\n",
" response = openai.audio.speech.create(\n",
" model=\"tts-1\",\n",
" voice=\"onyx\", # Also, try replacing onyx with alloy\n",
" input=message\n",
" )\n",
" audio_stream = BytesIO(response.content)\n",
" audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n",
" play_audio(audio)\n",
"\n",
"talker(\"Well hi there\")"
]
},
{
"cell_type": "markdown",
"id": "96f90e35-f71e-468e-afea-07b98f74dbcf",
"metadata": {},
"source": [
"## PC Variation 3"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8597c7f8-7b50-44ad-9b31-db12375cd57b",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from pydub import AudioSegment\n",
"from pydub.playback import play\n",
"from io import BytesIO\n",
"\n",
"def talker(message):\n",
" # Set a custom directory for temporary files on Windows\n",
" custom_temp_dir = os.path.expanduser(\"~/Documents/temp_audio\")\n",
" os.environ['TEMP'] = custom_temp_dir # You can also use 'TMP' if necessary\n",
" \n",
" # Create the folder if it doesn't exist\n",
" if not os.path.exists(custom_temp_dir):\n",
" os.makedirs(custom_temp_dir)\n",
" \n",
" response = openai.audio.speech.create(\n",
" model=\"tts-1\",\n",
" voice=\"onyx\", # Also, try replacing onyx with alloy\n",
" input=message\n",
" )\n",
" \n",
" audio_stream = BytesIO(response.content)\n",
" audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n",
"\n",
" play(audio)\n",
"\n",
"talker(\"Well hi there\")"
]
},
{
"cell_type": "markdown",
"id": "e821224c-b069-4f9b-9535-c15fdb0e411c",
"metadata": {},
"source": [
"## PC Variation 4\n",
"\n",
"### Let's try a completely different sound library\n",
"\n",
"First run the next cell to install a new library, then try the cell below it."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "69d3c0d9-afcc-49e3-b829-9c9869d8b472",
"metadata": {},
"outputs": [],
"source": [
"!pip install simpleaudio"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "28f9cc99-36b7-4554-b3f4-f2012f614a13",
"metadata": {},
"outputs": [],
"source": [
"from pydub import AudioSegment\n",
"from io import BytesIO\n",
"import tempfile\n",
"import os\n",
"import simpleaudio as sa\n",
"\n",
"def talker(message):\n",
" response = openai.audio.speech.create(\n",
" model=\"tts-1\",\n",
" voice=\"onyx\", # Also, try replacing onyx with alloy\n",
" input=message\n",
" )\n",
" \n",
" audio_stream = BytesIO(response.content)\n",
" audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n",
"\n",
" # Create a temporary file in a folder where you have write permissions\n",
" with tempfile.NamedTemporaryFile(suffix=\".wav\", delete=False, dir=os.path.expanduser(\"~/Documents\")) as temp_audio_file:\n",
" temp_file_name = temp_audio_file.name\n",
" audio.export(temp_file_name, format=\"wav\")\n",
" \n",
" # Load and play audio using simpleaudio\n",
" wave_obj = sa.WaveObject.from_wave_file(temp_file_name)\n",
" play_obj = wave_obj.play()\n",
" play_obj.wait_done() # Wait for playback to finish\n",
"\n",
" # Clean up the temporary file afterward\n",
" os.remove(temp_file_name)\n",
" \n",
"talker(\"Well hi there\")"
]
},
{
"cell_type": "markdown",
"id": "7986176b-cd04-495f-a47f-e057b0e462ed",
"metadata": {},
"source": [
"## PC Users - if none of those 4 variations worked!\n",
"\n",
"Please get in touch with me. I'm sorry this is causing problems! We'll figure it out.\n",
"\n",
"Alternatively: playing audio from your PC isn't super-critical for this course, and you can feel free to focus on image generation and skip audio for now, or come back to it later."
]
},
{
"cell_type": "markdown",
"id": "1d48876d-c4fa-46a8-a04f-f9fadf61fb0d",
"metadata": {},
"source": [
"# Our Agent Framework\n",
"\n",
"The term 'Agentic AI' and Agentization is an umbrella term that refers to a number of techniques, such as:\n",
"\n",
"1. Breaking a complex problem into smaller steps, with multiple LLMs carrying out specialized tasks\n",
"2. The ability for LLMs to use Tools to give them additional capabilities\n",
"3. The 'Agent Environment' which allows Agents to collaborate\n",
"4. An LLM can act as the Planner, dividing bigger tasks into smaller ones for the specialists\n",
"5. The concept of an Agent having autonomy / agency, beyond just responding to a prompt - such as Memory\n",
"\n",
"We're showing 1 and 2 here, and to a lesser extent 3 and 5. In week 8 we will do the lot!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ba820c95-02f5-499e-8f3c-8727ee0a6c0c",
"id": "b119ed1b",
"metadata": {},
"outputs": [],
"source": [
"def chat(history):\n",
" history = [{\"role\":h[\"role\"], \"content\":h[\"content\"]} for h in history]\n",
" messages = [{\"role\": \"system\", \"content\": system_message}] + history\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n",
" cities = []\n",
" image = None\n",
" \n",
" if response.choices[0].finish_reason==\"tool_calls\":\n",
"\n",
" while response.choices[0].finish_reason==\"tool_calls\":\n",
" message = response.choices[0].message\n",
" response, city = handle_tool_call(message)\n",
" responses, cities = handle_tool_calls_and_return_cities(message)\n",
" messages.append(message)\n",
" messages.append(response)\n",
" image = artist(city)\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages)\n",
" \n",
" messages.extend(responses)\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n",
"\n",
" reply = response.choices[0].message.content\n",
" history += [{\"role\":\"assistant\", \"content\":reply}]\n",
"\n",
" # Comment out or delete the next line if you'd rather skip Audio for now..\n",
" talker(reply)\n",
" voice = talker(reply)\n",
"\n",
" if cities:\n",
" image = artist(cities[0])\n",
" \n",
" return history, image"
" return history, voice, image\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f38d0d27-33bf-4992-a2e5-5dbed973cde7",
"id": "5846bc77",
"metadata": {},
"outputs": [],
"source": [
"# More involved Gradio code as we're not using the preset Chat interface!\n",
"# Passing in inbrowser=True in the last line will cause a Gradio window to pop up immediately.\n",
"def handle_tool_calls_and_return_cities(message):\n",
" responses = []\n",
" cities = []\n",
" for tool_call in message.tool_calls:\n",
" if tool_call.function.name == \"get_ticket_price\":\n",
" arguments = json.loads(tool_call.function.arguments)\n",
" city = arguments.get('destination_city')\n",
" cities.append(city)\n",
" price_details = get_ticket_price(city)\n",
" responses.append({\n",
" \"role\": \"tool\",\n",
" \"content\": price_details,\n",
" \"tool_call_id\": tool_call.id\n",
" })\n",
" return responses, cities"
]
},
{
"cell_type": "markdown",
"id": "6e520161",
"metadata": {},
"source": [
"## The 3 types of Gradio UI\n",
"\n",
"`gr.Interface` is for standard, simple UIs\n",
"\n",
"`gr.ChatInterface` is for standard ChatBot UIs\n",
"\n",
"`gr.Blocks` is for custom UIs where you control the components and the callbacks"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9f250915",
"metadata": {},
"outputs": [],
"source": [
"# Callbacks (along with the chat() function above)\n",
"\n",
"def put_message_in_chatbot(message, history):\n",
" return \"\", history + [{\"role\":\"user\", \"content\":message}]\n",
"\n",
"# UI definition\n",
"\n",
"with gr.Blocks() as ui:\n",
" with gr.Row():\n",
" chatbot = gr.Chatbot(height=500, type=\"messages\")\n",
" image_output = gr.Image(height=500)\n",
" image_output = gr.Image(height=500, interactive=False)\n",
" with gr.Row():\n",
" entry = gr.Textbox(label=\"Chat with our AI Assistant:\")\n",
" audio_output = gr.Audio(autoplay=True)\n",
" with gr.Row():\n",
" clear = gr.Button(\"Clear\")\n",
" message = gr.Textbox(label=\"Chat with our AI Assistant:\")\n",
"\n",
" def do_entry(message, history):\n",
" history += [{\"role\":\"user\", \"content\":message}]\n",
" return \"\", history\n",
"# Hooking up events to callbacks\n",
"\n",
" entry.submit(do_entry, inputs=[entry, chatbot], outputs=[entry, chatbot]).then(\n",
" chat, inputs=chatbot, outputs=[chatbot, image_output]\n",
" message.submit(put_message_in_chatbot, inputs=[message, chatbot], outputs=[message, chatbot]).then(\n",
" chat, inputs=chatbot, outputs=[chatbot, audio_output, image_output]\n",
" )\n",
" clear.click(lambda: None, inputs=None, outputs=chatbot, queue=False)\n",
"\n",
"ui.launch(inbrowser=True)"
"ui.launch(inbrowser=True, auth=(\"ed\", \"bananas\"))"
]
},
{
@@ -719,7 +420,7 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../thankyou.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/thankyou.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#090;\">I have a special request for you</h2>\n",
@@ -734,7 +435,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": ".venv",
"language": "python",
"name": "python3"
},
@@ -748,7 +449,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.13"
"version": "3.12.9"
}
},
"nbformat": 4,

4459
week2/hamlet.txt Normal file
View File

File diff suppressed because it is too large Load Diff

37
week2/scraper.py Normal file
View File

@@ -0,0 +1,37 @@
from bs4 import BeautifulSoup
import requests
# Standard headers to fetch a website
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36"
}
def fetch_website_contents(url):
"""
Return the title and contents of the website at the given url;
truncate to 2,000 characters as a sensible limit
"""
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, "html.parser")
title = soup.title.string if soup.title else "No title found"
if soup.body:
for irrelevant in soup.body(["script", "style", "img", "input"]):
irrelevant.decompose()
text = soup.body.get_text(separator="\n", strip=True)
else:
text = ""
return (title + "\n\n" + text)[:2_000]
def fetch_website_links(url):
"""
Return the links on the webiste at the given url
I realize this is inefficient as we're parsing twice! This is to keep the code in the lab simple.
Feel free to use a class and optimize it!
"""
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, "html.parser")
links = [link.get("href") for link in soup.find_all("a")]
return [link for link in links if link]

6
week3/day1.ipynb
View File

@@ -16,6 +16,12 @@
"https://colab.research.google.com/drive/1DjcrYDZldAXKJ08x1uYIVCtItoLPk1Wr?usp=sharing"
]
},
{
"cell_type": "markdown",
"id": "0654c6f0",
"metadata": {},
"source": []
},
{
"cell_type": "code",
"execution_count": null,

31
week3/day5.ipynb
View File

@@ -18,18 +18,45 @@
"This should run nicely on a low-cost or free T4 box."
]
},
{
"cell_type": "markdown",
"id": "501aa674",
"metadata": {},
"source": [
"### BUT FIRST - Something cool - really showing you how \"model inference\" works via OpenAI"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e9289ba7-200c-43a9-b67a-c5ce826c9537",
"metadata": {},
"outputs": [],
"source": [
"from visualizer import TokenPredictor, create_token_graph, visualize_predictions\n",
"\n",
"message = \"In one sentence, describe the color orange to someone who has never been able to see\"\n",
"model_name = \"gpt-4.1-mini\"\n",
"\n",
"predictor = TokenPredictor(model_name)\n",
"predictions = predictor.predict_tokens(message)\n",
"G = create_token_graph(model_name, predictions)\n",
"plt = visualize_predictions(G)\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "540a8255",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": ".venv",
"language": "python",
"name": "python3"
},
@@ -43,7 +70,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.11"
"version": "3.12.9"
}
},
"nbformat": 4,

157
week3/visualizer.py Normal file
View File

@@ -0,0 +1,157 @@
import networkx as nx
import matplotlib.pyplot as plt
from typing import List, Dict
import math
from openai import OpenAI
from dotenv import load_dotenv
load_dotenv(override=True)
class TokenPredictor:
def __init__(self, model_name: str):
self.client = OpenAI()
self.messages = []
self.predictions = []
self.model_name = model_name
def predict_tokens(self, prompt: str, max_tokens: int = 100) -> List[Dict]:
"""
Generate text token by token and track prediction probabilities.
Returns list of predictions with top token and alternatives.
"""
response = self.client.chat.completions.create(
model=self.model_name,
messages=[{"role": "user", "content": prompt}],
max_tokens=max_tokens,
temperature=0, # Use temperature 0 for deterministic output
logprobs=True,
seed=42,
top_logprobs=3, # Get top 3 token predictions
stream=True, # Stream the response
)
predictions = []
for chunk in response:
if chunk.choices[0].delta.content:
token = chunk.choices[0].delta.content
logprobs = chunk.choices[0].logprobs.content[0].top_logprobs
logprob_dict = {item.token: item.logprob for item in logprobs}
# Get top predicted token and probability
top_token = token
top_prob = logprob_dict[token]
# Get alternative predictions
alternatives = []
for alt_token, alt_prob in logprob_dict.items():
if alt_token != token:
alternatives.append((alt_token, math.exp(alt_prob)))
alternatives.sort(key=lambda x: x[1], reverse=True)
prediction = {
"token": top_token,
"probability": math.exp(top_prob),
"alternatives": alternatives[:2], # Keep top 2 alternatives
}
predictions.append(prediction)
return predictions
def create_token_graph(model_name: str, predictions: List[Dict]) -> nx.DiGraph:
"""
Create a directed graph showing token predictions and alternatives.
"""
G = nx.DiGraph()
G.add_node("START", token=model_name, prob="START", color="lightgreen", size=4000)
# First, create all main token nodes in sequence
for i, pred in enumerate(predictions):
token_id = f"t{i}"
G.add_node(
token_id,
token=pred["token"],
prob=f"{pred['probability'] * 100:.1f}%",
color="lightblue",
size=6000,
)
if i == 0:
G.add_edge("START", token_id)
else:
G.add_edge(f"t{i - 1}", token_id)
# Then add alternative nodes with a different y-position
last_id = None
for i, pred in enumerate(predictions):
parent_token = "START" if i == 0 else f"t{i - 1}"
# Add alternative token nodes slightly below main sequence
for j, (alt_token, alt_prob) in enumerate(pred["alternatives"]):
alt_id = f"t{i}_alt{j}"
G.add_node(
alt_id, token=alt_token, prob=f"{alt_prob * 100:.1f}%", color="lightgray", size=6000
)
# Add edge from main token to its alternatives only
G.add_edge(parent_token, alt_id)
last_id = parent_token
G.add_node("END", token="END", prob="100%", color="red", size=6000)
G.add_edge(last_id, "END")
return G
def visualize_predictions(G: nx.DiGraph, figsize=(14, 80)):
"""
Visualize the token prediction graph with vertical layout and alternating alternatives.
"""
plt.figure(figsize=figsize)
# Create custom positioning for nodes
pos = {}
spacing_y = 5 # Vertical spacing between main tokens
spacing_x = 5 # Horizontal spacing for alternatives
# Position main token nodes in a vertical line
main_nodes = [n for n in G.nodes() if "_alt" not in n]
for i, node in enumerate(main_nodes):
pos[node] = (0, -i * spacing_y) # Center main tokens vertically
# Position alternative nodes to left and right of main tokens
for node in G.nodes():
if "_alt" in node:
main_token = node.split("_")[0]
alt_num = int(node.split("_alt")[1])
if main_token in pos:
# Place first alternative to left, second to right
x_offset = -spacing_x if alt_num == 0 else spacing_x
pos[node] = (x_offset, pos[main_token][1] + 0.05)
# Draw nodes
node_colors = [G.nodes[node]["color"] for node in G.nodes()]
node_sizes = [G.nodes[node]["size"] for node in G.nodes()]
nx.draw_networkx_nodes(G, pos, node_color=node_colors, node_size=node_sizes)
# Draw all edges as straight lines
nx.draw_networkx_edges(G, pos, edge_color="gray", arrows=True, arrowsize=20, alpha=0.7)
# Add labels with token and probability
labels = {node: f"{G.nodes[node]['token']}\n{G.nodes[node]['prob']}" for node in G.nodes()}
nx.draw_networkx_labels(G, pos, labels, font_size=14)
plt.title("Token prediction.")
plt.axis("off")
# Adjust plot limits to ensure all nodes are visible
margin = 8
x_values = [x for x, y in pos.values()]
y_values = [y for x, y in pos.values()]
plt.xlim(min(x_values) - margin, max(x_values) + margin)
plt.ylim(min(y_values) - margin, max(y_values) + margin)
# plt.tight_layout()
return plt

655
week4/day3.ipynb
View File

@@ -18,18 +18,15 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#f71;\">Reminder: fetch latest code</h2>\n",
" <span style=\"color:#f71;\">I'm continually improving these labs, adding more examples and exercises.\n",
" At the start of each week, it's worth checking you have the latest code.<br/>\n",
" First do a <a href=\"https://chatgpt.com/share/6734e705-3270-8012-a074-421661af6ba9\">git pull and merge your changes as needed</a>. Any problems? Try asking ChatGPT to clarify how to merge - or contact me!<br/><br/>\n",
" After you've pulled the code, from the llm_engineering directory, in an Anaconda prompt (PC) or Terminal (Mac), run:<br/>\n",
" <code>conda env update --f environment.yml --prune</code><br/>\n",
" Or if you used virtualenv rather than Anaconda, then run this from your activated environment in a Powershell (PC) or Terminal (Mac):<br/>\n",
" <code>pip install -r requirements.txt</code>\n",
" <br/>Then restart the kernel (Kernel menu >> Restart Kernel and Clear Outputs Of All Cells) to pick up the changes.\n",
" After you've pulled the code, from the llm_engineering directory, in a Cursor Terminal, run:<br/>\n",
" <code>uv sync</code><br/>\n",
" </span>\n",
" </td>\n",
" </tr>\n",
@@ -44,12 +41,12 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h1 style=\"color:#900;\">Important Note</h1>\n",
" <span style=\"color:#900;\">\n",
" In this lab, I use GPT-4o and Claude-3.5-Sonnet, which are the slightly higher priced models. The costs are still low, but if you'd prefer to keep costs ultra low, please make the suggested switches to the models (3 cells down from here).\n",
" In this lab, I use high end models GPT 5, Claude 4.5 Sonnet, Gemini 2.5 Pro, Grok 4, which are the slightly higher priced models. The costs are still low, but if you'd prefer to keep costs ultra low, please pick lower cost models like gpt-5-nano.\n",
" </span>\n",
" </td>\n",
" </tr>\n",
@@ -66,15 +63,10 @@
"# imports\n",
"\n",
"import os\n",
"import io\n",
"import sys\n",
"from dotenv import load_dotenv\n",
"from openai import OpenAI\n",
"import google.generativeai\n",
"import anthropic\n",
"from IPython.display import Markdown, display, update_display\n",
"import gradio as gr\n",
"import subprocess"
"import subprocess\n",
"from IPython.display import Markdown, display"
]
},
{
@@ -84,11 +76,51 @@
"metadata": {},
"outputs": [],
"source": [
"# environment\n",
"\n",
"load_dotenv(override=True)\n",
"os.environ['OPENAI_API_KEY'] = os.getenv('OPENAI_API_KEY', 'your-key-if-not-using-env')\n",
"os.environ['ANTHROPIC_API_KEY'] = os.getenv('ANTHROPIC_API_KEY', 'your-key-if-not-using-env')"
"openai_api_key = os.getenv('OPENAI_API_KEY')\n",
"anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')\n",
"google_api_key = os.getenv('GOOGLE_API_KEY')\n",
"grok_api_key = os.getenv('GROK_API_KEY')\n",
"\n",
"if openai_api_key:\n",
" print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n",
"else:\n",
" print(\"OpenAI API Key not set\")\n",
" \n",
"if anthropic_api_key:\n",
" print(f\"Anthropic API Key exists and begins {anthropic_api_key[:7]}\")\n",
"else:\n",
" print(\"Anthropic API Key not set (and this is optional)\")\n",
"\n",
"if google_api_key:\n",
" print(f\"Google API Key exists and begins {google_api_key[:2]}\")\n",
"else:\n",
" print(\"Google API Key not set (and this is optional)\")\n",
"\n",
"if grok_api_key:\n",
" print(f\"Grok API Key exists and begins {grok_api_key[:4]}\")\n",
"else:\n",
" print(\"Grok API Key not set (and this is optional)\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "59863df1",
"metadata": {},
"outputs": [],
"source": [
"# Connect to client libraries\n",
"\n",
"openai = OpenAI()\n",
"\n",
"anthropic_url = \"https://api.anthropic.com/v1/\"\n",
"gemini_url = \"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
"grok_url = \"https://api.x.ai/v1\"\n",
"\n",
"anthropic = OpenAI(api_key=anthropic_api_key, base_url=anthropic_url)\n",
"gemini = OpenAI(api_key=google_api_key, base_url=gemini_url)\n",
"grok = OpenAI(api_key=grok_api_key, base_url=grok_url)"
]
},
{
@@ -98,17 +130,110 @@
"metadata": {},
"outputs": [],
"source": [
"# initialize\n",
"# NOTE - option to use ultra-low cost models by uncommenting last 2 lines\n",
"\n",
"openai = OpenAI()\n",
"claude = anthropic.Anthropic()\n",
"OPENAI_MODEL = \"gpt-4o\"\n",
"CLAUDE_MODEL = \"claude-3-5-sonnet-20240620\"\n",
"OPENAI_MODEL = \"gpt-5\"\n",
"CLAUDE_MODEL = \"claude-sonnet-4-5-20250929\"\n",
"GROK_MODEL = \"grok-4\"\n",
"GEMINI_MODEL = \"gemini-2.5-pro\"\n",
"\n",
"# Want to keep costs ultra-low? Uncomment these lines:\n",
"# OPENAI_MODEL = \"gpt-4o-mini\"\n",
"# CLAUDE_MODEL = \"claude-3-haiku-20240307\""
"\n",
"# OPENAI_MODEL = \"gpt-5-nano\"\n",
"# CLAUDE_MODEL = \"claude-3-5-haiku-latest\"\n",
"# GROK_MODEL = \"grok-4-fast-non-reasoning\"\n",
"# GEMINI_MODEL = \"gemini-2.5-flash-lite\""
]
},
{
"cell_type": "markdown",
"id": "7eab38a7",
"metadata": {},
"source": [
"## PLEASE NOTE:\n",
"\n",
"We will be writing a solution to convert Python into efficient, optimized C++ code for your machine, which can be compiled to native machine code and executed.\n",
"\n",
"It is not necessary for you to execute the code yourself - that's not the point of the exercise!\n",
"\n",
"But if you would like to (because it's satisfying!) then I'm including the steps here. Very optional!\n",
"\n",
"As an alternative, I'll also show you a website where you can run the C++ code."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8a2fbb68",
"metadata": {},
"outputs": [],
"source": [
"from system_info import retrieve_system_info\n",
"\n",
"system_info = retrieve_system_info()\n",
"system_info"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c6d29a5f",
"metadata": {},
"outputs": [],
"source": [
"message = f\"\"\"\n",
"Here is a report of the system information for my computer.\n",
"I want to run a C++ compiler to compile a single C++ file called main.cpp and then execute it in the simplest way possible.\n",
"Please reply with whether I need to install any C++ compiler to do this. If so, please provide the simplest step by step instructions to do so.\n",
"\n",
"If I'm already set up to compile C++ code, then I'd like to run something like this in Python to compile and execute the code:\n",
"```python\n",
"compile_command = # something here - to achieve the fastest possible runtime performance\n",
"compile_result = subprocess.run(compile_command, check=True, text=True, capture_output=True)\n",
"run_command = # something here\n",
"run_result = subprocess.run(run_command, check=True, text=True, capture_output=True)\n",
"return run_result.stdout\n",
"```\n",
"Please tell me exactly what I should use for the compile_command and run_command.\n",
"\n",
"System information:\n",
"{system_info}\n",
"\"\"\"\n",
"\n",
"response = openai.chat.completions.create(model=OPENAI_MODEL, messages=[{\"role\": \"user\", \"content\": message}])\n",
"display(Markdown(response.choices[0].message.content))\n",
" "
]
},
{
"cell_type": "markdown",
"id": "81e92c12",
"metadata": {},
"source": [
"## If you need to install something\n",
"\n",
"If you would like to, please follow GPTs instructions! Then rerun the analysis afterwards (you might need to Restart the notebook) to confirm you're set.\n",
"\n",
"You should now be equipped with the command to compile the code, and the command to run it!\n",
"\n",
"Enter that in the cell below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d734a634",
"metadata": {},
"outputs": [],
"source": [
"compile_command = [\"clang++\", \"-std=c++17\", \"-Ofast\", \"-mcpu=native\", \"-flto=thin\", \"-fvisibility=hidden\", \"-DNDEBUG\", \"main.cpp\", \"-o\", \"main\"]\n",
"run_command = [\"./main\"]"
]
},
{
"cell_type": "markdown",
"id": "f0b0a437",
"metadata": {},
"source": [
"## And now, on with the main task"
]
},
{
@@ -118,9 +243,26 @@
"metadata": {},
"outputs": [],
"source": [
"system_message = \"You are an assistant that reimplements Python code in high performance C++ for an M1 Mac. \"\n",
"system_message += \"Respond only with C++ code; use comments sparingly and do not provide any explanation other than occasional comments. \"\n",
"system_message += \"The C++ response needs to produce an identical output in the fastest possible time.\""
"system_prompt = \"\"\"\n",
"Your task is to convert Python code into high performance C++ code.\n",
"Respond only with C++ code. Do not provide any explanation other than occasional comments.\n",
"The C++ response needs to produce an identical output in the fastest possible time.\n",
"\"\"\"\n",
"\n",
"def user_prompt_for(python):\n",
" return f\"\"\"\n",
"Port this Python code to C++ with the fastest possible implementation that produces identical output in the least time.\n",
"The system information is:\n",
"{system_info}\n",
"Your response will be written to a file called main.cpp and then compiled and executed; the compilation command is:\n",
"{compile_command}\n",
"Respond only with C++ code.\n",
"Python code to port:\n",
"\n",
"```python\n",
"{python}\n",
"```\n",
"\"\"\""
]
},
{
@@ -130,12 +272,12 @@
"metadata": {},
"outputs": [],
"source": [
"def user_prompt_for(python):\n",
" user_prompt = \"Rewrite this Python code in C++ with the fastest possible implementation that produces identical output in the least time. \"\n",
" user_prompt += \"Respond only with C++ code; do not explain your work other than a few comments. \"\n",
" user_prompt += \"Pay attention to number types to ensure no int overflows. Remember to #include all necessary C++ packages such as iomanip.\\n\\n\"\n",
" user_prompt += python\n",
" return user_prompt"
"def messages_for(python):\n",
" return [\n",
" {\"role\": \"system\", \"content\": system_prompt},\n",
" {\"role\": \"user\", \"content\": user_prompt_for(python)}\n",
" ]\n",
" "
]
},
{
@@ -145,26 +287,9 @@
"metadata": {},
"outputs": [],
"source": [
"def messages_for(python):\n",
" return [\n",
" {\"role\": \"system\", \"content\": system_message},\n",
" {\"role\": \"user\", \"content\": user_prompt_for(python)}\n",
" ]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "71e1ba8c-5b05-4726-a9f3-8d8c6257350b",
"metadata": {},
"outputs": [],
"source": [
"# write to a file called optimized.cpp\n",
"\n",
"def write_output(cpp):\n",
" code = cpp.replace(\"```cpp\",\"\").replace(\"```\",\"\")\n",
" with open(\"optimized.cpp\", \"w\") as f:\n",
" f.write(code)"
" with open(\"main.cpp\", \"w\", encoding=\"utf-8\") as f:\n",
" f.write(cpp)"
]
},
{
@@ -174,35 +299,11 @@
"metadata": {},
"outputs": [],
"source": [
"def optimize_gpt(python): \n",
" stream = openai.chat.completions.create(model=OPENAI_MODEL, messages=messages_for(python), stream=True)\n",
" reply = \"\"\n",
" for chunk in stream:\n",
" fragment = chunk.choices[0].delta.content or \"\"\n",
" reply += fragment\n",
" print(fragment, end='', flush=True)\n",
" write_output(reply)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7cd84ad8-d55c-4fe0-9eeb-1895c95c4a9d",
"metadata": {},
"outputs": [],
"source": [
"def optimize_claude(python):\n",
" result = claude.messages.stream(\n",
" model=CLAUDE_MODEL,\n",
" max_tokens=2000,\n",
" system=system_message,\n",
" messages=[{\"role\": \"user\", \"content\": user_prompt_for(python)}],\n",
" )\n",
" reply = \"\"\n",
" with result as stream:\n",
" for text in stream.text_stream:\n",
" reply += text\n",
" print(text, end=\"\", flush=True)\n",
"def port(client, model, python):\n",
" reasoning_effort = \"high\" if 'gpt' in model else None\n",
" response = client.chat.completions.create(model=model, messages=messages_for(python), reasoning_effort=reasoning_effort)\n",
" reply = response.choices[0].message.content\n",
" reply = reply.replace('```cpp','').replace('```','')\n",
" write_output(reply)"
]
},
@@ -226,7 +327,7 @@
" return result\n",
"\n",
"start_time = time.time()\n",
"result = calculate(100_000_000, 4, 1) * 4\n",
"result = calculate(200_000_000, 4, 1) * 4\n",
"end_time = time.time()\n",
"\n",
"print(f\"Result: {result:.12f}\")\n",
@@ -241,7 +342,19 @@
"metadata": {},
"outputs": [],
"source": [
"exec(pi)"
"def run_python(code):\n",
" globals = {\"__builtins__\": __builtins__}\n",
" exec(code, globals)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7faa90da",
"metadata": {},
"outputs": [],
"source": [
"run_python(pi)"
]
},
{
@@ -251,17 +364,7 @@
"metadata": {},
"outputs": [],
"source": [
"optimize_gpt(pi)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bf26ee95-0c77-491d-9a91-579a1e96a8a3",
"metadata": {},
"outputs": [],
"source": [
"exec(pi)"
"port(openai, OPENAI_MODEL, pi)"
]
},
{
@@ -271,18 +374,12 @@
"source": [
"# Compiling C++ and executing\n",
"\n",
"This next cell contains the command to compile a C++ file on my M1 Mac. \n",
"It compiles the file `optimized.cpp` into an executable called `optimized` \n",
"Then it runs the program called `optimized`\n",
"\n",
"In the next lab (day4), a student has contributed a full solution that compiles to efficient code on Mac, PC and Linux!\n",
"\n",
"You can wait for this, or you can google (or ask ChatGPT!) for how to do this on your platform, then replace the lines below.\n",
"If you're not comfortable with this step, you can skip it for sure - I'll show you exactly how it performs on my Mac.\n",
"This next cell contains the command to compile a C++ file based on the instructions from GPT.\n",
"\n",
"Again, it's not crucial to do this step if you don't wish to!\n",
"\n",
"OR alternatively: student Sandeep K.G. points out that you can run Python and C++ code online to test it out that way. Thank you Sandeep! \n",
"> Not an exact comparison but you can still get the idea of performance difference.\n",
"> Not an exact comparison but you can still get the idea of performance difference. \n",
"> For example here: https://www.programiz.com/cpp-programming/online-compiler/"
]
},
@@ -293,10 +390,41 @@
"metadata": {},
"outputs": [],
"source": [
"# Compile C++ and run the executable\n",
"# Use the commands from GPT 5\n",
"\n",
"!clang++ -O3 -std=c++17 -march=armv8.3-a -o optimized optimized.cpp\n",
"!./optimized"
"def compile_and_run():\n",
" subprocess.run(compile_command, check=True, text=True, capture_output=True)\n",
" print(subprocess.run(run_command, check=True, text=True, capture_output=True).stdout)\n",
" print(subprocess.run(run_command, check=True, text=True, capture_output=True).stdout)\n",
" print(subprocess.run(run_command, check=True, text=True, capture_output=True).stdout)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "22f8f43a",
"metadata": {},
"outputs": [],
"source": [
"compile_and_run()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "faaa39de",
"metadata": {},
"outputs": [],
"source": [
"19.178207/0.082168"
]
},
{
"cell_type": "markdown",
"id": "4f3b8ef9",
"metadata": {},
"source": [
"## OK let's try the other contenders!"
]
},
{
@@ -306,294 +434,79 @@
"metadata": {},
"outputs": [],
"source": [
"optimize_claude(pi)"
"port(anthropic, CLAUDE_MODEL, pi)\n",
"compile_and_run()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d5a766f9-3d23-4bb4-a1d4-88ec44b61ddf",
"id": "138f63c8",
"metadata": {},
"outputs": [],
"source": [
"# Repeat for Claude - again, use the right approach for your platform\n",
"port(grok, GROK_MODEL, pi)\n",
"compile_and_run()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0a0243c5",
"metadata": {},
"outputs": [],
"source": [
"port(gemini, GEMINI_MODEL, pi)\n",
"compile_and_run()\n"
]
},
{
"cell_type": "markdown",
"id": "0689e200",
"metadata": {},
"source": [
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a6ffb0bb",
"metadata": {},
"outputs": [],
"source": [
"print(f\"\"\"\n",
"In Ed's experiments, the performance speedups were:\n",
"\n",
"!clang++ -O3 -std=c++17 -march=armv8.3-a -o optimized optimized.cpp\n",
"!./optimized"
"4th place: Claude Sonnet 4.5: {19.178207/0.104241:.0f}X speedup\n",
"3rd place: GPT-5: {19.178207/0.082168:.0f}X speedup\n",
"2nd place: Grok 4: {19.178207/0.018092:.0f}X speedup\n",
"1st place: Gemini 2.5 Pro: {19.178207/0.013314:.0f}X speedup\n",
"\"\"\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c3b497b3-f569-420e-b92e-fb0f49957ce0",
"id": "8d58753b",
"metadata": {},
"outputs": [],
"source": [
"python_hard = \"\"\"# Be careful to support large number sizes\n",
"\n",
"def lcg(seed, a=1664525, c=1013904223, m=2**32):\n",
" value = seed\n",
" while True:\n",
" value = (a * value + c) % m\n",
" yield value\n",
" \n",
"def max_subarray_sum(n, seed, min_val, max_val):\n",
" lcg_gen = lcg(seed)\n",
" random_numbers = [next(lcg_gen) % (max_val - min_val + 1) + min_val for _ in range(n)]\n",
" max_sum = float('-inf')\n",
" for i in range(n):\n",
" current_sum = 0\n",
" for j in range(i, n):\n",
" current_sum += random_numbers[j]\n",
" if current_sum > max_sum:\n",
" max_sum = current_sum\n",
" return max_sum\n",
"\n",
"def total_max_subarray_sum(n, initial_seed, min_val, max_val):\n",
" total_sum = 0\n",
" lcg_gen = lcg(initial_seed)\n",
" for _ in range(20):\n",
" seed = next(lcg_gen)\n",
" total_sum += max_subarray_sum(n, seed, min_val, max_val)\n",
" return total_sum\n",
"\n",
"# Parameters\n",
"n = 10000 # Number of random numbers\n",
"initial_seed = 42 # Initial seed for the LCG\n",
"min_val = -10 # Minimum value of random numbers\n",
"max_val = 10 # Maximum value of random numbers\n",
"\n",
"# Timing the function\n",
"import time\n",
"start_time = time.time()\n",
"result = total_max_subarray_sum(n, initial_seed, min_val, max_val)\n",
"end_time = time.time()\n",
"\n",
"print(\"Total Maximum Subarray Sum (20 runs):\", result)\n",
"print(\"Execution Time: {:.6f} seconds\".format(end_time - start_time))\n",
"\"\"\""
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dab5e4bc-276c-4555-bd4c-12c699d5e899",
"id": "7202e513",
"metadata": {},
"outputs": [],
"source": [
"exec(python_hard)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e8d24ed5-2c15-4f55-80e7-13a3952b3cb8",
"metadata": {},
"outputs": [],
"source": [
"optimize_gpt(python_hard)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e0b3d073-88a2-40b2-831c-6f0c345c256f",
"metadata": {},
"outputs": [],
"source": [
"# Replace this with the right C++ compile + execute command for your platform\n",
"\n",
"!clang++ -O3 -std=c++17 -march=armv8.3-a -o optimized optimized.cpp\n",
"!./optimized"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e9305446-1d0c-4b51-866a-b8c1e299bf5c",
"metadata": {},
"outputs": [],
"source": [
"optimize_claude(python_hard)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0c181036-8193-4fdd-aef3-fc513b218d43",
"metadata": {},
"outputs": [],
"source": [
"# Replace this with the right C++ compile + execute command for your platform\n",
"\n",
"!clang++ -O3 -std=c++17 -march=armv8.3-a -o optimized optimized.cpp\n",
"!./optimized"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0be9f47d-5213-4700-b0e2-d444c7c738c0",
"metadata": {},
"outputs": [],
"source": [
"def stream_gpt(python): \n",
" stream = openai.chat.completions.create(model=OPENAI_MODEL, messages=messages_for(python), stream=True)\n",
" reply = \"\"\n",
" for chunk in stream:\n",
" fragment = chunk.choices[0].delta.content or \"\"\n",
" reply += fragment\n",
" yield reply.replace('```cpp\\n','').replace('```','')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8669f56b-8314-4582-a167-78842caea131",
"metadata": {},
"outputs": [],
"source": [
"def stream_claude(python):\n",
" result = claude.messages.stream(\n",
" model=CLAUDE_MODEL,\n",
" max_tokens=2000,\n",
" system=system_message,\n",
" messages=[{\"role\": \"user\", \"content\": user_prompt_for(python)}],\n",
" )\n",
" reply = \"\"\n",
" with result as stream:\n",
" for text in stream.text_stream:\n",
" reply += text\n",
" yield reply.replace('```cpp\\n','').replace('```','')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2f1ae8f5-16c8-40a0-aa18-63b617df078d",
"metadata": {},
"outputs": [],
"source": [
"def optimize(python, model):\n",
" if model==\"GPT\":\n",
" result = stream_gpt(python)\n",
" elif model==\"Claude\":\n",
" result = stream_claude(python)\n",
" else:\n",
" raise ValueError(\"Unknown model\")\n",
" for stream_so_far in result:\n",
" yield stream_so_far "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f1ddb38e-6b0a-4c37-baa4-ace0b7de887a",
"metadata": {},
"outputs": [],
"source": [
"with gr.Blocks() as ui:\n",
" with gr.Row():\n",
" python = gr.Textbox(label=\"Python code:\", lines=10, value=python_hard)\n",
" cpp = gr.Textbox(label=\"C++ code:\", lines=10)\n",
" with gr.Row():\n",
" model = gr.Dropdown([\"GPT\", \"Claude\"], label=\"Select model\", value=\"GPT\")\n",
" convert = gr.Button(\"Convert code\")\n",
"\n",
" convert.click(optimize, inputs=[python, model], outputs=[cpp])\n",
"\n",
"ui.launch(inbrowser=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "19bf2bff-a822-4009-a539-f003b1651383",
"metadata": {},
"outputs": [],
"source": [
"def execute_python(code):\n",
" try:\n",
" output = io.StringIO()\n",
" sys.stdout = output\n",
" exec(code)\n",
" finally:\n",
" sys.stdout = sys.__stdout__\n",
" return output.getvalue()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "77f3ab5d-fcfb-4d3f-8728-9cacbf833ea6",
"metadata": {},
"outputs": [],
"source": [
"# You'll need to change the code in the try block to compile the C++ code for your platform\n",
"# I pasted this into Claude's chat UI with a request for it to give me a version for an Intel PC,\n",
"# and it responded with something that looks perfect - you can try a similar approach for your platform.\n",
"\n",
"# M1 Mac version to compile and execute optimized C++ code:\n",
"\n",
"def execute_cpp(code):\n",
" write_output(code)\n",
" try:\n",
" compile_cmd = [\"clang++\", \"-Ofast\", \"-std=c++17\", \"-march=armv8.5-a\", \"-mtune=apple-m1\", \"-mcpu=apple-m1\", \"-o\", \"optimized\", \"optimized.cpp\"]\n",
" compile_result = subprocess.run(compile_cmd, check=True, text=True, capture_output=True)\n",
" run_cmd = [\"./optimized\"]\n",
" run_result = subprocess.run(run_cmd, check=True, text=True, capture_output=True)\n",
" return run_result.stdout\n",
" except subprocess.CalledProcessError as e:\n",
" return f\"An error occurred:\\n{e.stderr}\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9a2274f1-d03b-42c0-8dcc-4ce159b18442",
"metadata": {},
"outputs": [],
"source": [
"css = \"\"\"\n",
".python {background-color: #306998;}\n",
".cpp {background-color: #050;}\n",
"\"\"\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f1303932-160c-424b-97a8-d28c816721b2",
"metadata": {},
"outputs": [],
"source": [
"with gr.Blocks(css=css) as ui:\n",
" gr.Markdown(\"## Convert code from Python to C++\")\n",
" with gr.Row():\n",
" python = gr.Textbox(label=\"Python code:\", value=python_hard, lines=10)\n",
" cpp = gr.Textbox(label=\"C++ code:\", lines=10)\n",
" with gr.Row():\n",
" model = gr.Dropdown([\"GPT\", \"Claude\"], label=\"Select model\", value=\"GPT\")\n",
" with gr.Row():\n",
" convert = gr.Button(\"Convert code\")\n",
" with gr.Row():\n",
" python_run = gr.Button(\"Run Python\")\n",
" cpp_run = gr.Button(\"Run C++\")\n",
" with gr.Row():\n",
" python_out = gr.TextArea(label=\"Python result:\", elem_classes=[\"python\"])\n",
" cpp_out = gr.TextArea(label=\"C++ result:\", elem_classes=[\"cpp\"])\n",
"\n",
" convert.click(optimize, inputs=[python, model], outputs=[cpp])\n",
" python_run.click(execute_python, inputs=[python], outputs=[python_out])\n",
" cpp_run.click(execute_cpp, inputs=[cpp], outputs=[cpp_out])\n",
"\n",
"ui.launch(inbrowser=True)"
]
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": ".venv",
"language": "python",
"name": "python3"
},
@@ -607,7 +520,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.11"
"version": "3.12.9"
}
},
"nbformat": 4,

865
week4/day4.ipynb
View File

@@ -7,30 +7,41 @@
"source": [
"# Code Generator\n",
"\n",
"The requirement: use an Open Source model to generate high performance C++ code from Python code\n",
"\n",
"To replicate this, you'll need to set up a HuggingFace endpoint as I do in the video. It's simple to do, and it's quite satisfying to see the results!\n",
"\n",
"It's also an important part of your learning; this is the first example of deploying an open source model to be behind an API. We'll return to this in Week 8, but this should plant a seed in your mind for what's involved in moving open source models into production."
"The requirement: use a Frontier model to generate high performance C++ code from Python code\n"
]
},
{
"cell_type": "markdown",
"id": "22e1567b-33fd-49e7-866e-4b635d15715a",
"id": "d5ccb926-7b49-44a4-99ab-8ef20b5778c0",
"metadata": {},
"source": [
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h1 style=\"color:#900;\">Important - Pause Endpoints when not in use</h1>\n",
" <h2 style=\"color:#f71;\">Reminder: OPTIONAL to execute C++ code</h2>\n",
" <span style=\"color:#f71;\">As an alternative, you can run it on the website given yesterday</span>\n",
" </td>\n",
" </tr>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"id": "d90e04a2-5b8a-4fd5-9db8-27c02f033313",
"metadata": {},
"source": [
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../assets/important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h1 style=\"color:#900;\">Important Note</h1>\n",
" <span style=\"color:#900;\">\n",
" If you do decide to use HuggingFace endpoints for this project, you should stop or pause the endpoints when you are done to avoid accruing unnecessary running cost. The costs are very low as long as you only run the endpoint when you're using it. Navigate to the HuggingFace endpoint UI <a href=\"https://ui.endpoints.huggingface.co/\">here,</a> open your endpoint, and click Pause to put it on pause so you no longer pay for it. \n",
"Many thanks to student John L. for raising this.\n",
"<br/><br/>\n",
"In week 8 we will use Modal instead of HuggingFace endpoints; with Modal you only pay for the time that you use it and you should get free credits.\n",
" In this lab, I use free open source models on Ollama. I also use paid open-source models via Groq and OpenRouter. Only pick the models you want to!\n",
" </span>\n",
" </td>\n",
" </tr>\n",
@@ -49,15 +60,10 @@
"import os\n",
"import io\n",
"import sys\n",
"import json\n",
"import requests\n",
"from dotenv import load_dotenv\n",
"from openai import OpenAI\n",
"import google.generativeai\n",
"import anthropic\n",
"from IPython.display import Markdown, display, update_display\n",
"import gradio as gr\n",
"import subprocess"
"import subprocess\n"
]
},
{
@@ -67,12 +73,70 @@
"metadata": {},
"outputs": [],
"source": [
"# environment\n",
"\n",
"load_dotenv(override=True)\n",
"os.environ['OPENAI_API_KEY'] = os.getenv('OPENAI_API_KEY', 'your-key-if-not-using-env')\n",
"os.environ['ANTHROPIC_API_KEY'] = os.getenv('ANTHROPIC_API_KEY', 'your-key-if-not-using-env')\n",
"os.environ['HF_TOKEN'] = os.getenv('HF_TOKEN', 'your-key-if-not-using-env')"
"openai_api_key = os.getenv('OPENAI_API_KEY')\n",
"anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')\n",
"google_api_key = os.getenv('GOOGLE_API_KEY')\n",
"grok_api_key = os.getenv('GROK_API_KEY')\n",
"groq_api_key = os.getenv('GROQ_API_KEY')\n",
"openrouter_api_key = os.getenv('OPENROUTER_API_KEY')\n",
"\n",
"if openai_api_key:\n",
" print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n",
"else:\n",
" print(\"OpenAI API Key not set\")\n",
" \n",
"if anthropic_api_key:\n",
" print(f\"Anthropic API Key exists and begins {anthropic_api_key[:7]}\")\n",
"else:\n",
" print(\"Anthropic API Key not set (and this is optional)\")\n",
"\n",
"if google_api_key:\n",
" print(f\"Google API Key exists and begins {google_api_key[:2]}\")\n",
"else:\n",
" print(\"Google API Key not set (and this is optional)\")\n",
"\n",
"if grok_api_key:\n",
" print(f\"Grok API Key exists and begins {grok_api_key[:4]}\")\n",
"else:\n",
" print(\"Grok API Key not set (and this is optional)\")\n",
"\n",
"if groq_api_key:\n",
" print(f\"Groq API Key exists and begins {groq_api_key[:4]}\")\n",
"else:\n",
" print(\"Groq API Key not set (and this is optional)\")\n",
"\n",
"if openrouter_api_key:\n",
" print(f\"OpenRouter API Key exists and begins {openrouter_api_key[:6]}\")\n",
"else:\n",
" print(\"OpenRouter API Key not set (and this is optional)\")\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "59863df1",
"metadata": {},
"outputs": [],
"source": [
"# Connect to client libraries\n",
"\n",
"openai = OpenAI()\n",
"\n",
"anthropic_url = \"https://api.anthropic.com/v1/\"\n",
"gemini_url = \"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
"grok_url = \"https://api.x.ai/v1\"\n",
"groq_url = \"https://api.groq.com/openai/v1\"\n",
"ollama_url = \"http://localhost:11434/v1\"\n",
"openrouter_url = \"https://openrouter.ai/api/v1\"\n",
"\n",
"anthropic = OpenAI(api_key=anthropic_api_key, base_url=anthropic_url)\n",
"gemini = OpenAI(api_key=google_api_key, base_url=gemini_url)\n",
"grok = OpenAI(api_key=grok_api_key, base_url=grok_url)\n",
"groq = OpenAI(api_key=groq_api_key, base_url=groq_url)\n",
"ollama = OpenAI(api_key=\"ollama\", base_url=ollama_url)\n",
"openrouter = OpenAI(api_key=openrouter_api_key, base_url=openrouter_url)\n"
]
},
{
@@ -82,12 +146,55 @@
"metadata": {},
"outputs": [],
"source": [
"# initialize\n",
"models = [\"gpt-5\", \"claude-sonnet-4-5-20250929\", \"grok-4\", \"gemini-2.5-pro\", \"qwen2.5-coder\", \"deepseek-coder-v2\", \"gpt-oss:20b\", \"qwen/qwen3-coder-30b-a3b-instruct\", \"openai/gpt-oss-120b\", ]\n",
"\n",
"openai = OpenAI()\n",
"claude = anthropic.Anthropic()\n",
"OPENAI_MODEL = \"gpt-4o\"\n",
"CLAUDE_MODEL = \"claude-3-5-sonnet-20240620\""
"clients = {\"gpt-5\": openai, \"claude-sonnet-4-5-20250929\": anthropic, \"grok-4\": grok, \"gemini-2.5-pro\": gemini, \"openai/gpt-oss-120b\": groq, \"qwen2.5-coder\": ollama, \"deepseek-coder-v2\": ollama, \"gpt-oss:20b\": ollama, \"qwen/qwen3-coder-30b-a3b-instruct\": openrouter}\n",
"\n",
"# Want to keep costs ultra-low? Replace this with models of your choice, using the examples from yesterday"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "68c1f1be",
"metadata": {},
"outputs": [],
"source": [
"from system_info import retrieve_system_info\n",
"\n",
"system_info = retrieve_system_info()\n",
"system_info"
]
},
{
"cell_type": "markdown",
"id": "81e92c12",
"metadata": {},
"source": [
"## Overwrite this with the commands from yesterday\n",
"\n",
"Or just use the website like yesterday:\n",
"\n",
" https://www.programiz.com/cpp-programming/online-compiler/"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d734a634",
"metadata": {},
"outputs": [],
"source": [
"compile_command = [\"clang++\", \"-std=c++17\", \"-Ofast\", \"-mcpu=native\", \"-flto=thin\", \"-fvisibility=hidden\", \"-DNDEBUG\", \"main.cpp\", \"-o\", \"main\"]\n",
"run_command = [\"./main\"]\n"
]
},
{
"cell_type": "markdown",
"id": "f0b0a437",
"metadata": {},
"source": [
"## And now, on with the main task"
]
},
{
@@ -97,9 +204,26 @@
"metadata": {},
"outputs": [],
"source": [
"system_message = \"You are an assistant that reimplements Python code in high performance C++ for an M1 Mac. \"\n",
"system_message += \"Respond only with C++ code; use comments sparingly and do not provide any explanation other than occasional comments. \"\n",
"system_message += \"The C++ response needs to produce an identical output in the fastest possible time. Keep implementations of random number generators identical so that results match exactly.\""
"system_prompt = \"\"\"\n",
"Your task is to convert Python code into high performance C++ code.\n",
"Respond only with C++ code. Do not provide any explanation other than occasional comments.\n",
"The C++ response needs to produce an identical output in the fastest possible time.\n",
"\"\"\"\n",
"\n",
"def user_prompt_for(python):\n",
" return f\"\"\"\n",
"Port this Python code to C++ with the fastest possible implementation that produces identical output in the least time.\n",
"The system information is:\n",
"{system_info}\n",
"Your response will be written to a file called main.cpp and then compiled and executed; the compilation command is:\n",
"{compile_command}\n",
"Respond only with C++ code.\n",
"Python code to port:\n",
"\n",
"```python\n",
"{python}\n",
"```\n",
"\"\"\""
]
},
{
@@ -109,12 +233,12 @@
"metadata": {},
"outputs": [],
"source": [
"def user_prompt_for(python):\n",
" user_prompt = \"Rewrite this Python code in C++ with the fastest possible implementation that produces identical output in the least time. \"\n",
" user_prompt += \"Respond only with C++ code; do not explain your work other than a few comments. \"\n",
" user_prompt += \"Pay attention to number types to ensure no int overflows. Remember to #include all necessary C++ packages such as iomanip.\\n\\n\"\n",
" user_prompt += python\n",
" return user_prompt"
"def messages_for(python):\n",
" return [\n",
" {\"role\": \"system\", \"content\": system_prompt},\n",
" {\"role\": \"user\", \"content\": user_prompt_for(python)}\n",
" ]\n",
" "
]
},
{
@@ -124,26 +248,9 @@
"metadata": {},
"outputs": [],
"source": [
"def messages_for(python):\n",
" return [\n",
" {\"role\": \"system\", \"content\": system_message},\n",
" {\"role\": \"user\", \"content\": user_prompt_for(python)}\n",
" ]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "71e1ba8c-5b05-4726-a9f3-8d8c6257350b",
"metadata": {},
"outputs": [],
"source": [
"# write to a file called optimized.cpp\n",
"\n",
"def write_output(cpp):\n",
" code = cpp.replace(\"```cpp\",\"\").replace(\"```\",\"\")\n",
" with open(\"optimized.cpp\", \"w\") as f:\n",
" f.write(code)"
" with open(\"main.cpp\", \"w\") as f:\n",
" f.write(cpp)"
]
},
{
@@ -153,36 +260,14 @@
"metadata": {},
"outputs": [],
"source": [
"def optimize_gpt(python): \n",
" stream = openai.chat.completions.create(model=OPENAI_MODEL, messages=messages_for(python), stream=True)\n",
" reply = \"\"\n",
" for chunk in stream:\n",
" fragment = chunk.choices[0].delta.content or \"\"\n",
" reply += fragment\n",
" print(fragment, end='', flush=True)\n",
" write_output(reply)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7cd84ad8-d55c-4fe0-9eeb-1895c95c4a9d",
"metadata": {},
"outputs": [],
"source": [
"def optimize_claude(python):\n",
" result = claude.messages.stream(\n",
" model=CLAUDE_MODEL,\n",
" max_tokens=2000,\n",
" system=system_message,\n",
" messages=[{\"role\": \"user\", \"content\": user_prompt_for(python)}],\n",
" )\n",
" reply = \"\"\n",
" with result as stream:\n",
" for text in stream.text_stream:\n",
" reply += text\n",
" print(text, end=\"\", flush=True)\n",
" write_output(reply)"
"def port(model, python):\n",
" client = clients[model]\n",
" reasoning_effort = \"high\" if 'gpt' in model else None\n",
" response = client.chat.completions.create(model=model, messages=messages_for(python), reasoning_effort=reasoning_effort)\n",
" reply = response.choices[0].message.content\n",
" reply = reply.replace('```cpp','').replace('```','')\n",
" write_output(reply)\n",
" return reply"
]
},
{
@@ -205,7 +290,7 @@
" return result\n",
"\n",
"start_time = time.time()\n",
"result = calculate(100_000_000, 4, 1) * 4\n",
"result = calculate(200_000_000, 4, 1) * 4\n",
"end_time = time.time()\n",
"\n",
"print(f\"Result: {result:.12f}\")\n",
@@ -220,27 +305,22 @@
"metadata": {},
"outputs": [],
"source": [
"exec(pi)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "105db6f9-343c-491d-8e44-3a5328b81719",
"metadata": {},
"outputs": [],
"source": [
"optimize_gpt(pi)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bf26ee95-0c77-491d-9a91-579a1e96a8a3",
"metadata": {},
"outputs": [],
"source": [
"exec(pi)"
"def run_python(code):\n",
" globals_dict = {\"__builtins__\": __builtins__}\n",
"\n",
" buffer = io.StringIO()\n",
" old_stdout = sys.stdout\n",
" sys.stdout = buffer\n",
"\n",
" try:\n",
" exec(code, globals_dict)\n",
" output = buffer.getvalue()\n",
" except Exception as e:\n",
" output = f\"Error: {e}\"\n",
" finally:\n",
" sys.stdout = old_stdout\n",
"\n",
" return output"
]
},
{
@@ -250,188 +330,14 @@
"metadata": {},
"outputs": [],
"source": [
"!clang++ -O3 -std=c++17 -march=armv8.3-a -o optimized optimized.cpp\n",
"!./optimized"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "983a11fe-e24d-4c65-8269-9802c5ef3ae6",
"metadata": {},
"outputs": [],
"source": [
"optimize_claude(pi)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d5a766f9-3d23-4bb4-a1d4-88ec44b61ddf",
"metadata": {},
"outputs": [],
"source": [
"!clang++ -O3 -std=c++17 -march=armv8.3-a -o optimized optimized.cpp\n",
"!./optimized"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c3b497b3-f569-420e-b92e-fb0f49957ce0",
"metadata": {},
"outputs": [],
"source": [
"python_hard = \"\"\"# Be careful to support large number sizes\n",
"\n",
"def lcg(seed, a=1664525, c=1013904223, m=2**32):\n",
" value = seed\n",
" while True:\n",
" value = (a * value + c) % m\n",
" yield value\n",
" \n",
"def max_subarray_sum(n, seed, min_val, max_val):\n",
" lcg_gen = lcg(seed)\n",
" random_numbers = [next(lcg_gen) % (max_val - min_val + 1) + min_val for _ in range(n)]\n",
" max_sum = float('-inf')\n",
" for i in range(n):\n",
" current_sum = 0\n",
" for j in range(i, n):\n",
" current_sum += random_numbers[j]\n",
" if current_sum > max_sum:\n",
" max_sum = current_sum\n",
" return max_sum\n",
"\n",
"def total_max_subarray_sum(n, initial_seed, min_val, max_val):\n",
" total_sum = 0\n",
" lcg_gen = lcg(initial_seed)\n",
" for _ in range(20):\n",
" seed = next(lcg_gen)\n",
" total_sum += max_subarray_sum(n, seed, min_val, max_val)\n",
" return total_sum\n",
"\n",
"# Parameters\n",
"n = 10000 # Number of random numbers\n",
"initial_seed = 42 # Initial seed for the LCG\n",
"min_val = -10 # Minimum value of random numbers\n",
"max_val = 10 # Maximum value of random numbers\n",
"\n",
"# Timing the function\n",
"import time\n",
"start_time = time.time()\n",
"result = total_max_subarray_sum(n, initial_seed, min_val, max_val)\n",
"end_time = time.time()\n",
"\n",
"print(\"Total Maximum Subarray Sum (20 runs):\", result)\n",
"print(\"Execution Time: {:.6f} seconds\".format(end_time - start_time))\n",
"\"\"\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dab5e4bc-276c-4555-bd4c-12c699d5e899",
"metadata": {},
"outputs": [],
"source": [
"exec(python_hard)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e8d24ed5-2c15-4f55-80e7-13a3952b3cb8",
"metadata": {},
"outputs": [],
"source": [
"optimize_gpt(python_hard)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e0b3d073-88a2-40b2-831c-6f0c345c256f",
"metadata": {},
"outputs": [],
"source": [
"!clang++ -O3 -std=c++17 -march=armv8.3-a -o optimized optimized.cpp\n",
"!./optimized"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e9305446-1d0c-4b51-866a-b8c1e299bf5c",
"metadata": {},
"outputs": [],
"source": [
"optimize_claude(python_hard)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0c181036-8193-4fdd-aef3-fc513b218d43",
"metadata": {},
"outputs": [],
"source": [
"!clang++ -O3 -std=c++17 -march=armv8.3-a -o optimized optimized.cpp\n",
"!./optimized"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0be9f47d-5213-4700-b0e2-d444c7c738c0",
"metadata": {},
"outputs": [],
"source": [
"def stream_gpt(python): \n",
" stream = openai.chat.completions.create(model=OPENAI_MODEL, messages=messages_for(python), stream=True)\n",
" reply = \"\"\n",
" for chunk in stream:\n",
" fragment = chunk.choices[0].delta.content or \"\"\n",
" reply += fragment\n",
" yield reply.replace('```cpp\\n','').replace('```','')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8669f56b-8314-4582-a167-78842caea131",
"metadata": {},
"outputs": [],
"source": [
"def stream_claude(python):\n",
" result = claude.messages.stream(\n",
" model=CLAUDE_MODEL,\n",
" max_tokens=2000,\n",
" system=system_message,\n",
" messages=[{\"role\": \"user\", \"content\": user_prompt_for(python)}],\n",
" )\n",
" reply = \"\"\n",
" with result as stream:\n",
" for text in stream.text_stream:\n",
" reply += text\n",
" yield reply.replace('```cpp\\n','').replace('```','')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2f1ae8f5-16c8-40a0-aa18-63b617df078d",
"metadata": {},
"outputs": [],
"source": [
"def optimize(python, model):\n",
" if model==\"GPT\":\n",
" result = stream_gpt(python)\n",
" elif model==\"Claude\":\n",
" result = stream_claude(python)\n",
" else:\n",
" raise ValueError(\"Unknown model\")\n",
" for stream_so_far in result:\n",
" yield stream_so_far "
"def compile_and_run():\n",
" try:\n",
" subprocess.run(compile_command, check=True, text=True, capture_output=True)\n",
" print(subprocess.run(run_command, check=True, text=True, capture_output=True).stdout)\n",
" print(subprocess.run(run_command, check=True, text=True, capture_output=True).stdout)\n",
" print(subprocess.run(run_command, check=True, text=True, capture_output=True).stdout)\n",
" except subprocess.CalledProcessError as e:\n",
" print(f\"An error occurred:\\n{e.stderr}\")"
]
},
{
@@ -443,13 +349,13 @@
"source": [
"with gr.Blocks() as ui:\n",
" with gr.Row():\n",
" python = gr.Textbox(label=\"Python code:\", lines=10, value=python_hard)\n",
" cpp = gr.Textbox(label=\"C++ code:\", lines=10)\n",
" python = gr.Textbox(label=\"Python code:\", lines=28, value=pi)\n",
" cpp = gr.Textbox(label=\"C++ code:\", lines=28)\n",
" with gr.Row():\n",
" model = gr.Dropdown([\"GPT\", \"Claude\"], label=\"Select model\", value=\"GPT\")\n",
" model = gr.Dropdown(models, label=\"Select model\", value=models[0])\n",
" convert = gr.Button(\"Convert code\")\n",
"\n",
" convert.click(optimize, inputs=[python, model], outputs=[cpp])\n",
" convert.click(port, inputs=[model, python], outputs=[cpp])\n",
"\n",
"ui.launch(inbrowser=True)"
]
@@ -457,376 +363,63 @@
{
"cell_type": "code",
"execution_count": null,
"id": "19bf2bff-a822-4009-a539-f003b1651383",
"id": "28969928",
"metadata": {},
"outputs": [],
"source": [
"def execute_python(code):\n",
" try:\n",
" output = io.StringIO()\n",
" sys.stdout = output\n",
" exec(code)\n",
" finally:\n",
" sys.stdout = sys.__stdout__\n",
" return output.getvalue()"
]
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "77f3ab5d-fcfb-4d3f-8728-9cacbf833ea6",
"id": "d9cc1c03",
"metadata": {},
"outputs": [],
"source": [
"def execute_cpp(code):\n",
" write_output(code)\n",
" compiler_cmd = [\"clang++\", \"-O3\", \"-std=c++17\", \"-march=armv8.3-a\", \"-o\", \"optimized\", \"optimized.cpp\"]\n",
" try:\n",
" compile_result = subprocess.run(compiler_cmd, check=True, text=True, capture_output=True)\n",
" run_cmd = [\"./optimized\"]\n",
" run_result = subprocess.run(run_cmd, check=True, text=True, capture_output=True)\n",
" return run_result.stdout\n",
" except subprocess.CalledProcessError as e:\n",
" return f\"An error occurred:\\n{e.stderr}\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9a2274f1-d03b-42c0-8dcc-4ce159b18442",
"metadata": {},
"outputs": [],
"source": [
"css = \"\"\"\n",
".python {background-color: #306998;}\n",
".cpp {background-color: #050;}\n",
"\"\"\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f1303932-160c-424b-97a8-d28c816721b2",
"metadata": {},
"outputs": [],
"source": [
"with gr.Blocks(css=css) as ui:\n",
" gr.Markdown(\"## Convert code from Python to C++\")\n",
" with gr.Row():\n",
" python = gr.Textbox(label=\"Python code:\", value=python_hard, lines=10)\n",
" cpp = gr.Textbox(label=\"C++ code:\", lines=10)\n",
" with gr.Row():\n",
" model = gr.Dropdown([\"GPT\", \"Claude\"], label=\"Select model\", value=\"GPT\")\n",
" with gr.Row():\n",
" convert = gr.Button(\"Convert code\")\n",
" with gr.Row():\n",
" python_run = gr.Button(\"Run Python\")\n",
" cpp_run = gr.Button(\"Run C++\")\n",
" with gr.Row():\n",
" python_out = gr.TextArea(label=\"Python result:\", elem_classes=[\"python\"])\n",
" cpp_out = gr.TextArea(label=\"C++ result:\", elem_classes=[\"cpp\"])\n",
"\n",
" convert.click(optimize, inputs=[python, model], outputs=[cpp])\n",
" python_run.click(execute_python, inputs=[python], outputs=[python_out])\n",
" cpp_run.click(execute_cpp, inputs=[cpp], outputs=[cpp_out])\n",
"\n",
"ui.launch(inbrowser=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bb8c5b4e-ec51-4f21-b3f8-6aa94fede86d",
"metadata": {},
"outputs": [],
"source": [
"from huggingface_hub import login, InferenceClient\n",
"from transformers import AutoTokenizer"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "13347633-4606-4e38-9927-80c39e65c1f1",
"metadata": {},
"outputs": [],
"source": [
"hf_token = os.environ['HF_TOKEN']\n",
"login(hf_token, add_to_git_credential=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ef60a4df-6267-4ebd-8eed-dcb917af0a5e",
"metadata": {},
"outputs": [],
"source": [
"code_qwen = \"Qwen/CodeQwen1.5-7B-Chat\"\n",
"code_gemma = \"google/codegemma-7b-it\"\n",
"CODE_QWEN_URL = \"https://h1vdol7jxhje3mpn.us-east-1.aws.endpoints.huggingface.cloud\"\n",
"CODE_GEMMA_URL = \"https://c5hggiyqachmgnqg.us-east-1.aws.endpoints.huggingface.cloud\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "695ce389-a903-4533-a2f1-cd9e2a6af8f2",
"metadata": {},
"outputs": [],
"source": [
"tokenizer = AutoTokenizer.from_pretrained(code_qwen)\n",
"messages = messages_for(pi)\n",
"text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d4548e96-0b32-4793-bdd6-1b072c2f26ab",
"metadata": {},
"outputs": [],
"source": [
"print(text)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bb2a126b-09e7-4966-bc97-0ef5c2cc7896",
"metadata": {},
"outputs": [],
"source": [
"client = InferenceClient(CODE_QWEN_URL, token=hf_token)\n",
"stream = client.text_generation(text, stream=True, details=True, max_new_tokens=3000)\n",
"for r in stream:\n",
" print(r.token.text, end = \"\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "127a52e5-ad85-42b7-a0f5-9afda5efe090",
"metadata": {},
"outputs": [],
"source": [
"def stream_code_qwen(python):\n",
" tokenizer = AutoTokenizer.from_pretrained(code_qwen)\n",
" messages = messages_for(python)\n",
" text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\n",
" client = InferenceClient(CODE_QWEN_URL, token=hf_token)\n",
" stream = client.text_generation(text, stream=True, details=True, max_new_tokens=3000)\n",
" result = \"\"\n",
" for r in stream:\n",
" result += r.token.text\n",
" yield result "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a82387d1-7651-4923-995b-fe18356fcaa6",
"metadata": {},
"outputs": [],
"source": [
"def optimize(python, model):\n",
" if model==\"GPT\":\n",
" result = stream_gpt(python)\n",
" elif model==\"Claude\":\n",
" result = stream_claude(python)\n",
" elif model==\"CodeQwen\":\n",
" result = stream_code_qwen(python)\n",
" else:\n",
" raise ValueError(\"Unknown model\")\n",
" for stream_so_far in result:\n",
" yield stream_so_far "
"compile_and_run()"
]
},
{
"cell_type": "markdown",
"id": "4b0a6a97-5b8a-4a9b-8ee0-7561e0ced673",
"id": "80037156",
"metadata": {},
"source": [
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../thankyou.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#090;\">Thank you to @CloudLlama for an amazing contribution</h2>\n",
" <span style=\"color:#090;\">\n",
" A student has contributed a chunk of code to improve this, in the next 2 cells. You can now select which Python porgram to run,\n",
" and a compiler is automatically selected that will work on PC, Windows and Mac. Massive thank you @CloudLlama!\n",
" </span>\n",
" </td>\n",
" </tr>\n",
"</table>"
"Qwen 2.5 Coder: Fail \n",
"DeepSeek Coder v2: 0.114050084 \n",
"OpenAI gpt-oss 20B: 0.080438 \n",
"Qwen 30B: 0.113734 \n",
"OpenAI gpt-oss 120B: 1.407383\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4ba311ec-c16a-4fe0-946b-4b940704cf65",
"cell_type": "markdown",
"id": "ad8d4e52",
"metadata": {},
"outputs": [],
"source": [
"def select_sample_program(sample_program):\n",
" if sample_program==\"pi\":\n",
" return pi\n",
" elif sample_program==\"python_hard\":\n",
" return python_hard\n",
" else:\n",
" return \"Type your Python program here\""
"In Ed's experiments, the performance speedups were:\n",
"\n",
"9th place: Qwen 2.5 Coder: Fail \n",
"8th place: OpenAI GPT-OSS 120B: 14X speedup \n",
"7th place: DeepSeek Coder v2: 168X speedup \n",
"6th place: Qwen3 Coder 30B: 168X speedup \n",
"5th place: Claude Sonnet 4.5: 184X speedup \n",
"4th place: GPT-5: 233X speedup \n",
"**3rd place: oss-20B: 238X speedup** \n",
"2nd place: Grok 4: 1060X speedup \n",
"1st place: Gemini 2.5 Pro: 1440X speedup "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e42286bc-085c-45dc-b101-234308e58269",
"cell_type": "markdown",
"id": "6e617df9",
"metadata": {},
"outputs": [],
"source": [
"import platform\n",
"\n",
"VISUAL_STUDIO_2022_TOOLS = \"C:\\\\Program Files\\\\Microsoft Visual Studio\\\\2022\\\\Community\\\\Common7\\Tools\\\\VsDevCmd.bat\"\n",
"VISUAL_STUDIO_2019_TOOLS = \"C:\\\\Program Files (x86)\\\\Microsoft Visual Studio\\\\2019\\\\BuildTools\\\\Common7\\\\Tools\\\\VsDevCmd.bat\"\n",
"\n",
"simple_cpp = \"\"\"\n",
"#include <iostream>\n",
"\n",
"int main() {\n",
" std::cout << \"Hello\";\n",
" return 0;\n",
"}\n",
"\"\"\"\n",
"\n",
"def run_cmd(command_to_run):\n",
" try:\n",
" run_result = subprocess.run(command_to_run, check=True, text=True, capture_output=True)\n",
" return run_result.stdout if run_result.stdout else \"SUCCESS\"\n",
" except:\n",
" return \"\"\n",
"\n",
"def c_compiler_cmd(filename_base):\n",
" my_platform = platform.system()\n",
" my_compiler = []\n",
"\n",
" try:\n",
" with open(\"simple.cpp\", \"w\") as f:\n",
" f.write(simple_cpp)\n",
" \n",
" if my_platform == \"Windows\":\n",
" if os.path.isfile(VISUAL_STUDIO_2022_TOOLS):\n",
" if os.path.isfile(\"./simple.exe\"):\n",
" os.remove(\"./simple.exe\")\n",
" compile_cmd = [\"cmd\", \"/c\", VISUAL_STUDIO_2022_TOOLS, \"&\", \"cl\", \"simple.cpp\"]\n",
" if run_cmd(compile_cmd):\n",
" if run_cmd([\"./simple.exe\"]) == \"Hello\":\n",
" my_compiler = [\"Windows\", \"Visual Studio 2022\", [\"cmd\", \"/c\", VISUAL_STUDIO_2022_TOOLS, \"&\", \"cl\", f\"{filename_base}.cpp\"]]\n",
" \n",
" if not my_compiler:\n",
" if os.path.isfile(VISUAL_STUDIO_2019_TOOLS):\n",
" if os.path.isfile(\"./simple.exe\"):\n",
" os.remove(\"./simple.exe\")\n",
" compile_cmd = [\"cmd\", \"/c\", VISUAL_STUDIO_2019_TOOLS, \"&\", \"cl\", \"simple.cpp\"]\n",
" if run_cmd(compile_cmd):\n",
" if run_cmd([\"./simple.exe\"]) == \"Hello\":\n",
" my_compiler = [\"Windows\", \"Visual Studio 2019\", [\"cmd\", \"/c\", VISUAL_STUDIO_2019_TOOLS, \"&\", \"cl\", f\"{filename_base}.cpp\"]]\n",
" \n",
" if not my_compiler:\n",
" my_compiler=[my_platform, \"Unavailable\", []]\n",
" \n",
" elif my_platform == \"Linux\":\n",
" if os.path.isfile(\"./simple\"):\n",
" os.remove(\"./simple\")\n",
" compile_cmd = [\"g++\", \"simple.cpp\", \"-o\", \"simple\"]\n",
" if run_cmd(compile_cmd):\n",
" if run_cmd([\"./simple\"]) == \"Hello\":\n",
" my_compiler = [\"Linux\", \"GCC (g++)\", [\"g++\", f\"{filename_base}.cpp\", \"-o\", f\"{filename_base}\" ]]\n",
" \n",
" if not my_compiler:\n",
" if os.path.isfile(\"./simple\"):\n",
" os.remove(\"./simple\")\n",
" compile_cmd = [\"clang++\", \"simple.cpp\", \"-o\", \"simple\"]\n",
" if run_cmd(compile_cmd):\n",
" if run_cmd([\"./simple\"]) == \"Hello\":\n",
" my_compiler = [\"Linux\", \"Clang++\", [\"clang++\", f\"{filename_base}.cpp\", \"-o\", f\"{filename_base}\"]]\n",
" \n",
" if not my_compiler:\n",
" my_compiler=[my_platform, \"Unavailable\", []]\n",
" \n",
" elif my_platform == \"Darwin\":\n",
" if os.path.isfile(\"./simple\"):\n",
" os.remove(\"./simple\")\n",
" compile_cmd = [\"clang++\", \"-Ofast\", \"-std=c++17\", \"-march=armv8.5-a\", \"-mtune=apple-m1\", \"-mcpu=apple-m1\", \"-o\", \"simple\", \"simple.cpp\"]\n",
" if run_cmd(compile_cmd):\n",
" if run_cmd([\"./simple\"]) == \"Hello\":\n",
" my_compiler = [\"Macintosh\", \"Clang++\", [\"clang++\", \"-Ofast\", \"-std=c++17\", \"-march=armv8.5-a\", \"-mtune=apple-m1\", \"-mcpu=apple-m1\", \"-o\", f\"{filename_base}\", f\"{filename_base}.cpp\"]]\n",
" \n",
" if not my_compiler:\n",
" my_compiler=[my_platform, \"Unavailable\", []]\n",
" except:\n",
" my_compiler=[my_platform, \"Unavailable\", []]\n",
" \n",
" if my_compiler:\n",
" return my_compiler\n",
" else:\n",
" return [\"Unknown\", \"Unavailable\", []]\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f9ca2e6f-60c1-4e5f-b570-63c75b2d189b",
"metadata": {},
"outputs": [],
"source": [
"compiler_cmd = c_compiler_cmd(\"optimized\")\n",
"\n",
"with gr.Blocks(css=css) as ui:\n",
" gr.Markdown(\"## Convert code from Python to C++\")\n",
" with gr.Row():\n",
" python = gr.Textbox(label=\"Python code:\", value=python_hard, lines=10)\n",
" cpp = gr.Textbox(label=\"C++ code:\", lines=10)\n",
" with gr.Row():\n",
" with gr.Column():\n",
" sample_program = gr.Radio([\"pi\", \"python_hard\"], label=\"Sample program\", value=\"python_hard\")\n",
" model = gr.Dropdown([\"GPT\", \"Claude\", \"CodeQwen\"], label=\"Select model\", value=\"GPT\")\n",
" with gr.Column():\n",
" architecture = gr.Radio([compiler_cmd[0]], label=\"Architecture\", interactive=False, value=compiler_cmd[0])\n",
" compiler = gr.Radio([compiler_cmd[1]], label=\"Compiler\", interactive=False, value=compiler_cmd[1])\n",
" with gr.Row():\n",
" convert = gr.Button(\"Convert code\")\n",
" with gr.Row():\n",
" python_run = gr.Button(\"Run Python\")\n",
" if not compiler_cmd[1] == \"Unavailable\":\n",
" cpp_run = gr.Button(\"Run C++\")\n",
" else:\n",
" cpp_run = gr.Button(\"No compiler to run C++\", interactive=False)\n",
" with gr.Row():\n",
" python_out = gr.TextArea(label=\"Python result:\", elem_classes=[\"python\"])\n",
" cpp_out = gr.TextArea(label=\"C++ result:\", elem_classes=[\"cpp\"])\n",
"\n",
" sample_program.change(select_sample_program, inputs=[sample_program], outputs=[python])\n",
" convert.click(optimize, inputs=[python, model], outputs=[cpp])\n",
" python_run.click(execute_python, inputs=[python], outputs=[python_out])\n",
" cpp_run.click(execute_cpp, inputs=[cpp], outputs=[cpp_out])\n",
"\n",
"ui.launch(inbrowser=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9d0ad093-425b-488e-8c3f-67f729dd9c06",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": ".venv",
"language": "python",
"name": "python3"
},
@@ -840,7 +433,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.11"
"version": "3.12.9"
}
},
"nbformat": 4,

522
week4/day5.ipynb Normal file
View File

@@ -0,0 +1,522 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "4a6ab9a2-28a2-445d-8512-a0dc8d1b54e9",
"metadata": {},
"source": [
"# Code Generator\n",
"\n",
"The requirement: use a Frontier model to generate high performance C++ code from Python code\n"
]
},
{
"cell_type": "markdown",
"id": "d5ccb926-7b49-44a4-99ab-8ef20b5778c0",
"metadata": {},
"source": [
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../assets/resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#f71;\">Reminder: OPTIONAL to execute C++ code or Rust code</h2>\n",
" <span style=\"color:#f71;\">As an alternative, you can run it on the website given yesterday</span>\n",
" </td>\n",
" </tr>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"id": "d90e04a2-5b8a-4fd5-9db8-27c02f033313",
"metadata": {},
"source": [
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../assets/important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h1 style=\"color:#900;\">Important Note</h1>\n",
" <span style=\"color:#900;\">\n",
" In this lab, I use high end models GPT 5, Claude 4.5 Sonnet, Gemini 2.5 Pro, Grok 4, which are the slightly higher priced models. The costs are still low, but if you'd prefer to keep costs ultra low, please pick lower cost models like gpt-5-nano.\n",
" </span>\n",
" </td>\n",
" </tr>\n",
"</table>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e610bf56-a46e-4aff-8de1-ab49d62b1ad3",
"metadata": {},
"outputs": [],
"source": [
"# imports\n",
"\n",
"import os\n",
"import io\n",
"import sys\n",
"from dotenv import load_dotenv\n",
"from openai import OpenAI\n",
"import gradio as gr\n",
"import subprocess\n",
"from IPython.display import Markdown, display\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4f672e1c-87e9-4865-b760-370fa605e614",
"metadata": {},
"outputs": [],
"source": [
"load_dotenv(override=True)\n",
"openai_api_key = os.getenv('OPENAI_API_KEY')\n",
"anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')\n",
"google_api_key = os.getenv('GOOGLE_API_KEY')\n",
"grok_api_key = os.getenv('GROK_API_KEY')\n",
"groq_api_key = os.getenv('GROQ_API_KEY')\n",
"openrouter_api_key = os.getenv('OPENROUTER_API_KEY')\n",
"\n",
"if openai_api_key:\n",
" print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n",
"else:\n",
" print(\"OpenAI API Key not set\")\n",
" \n",
"if anthropic_api_key:\n",
" print(f\"Anthropic API Key exists and begins {anthropic_api_key[:7]}\")\n",
"else:\n",
" print(\"Anthropic API Key not set (and this is optional)\")\n",
"\n",
"if google_api_key:\n",
" print(f\"Google API Key exists and begins {google_api_key[:2]}\")\n",
"else:\n",
" print(\"Google API Key not set (and this is optional)\")\n",
"\n",
"if grok_api_key:\n",
" print(f\"Grok API Key exists and begins {grok_api_key[:4]}\")\n",
"else:\n",
" print(\"Grok API Key not set (and this is optional)\")\n",
"\n",
"if groq_api_key:\n",
" print(f\"Groq API Key exists and begins {groq_api_key[:4]}\")\n",
"else:\n",
" print(\"Groq API Key not set (and this is optional)\")\n",
"\n",
"if openrouter_api_key:\n",
" print(f\"OpenRouter API Key exists and begins {openrouter_api_key[:6]}\")\n",
"else:\n",
" print(\"OpenRouter API Key not set (and this is optional)\")\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "59863df1",
"metadata": {},
"outputs": [],
"source": [
"# Connect to client libraries\n",
"\n",
"openai = OpenAI()\n",
"\n",
"anthropic_url = \"https://api.anthropic.com/v1/\"\n",
"gemini_url = \"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
"grok_url = \"https://api.x.ai/v1\"\n",
"groq_url = \"https://api.groq.com/openai/v1\"\n",
"ollama_url = \"http://localhost:11434/v1\"\n",
"openrouter_url = \"https://openrouter.ai/api/v1\"\n",
"\n",
"anthropic = OpenAI(api_key=anthropic_api_key, base_url=anthropic_url)\n",
"gemini = OpenAI(api_key=google_api_key, base_url=gemini_url)\n",
"grok = OpenAI(api_key=grok_api_key, base_url=grok_url)\n",
"groq = OpenAI(api_key=groq_api_key, base_url=groq_url)\n",
"ollama = OpenAI(api_key=\"ollama\", base_url=ollama_url)\n",
"openrouter = OpenAI(api_key=openrouter_api_key, base_url=openrouter_url)\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8aa149ed-9298-4d69-8fe2-8f5de0f667da",
"metadata": {},
"outputs": [],
"source": [
"models = [\"gpt-5\", \"claude-sonnet-4-5-20250929\", \"grok-4\", \"gemini-2.5-pro\", \"qwen2.5-coder\", \"deepseek-coder-v2\", \"gpt-oss:20b\", \"qwen/qwen3-coder-30b-a3b-instruct\", \"openai/gpt-oss-120b\", ]\n",
"\n",
"clients = {\"gpt-5\": openai, \"claude-sonnet-4-5-20250929\": anthropic, \"grok-4\": grok, \"gemini-2.5-pro\": gemini, \"openai/gpt-oss-120b\": groq, \"qwen2.5-coder\": ollama, \"deepseek-coder-v2\": ollama, \"gpt-oss:20b\": ollama, \"qwen/qwen3-coder-30b-a3b-instruct\": openrouter}\n",
"\n",
"# Want to keep costs ultra-low? Replace this with models of your choice, using the examples from yesterday"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "68c1f1be",
"metadata": {},
"outputs": [],
"source": [
"from system_info import retrieve_system_info, rust_toolchain_info\n",
"\n",
"system_info = retrieve_system_info()\n",
"rust_info = rust_toolchain_info()\n",
"rust_info"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b8bd44f5",
"metadata": {},
"outputs": [],
"source": [
"message = f\"\"\"\n",
"Here is a report of the system information for my computer.\n",
"I want to run a Rust compiler to compile a single rust file called main.rs and then execute it in the simplest way possible.\n",
"Please reply with whether I need to install a Rust toolchain to do this. If so, please provide the simplest step by step instructions to do so.\n",
"\n",
"If I'm already set up to compile Rust code, then I'd like to run something like this in Python to compile and execute the code:\n",
"```python\n",
"compile_command = # something here - to achieve the fastest possible runtime performance\n",
"compile_result = subprocess.run(compile_command, check=True, text=True, capture_output=True)\n",
"run_command = # something here\n",
"run_result = subprocess.run(run_command, check=True, text=True, capture_output=True)\n",
"return run_result.stdout\n",
"```\n",
"Please tell me exactly what I should use for the compile_command and run_command.\n",
"Have the maximum possible runtime performance in mind; compile time can be slow. Fastest possible runtime performance for this platform is key.\n",
"Reply with the commands in markdown.\n",
"\n",
"System information:\n",
"{system_info}\n",
"\n",
"Rust toolchain information:\n",
"{rust_info}\n",
"\"\"\"\n",
"\n",
"response = openai.chat.completions.create(model=models[0], messages=[{\"role\": \"user\", \"content\": message}])\n",
"display(Markdown(response.choices[0].message.content))"
]
},
{
"cell_type": "markdown",
"id": "81e92c12",
"metadata": {},
"source": [
"## For C++, overwrite this with the commands from yesterday, or for Rust, use the new commands\n",
"\n",
"Or just use the website like yesterday:\n",
"\n",
" https://www.programiz.com/cpp-programming/online-compiler/"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d734a634",
"metadata": {},
"outputs": [],
"source": [
"compile_command = [\n",
" \"/Users/ed/.cargo/bin/rustc\",\n",
" \"main.rs\",\n",
" \"-C\", \"opt-level=3\",\n",
" \"-C\", \"target-cpu=native\",\n",
" \"-C\", \"codegen-units=1\",\n",
" \"-C\", \"lto=fat\",\n",
" \"-C\", \"panic=abort\",\n",
" \"-C\", \"strip=symbols\",\n",
" \"-o\", \"main\",\n",
"]\n",
"\n",
"run_command = [\"./main\"]\n"
]
},
{
"cell_type": "markdown",
"id": "f0b0a437",
"metadata": {},
"source": [
"## And now, on with the main task"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6896636f-923e-4a2c-9d6c-fac07828a201",
"metadata": {},
"outputs": [],
"source": [
"language = \"Rust\" # or \"C++\"\n",
"extension = \"rs\" if language == \"Rust\" else \"cpp\"\n",
"\n",
"system_prompt = f\"\"\"\n",
"Your task is to convert Python code into high performance {language} code.\n",
"Respond only with {language} code. Do not provide any explanation other than occasional comments.\n",
"The {language} response needs to produce an identical output in the fastest possible time.\n",
"\"\"\"\n",
"\n",
"def user_prompt_for(python):\n",
" return f\"\"\"\n",
"Port this Python code to {language} with the fastest possible implementation that produces identical output in the least time.\n",
"The system information is:\n",
"{system_info}\n",
"Your response will be written to a file called main.{language} and then compiled and executed; the compilation command is:\n",
"{compile_command}\n",
"Respond only with {language} code.\n",
"Python code to port:\n",
"\n",
"```python\n",
"{python}\n",
"```\n",
"\"\"\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8e7b3546-57aa-4c29-bc5d-f211970d04eb",
"metadata": {},
"outputs": [],
"source": [
"def messages_for(python):\n",
" return [\n",
" {\"role\": \"system\", \"content\": system_prompt},\n",
" {\"role\": \"user\", \"content\": user_prompt_for(python)}\n",
" ]\n",
" "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c6190659-f54c-4951-bef4-4960f8e51cc4",
"metadata": {},
"outputs": [],
"source": [
"def write_output(code):\n",
" with open(f\"main.{extension}\", \"w\") as f:\n",
" f.write(code)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e7d2fea8-74c6-4421-8f1e-0e76d5b201b9",
"metadata": {},
"outputs": [],
"source": [
"def port(model, python):\n",
" client = clients[model]\n",
" reasoning_effort = \"high\" if 'gpt' in model else None\n",
" response = client.chat.completions.create(model=model, messages=messages_for(python), reasoning_effort=reasoning_effort)\n",
" reply = response.choices[0].message.content\n",
" reply = reply.replace('```cpp','').replace('```rust','').replace('```','')\n",
" return reply"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7fe1cd4b-d2c5-4303-afed-2115a3fef200",
"metadata": {},
"outputs": [],
"source": [
"def run_python(code):\n",
" globals_dict = {\"__builtins__\": __builtins__}\n",
"\n",
" buffer = io.StringIO()\n",
" old_stdout = sys.stdout\n",
" sys.stdout = buffer\n",
"\n",
" try:\n",
" exec(code, globals_dict)\n",
" output = buffer.getvalue()\n",
" except Exception as e:\n",
" output = f\"Error: {e}\"\n",
" finally:\n",
" sys.stdout = old_stdout\n",
"\n",
" return output"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4194e40c-04ab-4940-9d64-b4ad37c5bb40",
"metadata": {},
"outputs": [],
"source": [
"# Use the commands from GPT 5\n",
"\n",
"def compile_and_run(code):\n",
" write_output(code)\n",
" try:\n",
" subprocess.run(compile_command, check=True, text=True, capture_output=True)\n",
" run_result = subprocess.run(run_command, check=True, text=True, capture_output=True)\n",
" return run_result.stdout\n",
" except subprocess.CalledProcessError as e:\n",
" return f\"An error occurred:\\n{e.stderr}\""
]
},
{
"cell_type": "markdown",
"id": "c3b497b3-f569-420e-b92e-fb0f49957ce0",
"metadata": {},
"source": [
"python_hard = \"\"\"# Be careful to support large numbers\n",
"\n",
"def lcg(seed, a=1664525, c=1013904223, m=2**32):\n",
" value = seed\n",
" while True:\n",
" value = (a * value + c) % m\n",
" yield value\n",
" \n",
"def max_subarray_sum(n, seed, min_val, max_val):\n",
" lcg_gen = lcg(seed)\n",
" random_numbers = [next(lcg_gen) % (max_val - min_val + 1) + min_val for _ in range(n)]\n",
" max_sum = float('-inf')\n",
" for i in range(n):\n",
" current_sum = 0\n",
" for j in range(i, n):\n",
" current_sum += random_numbers[j]\n",
" if current_sum > max_sum:\n",
" max_sum = current_sum\n",
" return max_sum\n",
"\n",
"def total_max_subarray_sum(n, initial_seed, min_val, max_val):\n",
" total_sum = 0\n",
" lcg_gen = lcg(initial_seed)\n",
" for _ in range(20):\n",
" seed = next(lcg_gen)\n",
" total_sum += max_subarray_sum(n, seed, min_val, max_val)\n",
" return total_sum\n",
"\n",
"# Parameters\n",
"n = 10000 # Number of random numbers\n",
"initial_seed = 42 # Initial seed for the LCG\n",
"min_val = -10 # Minimum value of random numbers\n",
"max_val = 10 # Maximum value of random numbers\n",
"\n",
"# Timing the function\n",
"import time\n",
"start_time = time.time()\n",
"result = total_max_subarray_sum(n, initial_seed, min_val, max_val)\n",
"end_time = time.time()\n",
"\n",
"print(\"Total Maximum Subarray Sum (20 runs):\", result)\n",
"print(\"Execution Time: {:.6f} seconds\".format(end_time - start_time))\n",
"\"\"\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "465d6cad",
"metadata": {},
"outputs": [],
"source": [
"from styles import CSS\n",
"\n",
"with gr.Blocks(css=CSS, theme=gr.themes.Monochrome(), title=f\"Port from Python to {language}\") as ui:\n",
" with gr.Row(equal_height=True):\n",
" with gr.Column(scale=6):\n",
" python = gr.Code(\n",
" label=\"Python (original)\",\n",
" value=python_hard,\n",
" language=\"python\",\n",
" lines=26\n",
" )\n",
" with gr.Column(scale=6):\n",
" cpp = gr.Code(\n",
" label=f\"{language} (generated)\",\n",
" value=\"\",\n",
" language=\"cpp\",\n",
" lines=26\n",
" )\n",
"\n",
" with gr.Row(elem_classes=[\"controls\"]):\n",
" python_run = gr.Button(\"Run Python\", elem_classes=[\"run-btn\", \"py\"])\n",
" model = gr.Dropdown(models, value=models[0], show_label=False)\n",
" convert = gr.Button(f\"Port to {language}\", elem_classes=[\"convert-btn\"])\n",
" cpp_run = gr.Button(f\"Run {language}\", elem_classes=[\"run-btn\", \"cpp\"])\n",
"\n",
" with gr.Row(equal_height=True):\n",
" with gr.Column(scale=6):\n",
" python_out = gr.TextArea(label=\"Python result\", lines=8, elem_classes=[\"py-out\"])\n",
" with gr.Column(scale=6):\n",
" cpp_out = gr.TextArea(label=f\"{language} result\", lines=8, elem_classes=[\"cpp-out\"])\n",
"\n",
" convert.click(fn=port, inputs=[model, python], outputs=[cpp])\n",
" python_run.click(fn=run_python, inputs=[python], outputs=[python_out])\n",
" cpp_run.click(fn=compile_and_run, inputs=[cpp], outputs=[cpp_out])\n",
"\n",
"ui.launch(inbrowser=True)\n"
]
},
{
"cell_type": "markdown",
"id": "2311ada8",
"metadata": {},
"source": [
"## RESULTS!\n",
"\n",
"Qwen 2.5 Coder: FAIL \n",
"Gemini 2.5 Pro: FAIL \n",
"DeepSeek Coder v2: FAIL \n",
"Qwen3 Coder 30B: FAIL \n",
"Claude Sonnet 4.5: FAIL \n",
"GPT-5: FAIL \n",
"\n",
"3rd place: GPT-oss-20B: 0.000341 \n",
"2nd place: Grok 4: 0.000317 \n",
"**1st place: OpenAI GPT-OSS 120B: 0.000304** "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b9b51dc7",
"metadata": {},
"outputs": [],
"source": [
"print(f\"In Ed's experimenet, the GPT-OSS 120B model outcome is {33.755209/0.000304:,.0f} times faster than the Python code.\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6197bb97",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

65
week4/optimized.cpp
View File

@@ -1,65 +0,0 @@
#include <iostream>
#include <vector>
#include <chrono>
#include <limits>
#include <iomanip>
class LCG {
private:
uint64_t value;
const uint64_t a = 1664525;
const uint64_t c = 1013904223;
const uint64_t m = 1ULL << 32;
public:
LCG(uint64_t seed) : value(seed) {}
uint64_t next() {
value = (a * value + c) % m;
return value;
}
};
int64_t max_subarray_sum(int n, uint64_t seed, int min_val, int max_val) {
LCG lcg(seed);
std::vector<int> random_numbers(n);
for (int i = 0; i < n; ++i) {
random_numbers[i] = static_cast<int>(lcg.next() % (max_val - min_val + 1) + min_val);
}
int64_t max_sum = std::numeric_limits<int64_t>::min();
int64_t current_sum = 0;
for (int i = 0; i < n; ++i) {
current_sum = std::max(static_cast<int64_t>(random_numbers[i]), current_sum + random_numbers[i]);
max_sum = std::max(max_sum, current_sum);
}
return max_sum;
}
int64_t total_max_subarray_sum(int n, uint64_t initial_seed, int min_val, int max_val) {
int64_t total_sum = 0;
LCG lcg(initial_seed);
for (int i = 0; i < 20; ++i) {
uint64_t seed = lcg.next();
total_sum += max_subarray_sum(n, seed, min_val, max_val);
}
return total_sum;
}
int main() {
int n = 10000;
uint64_t initial_seed = 42;
int min_val = -10;
int max_val = 10;
auto start_time = std::chrono::high_resolution_clock::now();
int64_t result = total_max_subarray_sum(n, initial_seed, min_val, max_val);
auto end_time = std::chrono::high_resolution_clock::now();
auto duration = std::chrono::duration_cast<std::chrono::microseconds>(end_time - start_time);
std::cout << "Total Maximum Subarray Sum (20 runs): " << result << std::endl;
std::cout << "Execution Time: " << std::fixed << std::setprecision(6) << duration.count() / 1e6 << " seconds" << std::endl;
return 0;
}

7
week4/simple.cpp
View File

@@ -1,7 +0,0 @@
#include <iostream>
int main() {
std::cout << "Hello";
return 0;
}

60
week4/styles.py Normal file
View File

@@ -0,0 +1,60 @@
CSS = """
:root {
--py-color: #209dd7;
--cpp-color: #ecad0a;
--accent: #753991;
--card: #161a22;
--text: #e9eef5;
}
/* Full-width layout */
.gradio-container {
max-width: 100% !important;
padding: 0 40px !important;
}
/* Code card styling */
.card {
background: var(--card);
border: 1px solid rgba(255,255,255,.08);
border-radius: 14px;
padding: 10px;
}
/* Buttons */
.convert-btn button {
background: var(--accent) !important;
border-color: rgba(255,255,255,.12) !important;
color: white !important;
font-weight: 700;
}
.run-btn button {
background: #202631 !important;
color: var(--text) !important;
border-color: rgba(255,255,255,.12) !important;
}
.run-btn.py button:hover { box-shadow: 0 0 0 2px var(--py-color) inset; }
.run-btn.cpp button:hover { box-shadow: 0 0 0 2px var(--cpp-color) inset; }
.convert-btn button:hover { box-shadow: 0 0 0 2px var(--accent) inset; }
/* Outputs with color tint */
.py-out textarea {
background: linear-gradient(180deg, rgba(32,157,215,.18), rgba(32,157,215,.10));
border: 1px solid rgba(32,157,215,.35) !important;
color: rgba(32,157,215,1) !important;
font-weight: 600;
}
.cpp-out textarea {
background: linear-gradient(180deg, rgba(236,173,10,.22), rgba(236,173,10,.12));
border: 1px solid rgba(236,173,10,.45) !important;
color: rgba(236,173,10,1) !important;
font-weight: 600;
}
/* Align controls neatly */
.controls .wrap {
gap: 10px;
justify-content: center;
align-items: center;
}
"""

359
week4/system_info.py Normal file
View File

@@ -0,0 +1,359 @@
import os
import platform
import shutil
import subprocess
# ------------------------- helpers -------------------------
def _run(cmd, timeout=3):
"""Run a command safely. Returns stdout text or ''.
Accepts either a string (shell) or list (no shell)."""
try:
if isinstance(cmd, str):
return subprocess.check_output(
cmd, shell=True, text=True, stderr=subprocess.DEVNULL, timeout=timeout
).strip()
else:
return subprocess.check_output(
cmd, shell=False, text=True, stderr=subprocess.DEVNULL, timeout=timeout
).strip()
except Exception:
return ""
def _first_line(s: str) -> str:
s = (s or "").strip()
return s.splitlines()[0].strip() if s else ""
def _which(name: str) -> str:
return shutil.which(name) or ""
def _bool_from_output(s: str) -> bool:
return s.strip() in {"1", "true", "True", "YES", "Yes", "yes"}
# ------------------------- OS & env -------------------------
def _os_block():
sysname = platform.system() # 'Windows', 'Darwin', 'Linux'
machine = platform.machine() or ""
release = platform.release() or ""
version = platform.version() or ""
kernel = release if sysname == "Windows" else (_run(["uname", "-r"]) or release)
distro = {"name": "", "version": ""}
if sysname == "Linux":
# Best-effort parse of /etc/os-release
try:
with open("/etc/os-release", "r") as f:
data = {}
for line in f:
if "=" in line:
k, v = line.rstrip().split("=", 1)
data[k] = v.strip('"')
distro["name"] = data.get("PRETTY_NAME") or data.get("NAME", "")
distro["version"] = data.get("VERSION_ID") or data.get("VERSION", "")
except Exception:
pass
# WSL / Rosetta detection (harmless if not present)
wsl = False
if sysname != "Windows":
try:
with open("/proc/version", "r") as f:
v = f.read().lower()
wsl = ("microsoft" in v) or ("wsl" in v)
except Exception:
wsl = False
rosetta = False
if sysname == "Darwin":
rosetta = _bool_from_output(_run(["sysctl", "-in", "sysctl.proc_translated"]))
# Target triple (best effort)
target = ""
for cc in ("clang", "gcc"):
if _which(cc):
out = _run([cc, "-dumpmachine"])
if out:
target = _first_line(out)
break
return {
"system": sysname,
"arch": machine,
"release": release,
"version": version,
"kernel": kernel,
"distro": distro if sysname == "Linux" else None,
"wsl": wsl,
"rosetta2_translated": rosetta,
"target_triple": target,
}
# ------------------------- package managers -------------------------
def _package_managers():
sysname = platform.system()
pms = []
if sysname == "Windows":
for pm in ("winget", "choco", "scoop"):
if _which(pm):
pms.append(pm)
elif sysname == "Darwin":
if _run(["xcode-select", "-p"]):
pms.append("xcode-select (CLT)")
for pm in ("brew", "port"):
if _which(pm):
pms.append(pm)
else:
for pm in ("apt", "dnf", "yum", "pacman", "zypper", "apk", "emerge"):
if _which(pm):
pms.append(pm)
return pms
# ------------------------- CPU (minimal) -------------------------
def _cpu_block():
sysname = platform.system()
brand = ""
# A simple brand/model read per OS; ignore failures
if sysname == "Linux":
brand = _run("grep -m1 'model name' /proc/cpuinfo | cut -d: -f2").strip()
elif sysname == "Darwin":
brand = _run(["sysctl", "-n", "machdep.cpu.brand_string"])
elif sysname == "Windows":
brand = _run('powershell -NoProfile -Command "(Get-CimInstance Win32_Processor).Name"')
if not brand:
brand = _run("wmic cpu get Name /value").replace("Name=", "").strip()
# Logical cores always available; physical is best-effort
cores_logical = os.cpu_count() or 0
cores_physical = 0
if sysname == "Darwin":
cores_physical = int(_run(["sysctl", "-n", "hw.physicalcpu"]) or "0")
elif sysname == "Windows":
cores_physical = int(
_run('powershell -NoProfile -Command "(Get-CimInstance Win32_Processor).NumberOfCores"')
or "0"
)
elif sysname == "Linux":
# This is a quick approximation; fine for our use (parallel -j suggestions)
try:
# Count unique "core id" per physical id
mapping = _run("LC_ALL=C lscpu -p=CORE,SOCKET | grep -v '^#'").splitlines()
unique = set(tuple(line.split(",")) for line in mapping if "," in line)
cores_physical = len(unique) or 0
except Exception:
cores_physical = 0
# A tiny SIMD hint set (best-effort, optional)
simd = []
if sysname == "Linux":
flags = _run("grep -m1 'flags' /proc/cpuinfo | cut -d: -f2")
if flags:
fset = set(flags.upper().split())
for x in ("AVX512F", "AVX2", "AVX", "FMA", "SSE4_2", "NEON", "SVE"):
if x in fset:
simd.append(x)
elif sysname == "Darwin":
feats = (
(
_run(["sysctl", "-n", "machdep.cpu.features"])
+ " "
+ _run(["sysctl", "-n", "machdep.cpu.leaf7_features"])
)
.upper()
.split()
)
for x in ("AVX512F", "AVX2", "AVX", "FMA", "SSE4_2", "NEON", "SVE"):
if x in feats:
simd.append(x)
# On Windows, skip flags — brand typically suffices for MSVC /arch choice.
return {
"brand": brand.strip(),
"cores_logical": cores_logical,
"cores_physical": cores_physical,
"simd": sorted(set(simd)),
}
# ------------------------- toolchain presence -------------------------
def _toolchain_block():
def ver_line(exe, args=("--version",)):
p = _which(exe)
if not p:
return ""
out = _run([p, *args])
return _first_line(out)
gcc = ver_line("gcc")
gpp = ver_line("g++")
clang = ver_line("clang")
# MSVC cl (only available inside proper dev shell; handle gracefully)
msvc_cl = ""
cl_path = _which("cl")
if cl_path:
msvc_cl = _first_line(_run("cl 2>&1"))
# Build tools (presence + short version line)
cmake = ver_line("cmake")
ninja = _first_line(_run([_which("ninja"), "--version"])) if _which("ninja") else ""
make = ver_line("make")
# Linker (we only care if lld is available)
lld = ver_line("ld.lld")
return {
"compilers": {"gcc": gcc, "g++": gpp, "clang": clang, "msvc_cl": msvc_cl},
"build_tools": {"cmake": cmake, "ninja": ninja, "make": make},
"linkers": {"ld_lld": lld},
}
# ------------------------- public API -------------------------
def retrieve_system_info():
"""
Returns a compact dict with enough info for an LLM to:
- Pick an install path (winget/choco/scoop, Homebrew/Xcode CLT, apt/dnf/...),
- Choose a compiler family (MSVC/clang/gcc),
- Suggest safe optimization flags (e.g., -O3/-march=native or MSVC /O2),
- Decide on a build system (cmake+ninja) and parallel -j value.
"""
return {
"os": _os_block(),
"package_managers": _package_managers(),
"cpu": _cpu_block(),
"toolchain": _toolchain_block(),
}
def rust_toolchain_info():
"""
Return a dict with Rust-related settings:
- presence and paths for rustc / cargo / rustup / rust-analyzer
- versions
- active/default toolchain (if rustup is present)
- installed targets
- common env vars (CARGO_HOME, RUSTUP_HOME, RUSTFLAGS, CARGO_BUILD_TARGET)
- simple execution examples
Works on Windows, macOS, and Linux. Uses the existing helpers: _run, _which, _first_line.
"""
info = {
"installed": False,
"rustc": {"path": "", "version": "", "host_triple": "", "release": "", "commit_hash": ""},
"cargo": {"path": "", "version": ""},
"rustup": {
"path": "",
"version": "",
"active_toolchain": "",
"default_toolchain": "",
"toolchains": [],
"targets_installed": [],
},
"rust_analyzer": {"path": ""},
"env": {
"CARGO_HOME": os.environ.get("CARGO_HOME", ""),
"RUSTUP_HOME": os.environ.get("RUSTUP_HOME", ""),
"RUSTFLAGS": os.environ.get("RUSTFLAGS", ""),
"CARGO_BUILD_TARGET": os.environ.get("CARGO_BUILD_TARGET", ""),
},
"execution_examples": [],
}
# Paths
rustc_path = _which("rustc")
cargo_path = _which("cargo")
rustup_path = _which("rustup")
ra_path = _which("rust-analyzer")
info["rustc"]["path"] = rustc_path or ""
info["cargo"]["path"] = cargo_path or ""
info["rustup"]["path"] = rustup_path or ""
info["rust_analyzer"]["path"] = ra_path or ""
# Versions & verbose details
if rustc_path:
ver_line = _first_line(_run([rustc_path, "--version"]))
info["rustc"]["version"] = ver_line
verbose = _run([rustc_path, "--version", "--verbose"])
host = release = commit = ""
for line in verbose.splitlines():
if line.startswith("host:"):
host = line.split(":", 1)[1].strip()
elif line.startswith("release:"):
release = line.split(":", 1)[1].strip()
elif line.startswith("commit-hash:"):
commit = line.split(":", 1)[1].strip()
info["rustc"]["host_triple"] = host
info["rustc"]["release"] = release
info["rustc"]["commit_hash"] = commit
if cargo_path:
info["cargo"]["version"] = _first_line(_run([cargo_path, "--version"]))
if rustup_path:
info["rustup"]["version"] = _first_line(_run([rustup_path, "--version"]))
# Active toolchain
active = _first_line(_run([rustup_path, "show", "active-toolchain"]))
info["rustup"]["active_toolchain"] = active
# Default toolchain (best effort)
# Try parsing `rustup toolchain list` and pick the line with "(default)"
tlist = _run([rustup_path, "toolchain", "list"]).splitlines()
info["rustup"]["toolchains"] = [t.strip() for t in tlist if t.strip()]
default_tc = ""
for line in tlist:
if "(default)" in line:
default_tc = line.strip()
break
if not default_tc:
# Fallback: sometimes `rustup show` includes "default toolchain: ..."
for line in _run([rustup_path, "show"]).splitlines():
if "default toolchain:" in line:
default_tc = line.split(":", 1)[1].strip()
break
info["rustup"]["default_toolchain"] = default_tc
# Installed targets
targets = _run([rustup_path, "target", "list", "--installed"]).split()
info["rustup"]["targets_installed"] = targets
# Execution examples (only include what will work on this system)
exec_examples = []
if cargo_path:
exec_examples.append(f'"{cargo_path}" build')
exec_examples.append(f'"{cargo_path}" run')
exec_examples.append(f'"{cargo_path}" test')
if rustc_path:
exec_examples.append(f'"{rustc_path}" hello.rs -o hello')
info["execution_examples"] = exec_examples
# Installed?
info["installed"] = bool(rustc_path or cargo_path or rustup_path)
# Fill in default homes if env vars are empty but typical locations exist
def _maybe_default_home(env_val, default_basename):
if env_val:
return env_val
home = os.path.expanduser("~") or ""
candidate = os.path.join(home, default_basename) if home else ""
return candidate if candidate and os.path.isdir(candidate) else ""
info["env"]["CARGO_HOME"] = _maybe_default_home(info["env"]["CARGO_HOME"], ".cargo")
info["env"]["RUSTUP_HOME"] = _maybe_default_home(info["env"]["RUSTUP_HOME"], ".rustup")
return info

4
week5/day4.5.ipynb
View File

@@ -38,7 +38,9 @@
"# Mac users - this may fail if you don't have a recent version of MacOS\n",
"# In which case I recommend you skip this lab -- FAISS is not essential! (Or upgrade MacOS if you wish..)\n",
"\n",
"!pip install faiss-cpu"
"!uv add faiss-cpu\n",
"\n",
"# Or use !pip install if you're on the Original version of the course and you're not using uv"
]
},
{

2
week5/day4.ipynb
View File

@@ -313,7 +313,7 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#900;\">PLEASE READ ME! Ignoring the Deprecation Warning</h2>\n",

28
week8/day1.ipynb
View File

@@ -20,7 +20,7 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#900;\">Especially important this week: pull the latest</h2>\n",
@@ -40,7 +40,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 1,
"id": "bc0e1c1c-be6a-4395-bbbd-eeafc9330d7e",
"metadata": {},
"outputs": [],
@@ -298,10 +298,28 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 2,
"id": "f56d1e55-2a03-4ce2-bb47-2ab6b9175a02",
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[2K\u001b[34m⠸\u001b[0m Creating objects.....\n",
"\u001b[38;5;244m└── \u001b[0m\u001b[34m⠋\u001b[0m Creating mount /Users/ed/dev/llm_engineering/week8/pricer_service2.py: \n",
"\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[34m⠴\u001b[0m Creating objects...\n",
"\u001b[38;5;244m├── \u001b[0m🔨 Created mount /Users/ed/dev/llm_engineering/week8/pricer_service2.py\n",
"\u001b[38;5;244m└── \u001b[0m🔨 Created function Pricer.*.\n",
"\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[32m✓\u001b[0m Created objects.\n",
"\u001b[38;5;244m├── \u001b[0m🔨 Created mount /Users/ed/dev/llm_engineering/week8/pricer_service2.py\n",
"\u001b[38;5;244m└── \u001b[0m🔨 Created function Pricer.*.\n",
"\u001b[32m✓\u001b[0m App deployed in 0.723s! 🎉\n",
"\n",
"View Deployment: \u001b[35mhttps://modal.com/apps/ed-donner/main/deployed/pricer-service\u001b[0m\n"
]
}
],
"source": [
"# You can also run \"modal deploy -m pricer_service2\" at the command line in an activated environment\n",
"\n",
@@ -394,7 +412,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.12"
"version": "3.11.13"
}
},
"nbformat": 4,

2
week8/day2.0.ipynb
View File

@@ -341,7 +341,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.12"
"version": "3.11.13"
}
},
"nbformat": 4,

2
week8/day2.3.ipynb
View File

@@ -519,7 +519,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.12"
"version": "3.11.13"
}
},
"nbformat": 4,

2
week8/day2.4.ipynb
View File

@@ -425,7 +425,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.12"
"version": "3.11.13"
}
},
"nbformat": 4,

2
week8/day4.ipynb
View File

@@ -74,7 +74,7 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#f71;\">Additional resource: more sophisticated planning agent</h2>\n",

4
week8/day5.ipynb
View File

@@ -164,7 +164,7 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#900;\">But wait!! There's more..</h2>\n",
@@ -186,7 +186,7 @@
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../thankyou.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" <img src=\"../assets/thankyou.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#090;\">CONGRATULATIONS AND THANK YOU!!!</h2>\n",