Launching refreshed version of LLM Engineering weeks 1-4 - see README

guides/10_intermediate_python.ipynb

@@ -0,0 +1,482 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "5c291475-8c7c-461c-9b12-545a887b2432",
+   "metadata": {},
+   "source": [
+    "# Intermediate Level Python\n",
+    "\n",
+    "## A briefing on more advanced features of Python\n",
+    "\n",
+    "This section assumes you're up to speed on the foundations - and now we cover some important features of python that we use on the course.\n",
+    "\n",
+    "1. Comprehensions  \n",
+    "2. Generators  \n",
+    "3. Sub-classes, Type Hints, Pydantic  \n",
+    "4. Decorators\n",
+    "5. Docker (not really python, but we use it to run python code!)\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5802e2f0-0ea0-4237-bbb7-f375a34260f0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# First let's create some things:\n",
+    "\n",
+    "fruits = [\"Apples\", \"Bananas\", \"Pears\"]\n",
+    "\n",
+    "book1 = {\"title\": \"Great Expectations\", \"author\": \"Charles Dickens\"}\n",
+    "book2 = {\"title\": \"Bleak House\", \"author\": \"Charles Dickens\"}\n",
+    "book3 = {\"title\": \"An Book By No Author\"}\n",
+    "book4 = {\"title\": \"Moby Dick\", \"author\": \"Herman Melville\"}\n",
+    "\n",
+    "books = [book1, book2, book3, book4]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9b941e6a-3658-4144-a8d4-72f5e72f3707",
+   "metadata": {},
+   "source": [
+    "# Part 1: List and dict comprehensions"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "61992bb8-735d-4dad-8747-8c10b63aec82",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Simple enough to start\n",
+    "\n",
+    "for fruit in fruits:\n",
+    "    print(fruit)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c89c3842-9b74-47fa-8424-0fcb08e4177c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Let's make a new version of fruits\n",
+    "\n",
+    "fruits_shouted = []\n",
+    "for fruit in fruits:\n",
+    "    fruits_shouted.append(fruit.upper())\n",
+    "\n",
+    "fruits_shouted"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4ec13b3a-9545-44f1-874a-2910a0663560",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# You probably already know this\n",
+    "# There's a nice Python construct called \"list comprehension\" that does this:\n",
+    "\n",
+    "fruits_shouted2 = [fruit.upper() for fruit in fruits]\n",
+    "fruits_shouted2"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ecc08c3c-181d-4b64-a3e1-b0ccffc6c0cd",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# But you may not know that you can do this to create dictionaries, too:\n",
+    "\n",
+    "fruit_mapping = {fruit: fruit.upper() for fruit in fruits}\n",
+    "fruit_mapping"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "500c2406-00d2-4793-b57b-f49b612760c8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# you can also use the if statement to filter the results\n",
+    "\n",
+    "fruits_with_longer_names_shouted = [fruit.upper() for fruit in fruits if len(fruit)>5]\n",
+    "fruits_with_longer_names_shouted"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "38c11c34-d71e-45ba-945b-a3d37dc29793",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "fruit_mapping_unless_starts_with_a = {fruit: fruit.upper() for fruit in fruits if not fruit.startswith('A')}\n",
+    "fruit_mapping_unless_starts_with_a"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5c97d8e8-31de-4afa-973e-28d8e5cab749",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Another comprehension\n",
+    "\n",
+    "[book['title'] for book in books]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "50be0edc-a4cd-493f-a680-06080bb497b4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# This code will fail with an error because one of our books doesn't have an author\n",
+    "\n",
+    "[book['author'] for book in books]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "53794083-cc09-4edb-b448-2ffb7e8495c2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# But this will work, because get() returns None\n",
+    "\n",
+    "[book.get('author') for book in books]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b8e4b859-24f8-4016-8d74-c2cef226d049",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# And this variation will filter out the None\n",
+    "\n",
+    "[book.get('author') for book in books if book.get('author')]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c44bb999-52b4-4dee-810b-8a400db8f25f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# And this version will convert it into a set, removing duplicates\n",
+    "\n",
+    "set([book.get('author') for book in books if book.get('author')])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "80a65156-6192-4bb4-b4e6-df3fdc933891",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# And finally, this version is even nicer\n",
+    "# curly braces creates a set, so this is a set comprehension\n",
+    "\n",
+    "{book.get('author') for book in books if book.get('author')}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c100e5db-5438-4715-921c-3f7152f83f4a",
+   "metadata": {},
+   "source": [
+    "# Part 2: Generators\n",
+    "\n",
+    "We use Generators in the course because AI models can stream back results.\n",
+    "\n",
+    "If you've not used Generators before, please start with this excellent intro from ChatGPT:\n",
+    "\n",
+    "https://chatgpt.com/share/672faa6e-7dd0-8012-aae5-44fc0d0ec218\n",
+    "\n",
+    "Try pasting some of its examples into a cell."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1efc26fa-9144-4352-9a17-dfec1d246aad",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# First define a generator; it looks like a function, but it has yield instead of return\n",
+    "\n",
+    "import time\n",
+    "\n",
+    "def come_up_with_fruit_names():\n",
+    "    for fruit in fruits:\n",
+    "        time.sleep(1) # thinking of a fruit\n",
+    "        yield fruit"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "eac338bb-285c-45c8-8a3e-dbfc41409ca3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Then use it\n",
+    "\n",
+    "for fruit in come_up_with_fruit_names():\n",
+    "    print(fruit)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f6880578-a3de-4502-952a-4572b95eb9ff",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Here's another one\n",
+    "\n",
+    "def authors_generator():\n",
+    "    for book in books:\n",
+    "        if book.get(\"author\"):\n",
+    "            yield book.get(\"author\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9e316f02-f87f-441d-a01f-024ade949607",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Use it\n",
+    "\n",
+    "for author in authors_generator():\n",
+    "    print(author)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7535c9d0-410e-4e56-a86c-ae6c0e16053f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Here's the same thing written with list comprehension\n",
+    "\n",
+    "def authors_generator():\n",
+    "    for author in [book.get(\"author\") for book in books if book.get(\"author\")]:\n",
+    "        yield author"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "dad34494-0f6c-4edb-b03f-b8d49ee186f2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Use it\n",
+    "\n",
+    "for author in authors_generator():\n",
+    "    print(author)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "abeb7e61-d8aa-4af0-b05a-ae17323e678c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Here's a nice shortcut\n",
+    "# You can use \"yield from\" to yield each item of an iterable\n",
+    "\n",
+    "def authors_generator():\n",
+    "    yield from [book.get(\"author\") for book in books if book.get(\"author\")]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "05b0cb43-aa83-4762-a797-d3beb0f22c44",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Use it\n",
+    "\n",
+    "for author in authors_generator():\n",
+    "    print(author)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "fdfea58e-d809-4dd4-b7b0-c26427f8be55",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# And finally - we can replace the list comprehension with a set comprehension\n",
+    "\n",
+    "def unique_authors_generator():\n",
+    "    yield from {book.get(\"author\") for book in books if book.get(\"author\")}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3e821d08-97be-4db9-9a5b-ce5dced3eff8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Use it\n",
+    "\n",
+    "for author in unique_authors_generator():\n",
+    "    print(author)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "905ba603-15d8-4d01-9a79-60ec293d7ca1",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# And for some fun - press the stop button in the toolbar when bored!\n",
+    "# It's like we've made our own Large Language Model... although not particularly large..\n",
+    "# See if you understand why it prints a letter at a time, instead of a word at a time. If you're unsure, try removing the keyword \"from\" everywhere in the code.\n",
+    "\n",
+    "import random\n",
+    "import time\n",
+    "\n",
+    "pronouns = [\"I\", \"You\", \"We\", \"They\"]\n",
+    "verbs = [\"eat\", \"detest\", \"bathe in\", \"deny the existence of\", \"resent\", \"pontificate about\", \"juggle\", \"impersonate\", \"worship\", \"misplace\", \"conspire with\", \"philosophize about\", \"tap dance on\", \"dramatically renounce\", \"secretly collect\"]\n",
+    "adjectives = [\"turqoise\", \"smelly\", \"arrogant\", \"festering\", \"pleasing\", \"whimsical\", \"disheveled\", \"pretentious\", \"wobbly\", \"melodramatic\", \"pompous\", \"fluorescent\", \"bewildered\", \"suspicious\", \"overripe\"]\n",
+    "nouns = [\"turnips\", \"rodents\", \"eels\", \"walruses\", \"kumquats\", \"monocles\", \"spreadsheets\", \"bagpipes\", \"wombats\", \"accordions\", \"mustaches\", \"calculators\", \"jellyfish\", \"thermostats\"]\n",
+    "\n",
+    "def infinite_random_sentences():\n",
+    "    while True:\n",
+    "        yield from random.choice(pronouns)\n",
+    "        yield \" \"\n",
+    "        yield from random.choice(verbs)\n",
+    "        yield \" \"\n",
+    "        yield from random.choice(adjectives)\n",
+    "        yield \" \"\n",
+    "        yield from random.choice(nouns)\n",
+    "        yield \". \"\n",
+    "\n",
+    "for letter in infinite_random_sentences():\n",
+    "    print(letter, end=\"\", flush=True)\n",
+    "    time.sleep(0.02)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "04832ea2-2447-4473-a449-104f80e24d85",
+   "metadata": {},
+   "source": [
+    "# Exercise\n",
+    "\n",
+    "Write some python classes for the books example.\n",
+    "\n",
+    "Write a Book class with a title and author. Include a method has_author()\n",
+    "\n",
+    "Write a BookShelf class with a list of books. Include a generator method unique_authors()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "35760406-fe6c-41f9-b0c0-3e8cf73aafd0",
+   "metadata": {},
+   "source": [
+    "# Part 3: Sub-classes, Type Hints, Pydantic\n",
+    "\n",
+    "Here are some intermediate level details of Classes from our AI friend, including use of type hints, inheritance and class methods. This includes a Book example.\n",
+    "\n",
+    "https://chatgpt.com/share/67348aca-65fc-8012-a4a9-fd1b8f04ba59\n",
+    "\n",
+    "And here is a comprehensive tutorial on Pydantic classes covering everything you need to know about Pydantic.\n",
+    "\n",
+    "https://chatgpt.com/share/68064537-6cfc-8012-93e1-f7dd0932f321"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6bbc9c63",
+   "metadata": {},
+   "source": [
+    "## Part 4: Decorators\n",
+    "\n",
+    "Here is a briefing, with an example from OpenAI Agents SDK:\n",
+    "\n",
+    "https://chatgpt.com/share/6806474d-3880-8012-b2a2-87b3ee4489da"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0beef7e9",
+   "metadata": {},
+   "source": [
+    "## Part 5: Docker\n",
+    "\n",
+    "Here is a convenient tutorial to introduce Docker.\n",
+    "\n",
+    "In the last section, this also covers an answer to a question in Week 6 - what does it mean to run an MCP server in Docker? But you can ignore this question if you're not on week 6 yet.\n",
+    "\n",
+    "https://chatgpt.com/share/6814bc1d-2f3c-8012-9b18-dddc82ea421b"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "73e215b2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# You need to install docker to run this example\n",
+    "# This will download the Docker image for python 3.12, create a container,\n",
+    "# Run some python code and print the result\n",
+    "\n",
+    "!docker run --rm python:3.12 python -c \"print(2 + 2)\""
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": ".venv",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}