LLM_Engineering_OLD/week1/day2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d15d8294-3328-4e07-ad16-8a03e9bbfdb9",
   "metadata": {},
   "source": [
    "# Welcome to the Day 2 Lab!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ada885d9-4d42-4d9b-97f0-74fbbbfe93a9",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left;\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#f71;\">Just before we get started --</h2>\n",
    "            <span style=\"color:#f71;\">I thought I'd take a second to point you at this page of useful resources for the course. This includes links to all the slides.<br/>\n",
    "            <a href=\"https://edwarddonner.com/2024/11/13/llm-engineering-resources/\">https://edwarddonner.com/2024/11/13/llm-engineering-resources/</a><br/>\n",
    "            Please keep this bookmarked, and I'll continue to add more useful links there over time.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79ffe36f",
   "metadata": {},
   "source": [
    "## First - let's talk about the Chat Completions API\n",
    "\n",
    "1. The simplest way to call an LLM\n",
    "2. It's called Chat Completions because it's saying: \"here is a conversation, please predict what should come next\"\n",
    "3. The Chat Completions API was invented by OpenAI, but it's so popular that everybody uses it!\n",
    "\n",
    "### We will start by calling OpenAI again - but don't worry non-OpenAI people, your time is coming!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e38f17a0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "API key found and looks good so far!\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "load_dotenv(override=True)\n",
    "api_key = os.getenv('OPENAI_API_KEY')\n",
    "\n",
    "if not api_key:\n",
    "    print(\"No API key was found - please head over to the troubleshooting notebook in this folder to identify & fix!\")\n",
    "elif not api_key.startswith(\"sk-proj-\"):\n",
    "    print(\"An API key was found, but it doesn't start sk-proj-; please check you're using the right key - see troubleshooting notebook\")\n",
    "else:\n",
    "    print(\"API key found and looks good so far!\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97846274",
   "metadata": {},
   "source": [
    "## Do you know what an Endpoint is?\n",
    "\n",
    "If not, please review the Technical Foundations guide in the guides folder\n",
    "\n",
    "And, here is an endpoint that might interest you..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "5af5c188",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'model': 'gpt-5-nano',\n",
       " 'messages': [{'role': 'user', 'content': 'Tell me a fun fact'}]}"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import requests\n",
    "\n",
    "headers = {\"Authorization\": f\"Bearer {api_key}\", \"Content-Type\": \"application/json\"}\n",
    "\n",
    "payload = {\n",
    "    \"model\": \"gpt-5-nano\",\n",
    "    \"messages\": [\n",
    "        {\"role\": \"user\", \"content\": \"Tell me a fun fact\"}]\n",
    "}\n",
    "\n",
    "payload"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2d0ab242",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'id': 'chatcmpl-CTJDqe3UuLrqPiBwX0loYddfVDpWx',\n",
       " 'object': 'chat.completion',\n",
       " 'created': 1761101438,\n",
       " 'model': 'gpt-5-nano-2025-08-07',\n",
       " 'choices': [{'index': 0,\n",
       "   'message': {'role': 'assistant',\n",
       "    'content': 'Fun fact: Honey never spoils. Its low moisture and acidic pH create an environment that bacteria and fungi can’t thrive in. In fact, archaeologists have found pots of honey in ancient Egyptian tombs that are over 3,000 years old and still edible. Want another fun fact?',\n",
       "    'refusal': None,\n",
       "    'annotations': []},\n",
       "   'finish_reason': 'stop'}],\n",
       " 'usage': {'prompt_tokens': 11,\n",
       "  'completion_tokens': 709,\n",
       "  'total_tokens': 720,\n",
       "  'prompt_tokens_details': {'cached_tokens': 0, 'audio_tokens': 0},\n",
       "  'completion_tokens_details': {'reasoning_tokens': 640,\n",
       "   'audio_tokens': 0,\n",
       "   'accepted_prediction_tokens': 0,\n",
       "   'rejected_prediction_tokens': 0}},\n",
       " 'service_tier': 'default',\n",
       " 'system_fingerprint': None}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = requests.post(\n",
    "    \"https://api.openai.com/v1/chat/completions\",\n",
    "    headers=headers,\n",
    "    json=payload\n",
    ")\n",
    "\n",
    "response.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "cb11a9f6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Fun fact: Honey never spoils. Its low moisture and acidic pH create an environment that bacteria and fungi can’t thrive in. In fact, archaeologists have found pots of honey in ancient Egyptian tombs that are over 3,000 years old and still edible. Want another fun fact?'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response.json()[\"choices\"][0][\"message\"][\"content\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cea3026a",
   "metadata": {},
   "source": [
    "# What is the openai package?\n",
    "\n",
    "It's known as a Python Client Library.\n",
    "\n",
    "It's nothing more than a wrapper around making this exact call to the http endpoint.\n",
    "\n",
    "It just allows you to work with nice Python code instead of messing around with janky json objects.\n",
    "\n",
    "But that's it. It's open-source and lightweight. Some people think it contains OpenAI model code - it doesn't!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "490fdf09",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Fun fact: Honey never spoils. Archaeologists have found pots of honey in ancient tombs that are still edible after thousands of years. Its low water content and acidic pH help keep bacteria away.'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create OpenAI client\n",
    "\n",
    "from openai import OpenAI\n",
    "\n",
    "openai = OpenAI()\n",
    "\n",
    "response = openai.chat.completions.create(model=\"gpt-5-nano\", messages=[{\"role\": \"user\", \"content\": \"Tell me a fun fact\"}])\n",
    "\n",
    "response.choices[0].message.content\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7739cda",
   "metadata": {},
   "source": [
    "## And then this great thing happened:\n",
    "\n",
    "OpenAI's Chat Completions API was so popular, that the other model providers created endpoints that are identical.\n",
    "\n",
    "They are known as the \"OpenAI Compatible Endpoints\".\n",
    "\n",
    "For example, google made one here: https://generativelanguage.googleapis.com/v1beta/openai/\n",
    "\n",
    "And OpenAI decided to be kind: they said, hey, you can just use the same client library that we made for GPT. We'll allow you to specify a different endpoint URL and a different key, to use another provider.\n",
    "\n",
    "So you can use:\n",
    "\n",
    "```python\n",
    "gemini = OpenAI(base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\", api_key=\"AIz....\")\n",
    "gemini.chat.completions.create(...)\n",
    "```\n",
    "\n",
    "And to be clear - even though OpenAI is in the code, we're only using this lightweight python client library to call the endpoint - there's no OpenAI model involved here.\n",
    "\n",
    "If you're confused, please review Guide 9 in the Guides folder!\n",
    "\n",
    "And now let's try it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f74293bc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No API key was found - please head over to the troubleshooting notebook in this folder to identify & fix!\n"
     ]
    }
   ],
   "source": [
    "GEMINI_BASE_URL = \"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
    "\n",
    "google_api_key = os.getenv(\"GOOGLE_API_KEY\")\n",
    "\n",
    "if not google_api_key:\n",
    "    print(\"No API key was found - please head over to the troubleshooting notebook in this folder to identify & fix!\")\n",
    "elif not google_api_key.startswith(\"AIz\"):\n",
    "    print(\"An API key was found, but it doesn't start AIz\")\n",
    "else:\n",
    "    print(\"API key found and looks good so far!\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d060f484",
   "metadata": {},
   "outputs": [],
   "source": [
    "gemini = OpenAI(base_url=GEMINI_BASE_URL, api_key=google_api_key)\n",
    "\n",
    "response = gemini.chat.completions.create(model=\"gemini-2.5-pro\", messages=[{\"role\": \"user\", \"content\": \"Tell me a fun fact\"}])\n",
    "\n",
    "response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a5b069be",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Here's one:\\n\\nDid you know that honey never spoils? Archaeologists have found pots of honey in ancient Egyptian tombs that are over 3,000 years old and still perfectly edible! Honey's longevity is due to its unique composition, which includes acid that creates an environment inhospitable to bacteria and microorganisms. Isn't that sweet?\""
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "OLLAMA_BASE_URL = \"http://localhost:11434/v1\"\n",
    "\n",
    "ollama = OpenAI(base_url=OLLAMA_BASE_URL, api_key='ollama')\n",
    "\n",
    "response = ollama.chat.completions.create(model=\"llama3.2\", messages=[{\"role\": \"user\", \"content\": \"Tell me a fun fact\"}])\n",
    "\n",
    "response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65272432",
   "metadata": {},
   "source": [
    "## And Ollama also gives an OpenAI compatible endpoint\n",
    "\n",
    "...and it's on your local machine!\n",
    "\n",
    "If the next cell doesn't print \"Ollama is running\" then please open a terminal and run `ollama serve`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f06280ad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'Ollama is running'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "requests.get(\"http://localhost:11434\").content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6ef3807",
   "metadata": {},
   "source": [
    "### Download llama3.2 from meta\n",
    "\n",
    "Change this to llama3.2:1b if your computer is smaller.\n",
    "\n",
    "Don't use llama3.3 or llama4! They are too big for your computer.."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e633481d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠋ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠙ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠹ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠸ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠼ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠴ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠦ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest ⠧ \u001b[K\u001b[?25h\u001b[?2026l\u001b[?2026h\u001b[?25l\u001b[1Gpulling manifest \u001b[K\n",
      "pulling dde5aa3fc5ff: 100% ▕██████████████████▏ 2.0 GB                         \u001b[K\n",
      "pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         \u001b[K\n",
      "pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         \u001b[K\n",
      "pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         \u001b[K\n",
      "pulling 56bb8bd477a5: 100% ▕██████████████████▏   96 B                         \u001b[K\n",
      "pulling 34bb5ab01051: 100% ▕██████████████████▏  561 B                         \u001b[K\n",
      "verifying sha256 digest \u001b[K\n",
      "writing manifest \u001b[K\n",
      "success \u001b[K\u001b[?25h\u001b[?2026l\n"
     ]
    }
   ],
   "source": [
    "!ollama pull llama3.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "d9419762",
   "metadata": {},
   "outputs": [],
   "source": [
    "OLLAMA_BASE_URL = \"http://localhost:11434/v1\"\n",
    "\n",
    "ollama = OpenAI(base_url=OLLAMA_BASE_URL, api_key='ollama')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e2456cdf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Here's one:\\n\\nDid you know that honey never spoils? Archaeologists have found pots of honey in ancient Egyptian tombs that are over 3,000 years old and still perfectly edible! This is because bees produce antibiotics as part of their hive-building efforts, which helps to preserve the honey. Also, honey doesn't contain water like other foods do, which makes it extremely difficult for bacteria or microorganisms to grow. So, even if you leave a jar of honey out at room temperature for centuries (not recommended, by the way!), it should still be safe to eat!\""
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get a fun fact\n",
    "\n",
    "response = ollama.chat.completions.create(model=\"llama3.2\", messages=[{\"role\": \"user\", \"content\": \"Tell me a fun fact\"}])\n",
    "\n",
    "response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e6cae7f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now let's try deepseek-r1:1.5b - this is DeepSeek \"distilled\" into Qwen from Alibaba Cloud\n",
    "\n",
    "!ollama pull deepseek-r1:1.5b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25002f25",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = ollama.chat.completions.create(model=\"deepseek-r1:1.5b\", messages=[{\"role\": \"user\", \"content\": \"Tell me a fun fact\"}])\n",
    "\n",
    "response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e9fa1fc-eac5-4d1d-9be4-541b3f2b3458",
   "metadata": {},
   "source": [
    "# HOMEWORK EXERCISE ASSIGNMENT\n",
    "\n",
    "Upgrade the day 1 project to summarize a webpage to use an Open Source model running locally via Ollama rather than OpenAI\n",
    "\n",
    "You'll be able to use this technique for all subsequent projects if you'd prefer not to use paid APIs.\n",
    "\n",
    "**Benefits:**\n",
    "1. No API charges - open-source\n",
    "2. Data doesn't leave your box\n",
    "\n",
    "**Disadvantages:**\n",
    "1. Significantly less power than Frontier Model\n",
    "\n",
    "## Recap on installation of Ollama\n",
    "\n",
    "Simply visit [ollama.com](https://ollama.com) and install!\n",
    "\n",
    "Once complete, the ollama server should already be running locally.  \n",
    "If you visit:  \n",
    "[http://localhost:11434/](http://localhost:11434/)\n",
    "\n",
    "You should see the message `Ollama is running`.  \n",
    "\n",
    "If not, bring up a new Terminal (Mac) or Powershell (Windows) and enter `ollama serve`  \n",
    "And in another Terminal (Mac) or Powershell (Windows), enter `ollama pull llama3.2`  \n",
    "Then try [http://localhost:11434/](http://localhost:11434/) again.\n",
    "\n",
    "If Ollama is slow on your machine, try using `llama3.2:1b` as an alternative. Run `ollama pull llama3.2:1b` from a Terminal or Powershell, and change the code from `MODEL = \"llama3.2\"` to `MODEL = \"llama3.2:1b\"`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "fa5edd63",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here's a suggested concise and professional subject line:\n",
      "\n",
      "\"Meeting Rescheduled to 3 PM Tomorrow\"\n",
      "\n",
      "This subject line captures the core intent of the email, which is a meeting rescheduling, in just a few words while maintaining clarity and tone alignment.\n"
     ]
    }
   ],
   "source": [
    "# Step 1: Create your prompts\n",
    "\n",
    "system_prompt = \"\"\"\n",
    "You are embedded in an email tool that analyzes an email’s body text and suggests a concise, relevant, and professional subject line.\n",
    "The subject line should capture the core intent of the email in 5–10 words\n",
    "Maintain clarity and tone alignment:\n",
    "Formal/business emails → neutral and polished\n",
    "Casual/internal updates → conversational and brief\n",
    "Urgent matters → include urgency subtly (e.g., “Action Needed”, “Follow-Up Required”)\n",
    "Avoid unnecessary punctuation, emojis, or filler words.\n",
    "Do not include explanations unless explicitly requested.\n",
    "\"\"\"\n",
    "user_prompt = \"\"\"\n",
    "    Suggest a short subject line for the following email:\n",
    "    Hi team, please note that tomorrow’s meeting will be moved to 3 PM instead of 2 PM to accommodate the new client call.\n",
    "\"\"\"\n",
    "\n",
    "# Step 2: Make the messages list\n",
    "\n",
    "messages = [\n",
    "    {\"role\": \"system\", \"content\": system_prompt},\n",
    "    {\"role\": \"user\", \"content\": user_prompt}\n",
    "]\n",
    "\n",
    "# Step 3: Call OpenAI\n",
    "response = ollama.chat.completions.create(\n",
    "    model=\"llama3.2\",\n",
    "    messages=messages\n",
    ")\n",
    "\n",
    "# Step 4: print the result\n",
    "print(response.choices[0].message.content)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}