436 lines
14 KiB
Plaintext
436 lines
14 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d15d8294-3328-4e07-ad16-8a03e9bbfdb9",
|
|
"metadata": {
|
|
"jp-MarkdownHeadingCollapsed": true
|
|
},
|
|
"source": [
|
|
"# Welcome to your first assignment!\n",
|
|
"\n",
|
|
"Instructions are below. Please give this a try, and look in the solutions folder if you get stuck (or feel free to ask me!)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ada885d9-4d42-4d9b-97f0-74fbbbfe93a9",
|
|
"metadata": {
|
|
"jupyter": {
|
|
"source_hidden": true
|
|
}
|
|
},
|
|
"source": [
|
|
"<table style=\"margin: 0; text-align: left;\">\n",
|
|
" <tr>\n",
|
|
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
|
|
" <img src=\"../resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
|
|
" </td>\n",
|
|
" <td>\n",
|
|
" <h2 style=\"color:#f71;\">Just before we get to the assignment --</h2>\n",
|
|
" <span style=\"color:#f71;\">I thought I'd take a second to point you at this page of useful resources for the course. This includes links to all the slides.<br/>\n",
|
|
" <a href=\"https://edwarddonner.com/2024/11/13/llm-engineering-resources/\">https://edwarddonner.com/2024/11/13/llm-engineering-resources/</a><br/>\n",
|
|
" Please keep this bookmarked, and I'll continue to add more useful links there over time.\n",
|
|
" </span>\n",
|
|
" </td>\n",
|
|
" </tr>\n",
|
|
"</table>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6e9fa1fc-eac5-4d1d-9be4-541b3f2b3458",
|
|
"metadata": {},
|
|
"source": [
|
|
"# HOMEWORK EXERCISE ASSIGNMENT\n",
|
|
"\n",
|
|
"Upgrade the day 1 project to summarize a webpage to use an Open Source model running locally via Ollama rather than OpenAI\n",
|
|
"\n",
|
|
"You'll be able to use this technique for all subsequent projects if you'd prefer not to use paid APIs.\n",
|
|
"\n",
|
|
"**Benefits:**\n",
|
|
"1. No API charges - open-source\n",
|
|
"2. Data doesn't leave your box\n",
|
|
"\n",
|
|
"**Disadvantages:**\n",
|
|
"1. Significantly less power than Frontier Model\n",
|
|
"\n",
|
|
"## Recap on installation of Ollama\n",
|
|
"\n",
|
|
"Simply visit [ollama.com](https://ollama.com) and install!\n",
|
|
"\n",
|
|
"Once complete, the ollama server should already be running locally. \n",
|
|
"If you visit: \n",
|
|
"[http://localhost:11434/](http://localhost:11434/)\n",
|
|
"\n",
|
|
"You should see the message `Ollama is running`. \n",
|
|
"\n",
|
|
"If not, bring up a new Terminal (Mac) or Powershell (Windows) and enter `ollama serve` \n",
|
|
"And in another Terminal (Mac) or Powershell (Windows), enter `ollama pull llama3.2` \n",
|
|
"Then try [http://localhost:11434/](http://localhost:11434/) again.\n",
|
|
"\n",
|
|
"If Ollama is slow on your machine, try using `llama3.2:1b` as an alternative. Run `ollama pull llama3.2:1b` from a Terminal or Powershell, and change the code below from `MODEL = \"llama3.2\"` to `MODEL = \"llama3.2:1b\"`"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "4e2a9393-7767-488e-a8bf-27c12dca35bd",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# imports\n",
|
|
"\n",
|
|
"import requests\n",
|
|
"from bs4 import BeautifulSoup\n",
|
|
"from IPython.display import Markdown, display"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "29ddd15d-a3c5-4f4e-a678-873f56162724",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Constants\n",
|
|
"\n",
|
|
"OLLAMA_API = \"http://localhost:11434/api/chat\"\n",
|
|
"HEADERS = {\"Content-Type\": \"application/json\"}\n",
|
|
"MODEL = \"llama3.2\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "dac0a679-599c-441f-9bf2-ddc73d35b940",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Create a messages list using the same format that we used for OpenAI\n",
|
|
"\n",
|
|
"messages = [\n",
|
|
" {\"role\": \"user\", \"content\": \"Describe some of the business applications of Generative AI\"}\n",
|
|
"]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "7bb9c624-14f0-4945-a719-8ddb64f66f47",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"payload = {\n",
|
|
" \"model\": MODEL,\n",
|
|
" \"messages\": messages,\n",
|
|
" \"stream\": False\n",
|
|
" }"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "479ff514-e8bd-4985-a572-2ea28bb4fa40",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Let's just make sure the model is loaded\n",
|
|
"\n",
|
|
"!ollama pull llama3.2"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "42b9f644-522d-4e05-a691-56e7658c0ea9",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# If this doesn't work for any reason, try the 2 versions in the following cells\n",
|
|
"# And double check the instructions in the 'Recap on installation of Ollama' at the top of this lab\n",
|
|
"# And if none of that works - contact me!\n",
|
|
"\n",
|
|
"response = requests.post(OLLAMA_API, json=payload, headers=HEADERS)\n",
|
|
"print(response.json()['message']['content'])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6a021f13-d6a1-4b96-8e18-4eae49d876fe",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Introducing the ollama package\n",
|
|
"\n",
|
|
"And now we'll do the same thing, but using the elegant ollama python package instead of a direct HTTP call.\n",
|
|
"\n",
|
|
"Under the hood, it's making the same call as above to the ollama server running at localhost:11434"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "7745b9c4-57dc-4867-9180-61fa5db55eb8",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import ollama\n",
|
|
"\n",
|
|
"response = ollama.chat(model=MODEL, messages=messages)\n",
|
|
"print(response['message']['content'])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a4704e10-f5fb-4c15-a935-f046c06fb13d",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Alternative approach - using OpenAI python library to connect to Ollama"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "23057e00-b6fc-4678-93a9-6b31cb704bff",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# There's actually an alternative approach that some people might prefer\n",
|
|
"# You can use the OpenAI client python library to call Ollama:\n",
|
|
"\n",
|
|
"from openai import OpenAI\n",
|
|
"ollama_via_openai = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n",
|
|
"\n",
|
|
"response = ollama_via_openai.chat.completions.create(\n",
|
|
" model=MODEL,\n",
|
|
" messages=messages\n",
|
|
")\n",
|
|
"\n",
|
|
"print(response.choices[0].message.content)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "bc7d1de3-e2ac-46ff-a302-3b4ba38c4c90",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Also trying the amazing reasoning model DeepSeek\n",
|
|
"\n",
|
|
"Here we use the version of DeepSeek-reasoner that's been distilled to 1.5B. \n",
|
|
"This is actually a 1.5B variant of Qwen that has been fine-tuned using synethic data generated by Deepseek R1.\n",
|
|
"\n",
|
|
"Other sizes of DeepSeek are [here](https://ollama.com/library/deepseek-r1) all the way up to the full 671B parameter version, which would use up 404GB of your drive and is far too large for most!"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "cf9eb44e-fe5b-47aa-b719-0bb63669ab3d",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!ollama pull deepseek-r1:1.5b"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "1d3d554b-e00d-4c08-9300-45e073950a76",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# This may take a few minutes to run! You should then see a fascinating \"thinking\" trace inside <think> tags, followed by some decent definitions\n",
|
|
"\n",
|
|
"response = ollama_via_openai.chat.completions.create(\n",
|
|
" model=\"deepseek-r1:1.5b\",\n",
|
|
" messages=[{\"role\": \"user\", \"content\": \"Please give definitions of some core concepts behind LLMs: a neural network, attention and the transformer\"}]\n",
|
|
")\n",
|
|
"\n",
|
|
"print(response.choices[0].message.content)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1622d9bb-5c68-4d4e-9ca4-b492c751f898",
|
|
"metadata": {},
|
|
"source": [
|
|
"# NOW the exercise for you\n",
|
|
"\n",
|
|
"Take the code from day1 and incorporate it here, to build a website summarizer that uses Llama 3.2 running locally instead of OpenAI; use either of the above approaches."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "6de38216-6d1c-48c4-877b-86d403f4e0f8",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"import requests\n",
|
|
"from dotenv import load_dotenv\n",
|
|
"from bs4 import BeautifulSoup\n",
|
|
"from IPython.display import Markdown, display\n",
|
|
"from openai import OpenAI\n",
|
|
"\n",
|
|
"load_dotenv(override=True)\n",
|
|
"api_key = os.getenv('OPENAI_API_KEY')\n",
|
|
"\n",
|
|
"HEADERS = {\n",
|
|
" \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36\",\n",
|
|
" \"Content-Type\": \"application/json\"\n",
|
|
"}\n",
|
|
"\n",
|
|
"OLLAMA_API = \"http://localhost:11434/api/chat\"\n",
|
|
"\n",
|
|
"MODEL = \"llama3.2\"\n",
|
|
"\n",
|
|
"system_prompt = \"Sei un assistente e analizzi il contenuto di un sito web \\\n",
|
|
"produci un breve sommario, ignora il testo o gli elementi relativi alla navigazione. \\\n",
|
|
"Rispondi markdown.\"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"id": "6f343c27-628c-4c54-9a5b-842e6ad5d176",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"class Website:\n",
|
|
"\n",
|
|
" def __init__(self, url):\n",
|
|
" \"\"\"\n",
|
|
" Create this Website object from the given url using the BeautifulSoup library\n",
|
|
" \"\"\"\n",
|
|
" self.url = url\n",
|
|
" response = requests.get(url, headers=HEADERS)\n",
|
|
" soup = BeautifulSoup(response.content, 'html.parser')\n",
|
|
" self.title = soup.title.string if soup.title else \"No title found\"\n",
|
|
" for irrelevant in soup.body([\"script\", \"style\", \"img\", \"input\"]):\n",
|
|
" irrelevant.decompose()\n",
|
|
" self.text = soup.body.get_text(separator=\"\\n\", strip=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"id": "bf6245ca-2d53-4fd8-a19c-0e6d052031fd",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def user_prompt_for(website):\n",
|
|
" user_prompt = f\"Stai cercando un sito dal titolo: {website.title}\"\n",
|
|
" user_prompt += \"\\nI contenuti di questo sito web sono i seguenti: \\\n",
|
|
"Per favore, fornisci un breve riassunto di questo sito web in markdown. \\\n",
|
|
"Se include notizie o annunci, riassumili anch'essi. \\n\\n\"\n",
|
|
" user_prompt += website.text\n",
|
|
" return user_prompt"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"id": "dec0636f-9efc-4f91-8861-3141276a9a6e",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def messages_for(website):\n",
|
|
" return [\n",
|
|
" {\"role\": \"system\", \"content\": system_prompt},\n",
|
|
" {\"role\": \"user\", \"content\": user_prompt_for(website)}\n",
|
|
" ]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"id": "f894b232-1ea1-4bd9-bf44-d7b1571f7913",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def summarize(url):\n",
|
|
" ollama_via_openai = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n",
|
|
" \n",
|
|
" website = Website(url)\n",
|
|
" response = ollama_via_openai.chat.completions.create(\n",
|
|
" model=MODEL,\n",
|
|
" messages=messages_for(website)\n",
|
|
" )\n",
|
|
" return response.choices[0].message.content\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"id": "d868d778-13b5-4934-acf5-dcb919a27d59",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def display_summary(url):\n",
|
|
" summary = summarize(url)\n",
|
|
" display(Markdown(summary))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"id": "0a0d9b79-de3c-4f77-9254-f02cf4d6217a",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/markdown": [
|
|
"**Sommario del sito WP Pisa - Il sito del Meetup WordPress di Pisa**\n",
|
|
"\n",
|
|
"Il sito web WP Pisa è il punto di riferimento per gli appassionati di WordPress a Pisa. Gli organizzatori offrono meetup mensili gratuiti per discutere conoscenze, esperienze e progetti correlati al mondo di WordPress.\n",
|
|
"\n",
|
|
"**Eventi e Annunci**\n",
|
|
"\n",
|
|
"* **WordCamp Pisa 2025**: Aperta la call for organizer\n",
|
|
"* **Il Tuo Sito Ovunque in Pochi Minuti**: Un incontro con Docker e sviluppatori WordPress - Partecipa!\n",
|
|
"* **Core Days Roma: Le novità sul core di WordPress per i dev**\n",
|
|
"* **NO MORE THUMBNAILS!**\n",
|
|
"\n",
|
|
"**Informazioni Generali**\n",
|
|
"\n",
|
|
"* Il meetuu è aperto a tutti, indipendentemente dal livello di competenza in WordPress\n",
|
|
"* Tutti gli eventi sono gratuiti e organizzati con la supervisione di WordPress Foundation tramite la piattaforma Meetup.com\n",
|
|
"* La comunità WP Pisa contiene 150+ iscritti"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Markdown object>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"display_summary(\"https://wppisa.it/\")"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.11"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|