{ "cells": [ { "cell_type": "markdown", "id": "4a6ab9a2-28a2-445d-8512-a0dc8d1b54e9", "metadata": {}, "source": [ "# Code Generator\n", "\n", "The requirement: use a Frontier model to generate high performance C++ code from Python code\n" ] }, { "cell_type": "markdown", "id": "d5ccb926-7b49-44a4-99ab-8ef20b5778c0", "metadata": {}, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \n", "

Reminder: fetch latest code

\n", " I'm continually improving these labs, adding more examples and exercises.\n", " At the start of each week, it's worth checking you have the latest code.
\n", " First do a git pull and merge your changes as needed. Any problems? Try asking ChatGPT to clarify how to merge - or contact me!

\n", " After you've pulled the code, from the llm_engineering directory, in a Cursor Terminal, run:
\n", " uv sync
\n", " \n", "
" ] }, { "cell_type": "markdown", "id": "d90e04a2-5b8a-4fd5-9db8-27c02f033313", "metadata": {}, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \n", "

Important Note

\n", " \n", " In this lab, I use high end models GPT 5, Claude 4.5 Sonnet, Gemini 2.5 Pro, Grok 4, which are the slightly higher priced models. The costs are still low, but if you'd prefer to keep costs ultra low, please pick lower cost models like gpt-5-nano.\n", " \n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "e610bf56-a46e-4aff-8de1-ab49d62b1ad3", "metadata": {}, "outputs": [], "source": [ "# imports\n", "\n", "import os\n", "from dotenv import load_dotenv\n", "from openai import OpenAI\n", "import subprocess\n", "from IPython.display import Markdown, display" ] }, { "cell_type": "code", "execution_count": null, "id": "4f672e1c-87e9-4865-b760-370fa605e614", "metadata": {}, "outputs": [], "source": [ "load_dotenv(override=True)\n", "openai_api_key = os.getenv('OPENAI_API_KEY')\n", "anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')\n", "google_api_key = os.getenv('GOOGLE_API_KEY')\n", "grok_api_key = os.getenv('GROK_API_KEY')\n", "\n", "if openai_api_key:\n", " print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n", "else:\n", " print(\"OpenAI API Key not set\")\n", " \n", "if anthropic_api_key:\n", " print(f\"Anthropic API Key exists and begins {anthropic_api_key[:7]}\")\n", "else:\n", " print(\"Anthropic API Key not set (and this is optional)\")\n", "\n", "if google_api_key:\n", " print(f\"Google API Key exists and begins {google_api_key[:2]}\")\n", "else:\n", " print(\"Google API Key not set (and this is optional)\")\n", "\n", "if grok_api_key:\n", " print(f\"Grok API Key exists and begins {grok_api_key[:4]}\")\n", "else:\n", " print(\"Grok API Key not set (and this is optional)\")" ] }, { "cell_type": "code", "execution_count": null, "id": "59863df1", "metadata": {}, "outputs": [], "source": [ "# Connect to client libraries\n", "\n", "openai = OpenAI()\n", "\n", "anthropic_url = \"https://api.anthropic.com/v1/\"\n", "gemini_url = \"https://generativelanguage.googleapis.com/v1beta/openai/\"\n", "grok_url = \"https://api.x.ai/v1\"\n", "\n", "anthropic = OpenAI(api_key=anthropic_api_key, base_url=anthropic_url)\n", "gemini = OpenAI(api_key=google_api_key, base_url=gemini_url)\n", "grok = OpenAI(api_key=grok_api_key, base_url=grok_url)" ] }, { "cell_type": "code", "execution_count": null, "id": "8aa149ed-9298-4d69-8fe2-8f5de0f667da", "metadata": {}, "outputs": [], "source": [ "OPENAI_MODEL = \"gpt-5\"\n", "CLAUDE_MODEL = \"claude-sonnet-4-5-20250929\"\n", "GROK_MODEL = \"grok-4\"\n", "GEMINI_MODEL = \"gemini-2.5-pro\"\n", "\n", "# Want to keep costs ultra-low? Uncomment these lines:\n", "\n", "# OPENAI_MODEL = \"gpt-5-nano\"\n", "# CLAUDE_MODEL = \"claude-3-5-haiku-latest\"\n", "# GROK_MODEL = \"grok-4-fast-non-reasoning\"\n", "# GEMINI_MODEL = \"gemini-2.5-flash-lite\"" ] }, { "cell_type": "markdown", "id": "7eab38a7", "metadata": {}, "source": [ "## PLEASE NOTE:\n", "\n", "We will be writing a solution to convert Python into efficient, optimized C++ code for your machine, which can be compiled to native machine code and executed.\n", "\n", "It is not necessary for you to execute the code yourself - that's not the point of the exercise!\n", "\n", "But if you would like to (because it's satisfying!) then I'm including the steps here. Very optional!\n", "\n", "As an alternative, I'll also show you a website where you can run the C++ code." ] }, { "cell_type": "code", "execution_count": null, "id": "8a2fbb68", "metadata": {}, "outputs": [], "source": [ "from system_info import retrieve_system_info\n", "\n", "system_info = retrieve_system_info()\n", "system_info" ] }, { "cell_type": "code", "execution_count": null, "id": "c6d29a5f", "metadata": {}, "outputs": [], "source": [ "message = f\"\"\"\n", "Here is a report of the system information for my computer.\n", "I want to run a C++ compiler to compile a single C++ file called main.cpp and then execute it in the simplest way possible.\n", "Please reply with whether I need to install any C++ compiler to do this. If so, please provide the simplest step by step instructions to do so.\n", "\n", "If I'm already set up to compile C++ code, then I'd like to run something like this in Python to compile and execute the code:\n", "```python\n", "compile_command = # something here - to achieve the fastest possible runtime performance\n", "compile_result = subprocess.run(compile_command, check=True, text=True, capture_output=True)\n", "run_command = # something here\n", "run_result = subprocess.run(run_command, check=True, text=True, capture_output=True)\n", "return run_result.stdout\n", "```\n", "Please tell me exactly what I should use for the compile_command and run_command.\n", "\n", "System information:\n", "{system_info}\n", "\"\"\"\n", "\n", "response = openai.chat.completions.create(model=OPENAI_MODEL, messages=[{\"role\": \"user\", \"content\": message}])\n", "display(Markdown(response.choices[0].message.content))\n", " " ] }, { "cell_type": "markdown", "id": "81e92c12", "metadata": {}, "source": [ "## If you need to install something\n", "\n", "If you would like to, please follow GPTs instructions! Then rerun the analysis afterwards (you might need to Restart the notebook) to confirm you're set.\n", "\n", "You should now be equipped with the command to compile the code, and the command to run it!\n", "\n", "Enter that in the cell below:" ] }, { "cell_type": "code", "execution_count": null, "id": "d734a634", "metadata": {}, "outputs": [], "source": [ "compile_command = [\"clang++\", \"-std=c++17\", \"-Ofast\", \"-mcpu=native\", \"-flto=thin\", \"-fvisibility=hidden\", \"-DNDEBUG\", \"main.cpp\", \"-o\", \"main\"]\n", "run_command = [\"./main\"]" ] }, { "cell_type": "markdown", "id": "f0b0a437", "metadata": {}, "source": [ "## And now, on with the main task" ] }, { "cell_type": "code", "execution_count": null, "id": "6896636f-923e-4a2c-9d6c-fac07828a201", "metadata": {}, "outputs": [], "source": [ "system_prompt = \"\"\"\n", "Your task is to convert Python code into high performance C++ code.\n", "Respond only with C++ code. Do not provide any explanation other than occasional comments.\n", "The C++ response needs to produce an identical output in the fastest possible time.\n", "\"\"\"\n", "\n", "def user_prompt_for(python):\n", " return f\"\"\"\n", "Port this Python code to C++ with the fastest possible implementation that produces identical output in the least time.\n", "The system information is:\n", "{system_info}\n", "Your response will be written to a file called main.cpp and then compiled and executed; the compilation command is:\n", "{compile_command}\n", "Respond only with C++ code.\n", "Python code to port:\n", "\n", "```python\n", "{python}\n", "```\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": null, "id": "8e7b3546-57aa-4c29-bc5d-f211970d04eb", "metadata": {}, "outputs": [], "source": [ "def messages_for(python):\n", " return [\n", " {\"role\": \"system\", \"content\": system_prompt},\n", " {\"role\": \"user\", \"content\": user_prompt_for(python)}\n", " ]\n", " " ] }, { "cell_type": "code", "execution_count": null, "id": "c6190659-f54c-4951-bef4-4960f8e51cc4", "metadata": {}, "outputs": [], "source": [ "def write_output(cpp):\n", " with open(\"main.cpp\", \"w\", encoding=\"utf-8\") as f:\n", " f.write(cpp)" ] }, { "cell_type": "code", "execution_count": null, "id": "e7d2fea8-74c6-4421-8f1e-0e76d5b201b9", "metadata": {}, "outputs": [], "source": [ "def port(client, model, python):\n", " reasoning_effort = \"high\" if 'gpt' in model else None\n", " response = client.chat.completions.create(model=model, messages=messages_for(python), reasoning_effort=reasoning_effort)\n", " reply = response.choices[0].message.content\n", " reply = reply.replace('```cpp','').replace('```','')\n", " write_output(reply)" ] }, { "cell_type": "code", "execution_count": null, "id": "a1cbb778-fa57-43de-b04b-ed523f396c38", "metadata": {}, "outputs": [], "source": [ "pi = \"\"\"\n", "import time\n", "\n", "def calculate(iterations, param1, param2):\n", " result = 1.0\n", " for i in range(1, iterations+1):\n", " j = i * param1 - param2\n", " result -= (1/j)\n", " j = i * param1 + param2\n", " result += (1/j)\n", " return result\n", "\n", "start_time = time.time()\n", "result = calculate(200_000_000, 4, 1) * 4\n", "end_time = time.time()\n", "\n", "print(f\"Result: {result:.12f}\")\n", "print(f\"Execution Time: {(end_time - start_time):.6f} seconds\")\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": null, "id": "7fe1cd4b-d2c5-4303-afed-2115a3fef200", "metadata": {}, "outputs": [], "source": [ "def run_python(code):\n", " globals = {\"__builtins__\": __builtins__}\n", " exec(code, globals)" ] }, { "cell_type": "code", "execution_count": null, "id": "7faa90da", "metadata": {}, "outputs": [], "source": [ "run_python(pi)" ] }, { "cell_type": "code", "execution_count": null, "id": "105db6f9-343c-491d-8e44-3a5328b81719", "metadata": {}, "outputs": [], "source": [ "port(openai, OPENAI_MODEL, pi)" ] }, { "cell_type": "markdown", "id": "bf8f8018-f64d-425c-a0e1-d7862aa9592d", "metadata": {}, "source": [ "# Compiling C++ and executing\n", "\n", "This next cell contains the command to compile a C++ file based on the instructions from GPT.\n", "\n", "Again, it's not crucial to do this step if you don't wish to!\n", "\n", "OR alternatively: student Sandeep K.G. points out that you can run Python and C++ code online to test it out that way. Thank you Sandeep! \n", "> Not an exact comparison but you can still get the idea of performance difference. \n", "> For example here: https://www.programiz.com/cpp-programming/online-compiler/" ] }, { "cell_type": "code", "execution_count": null, "id": "4194e40c-04ab-4940-9d64-b4ad37c5bb40", "metadata": {}, "outputs": [], "source": [ "# Use the commands from GPT 5\n", "\n", "def compile_and_run():\n", " subprocess.run(compile_command, check=True, text=True, capture_output=True)\n", " print(subprocess.run(run_command, check=True, text=True, capture_output=True).stdout)\n", " print(subprocess.run(run_command, check=True, text=True, capture_output=True).stdout)\n", " print(subprocess.run(run_command, check=True, text=True, capture_output=True).stdout)" ] }, { "cell_type": "code", "execution_count": null, "id": "22f8f43a", "metadata": {}, "outputs": [], "source": [ "compile_and_run()" ] }, { "cell_type": "code", "execution_count": null, "id": "faaa39de", "metadata": {}, "outputs": [], "source": [ "19.178207/0.082168" ] }, { "cell_type": "markdown", "id": "4f3b8ef9", "metadata": {}, "source": [ "## OK let's try the other contenders!" ] }, { "cell_type": "code", "execution_count": null, "id": "983a11fe-e24d-4c65-8269-9802c5ef3ae6", "metadata": {}, "outputs": [], "source": [ "port(anthropic, CLAUDE_MODEL, pi)\n", "compile_and_run()" ] }, { "cell_type": "code", "execution_count": null, "id": "138f63c8", "metadata": {}, "outputs": [], "source": [ "port(grok, GROK_MODEL, pi)\n", "compile_and_run()" ] }, { "cell_type": "code", "execution_count": null, "id": "0a0243c5", "metadata": {}, "outputs": [], "source": [ "port(gemini, GEMINI_MODEL, pi)\n", "compile_and_run()\n" ] }, { "cell_type": "markdown", "id": "0689e200", "metadata": {}, "source": [ "\n" ] }, { "cell_type": "code", "execution_count": null, "id": "a6ffb0bb", "metadata": {}, "outputs": [], "source": [ "print(f\"\"\"\n", "In Ed's experiments, the performance speedups were:\n", "\n", "4th place: Claude Sonnet 4.5: {19.178207/0.104241:.0f}X speedup\n", "3rd place: GPT-5: {19.178207/0.082168:.0f}X speedup\n", "2nd place: Grok 4: {19.178207/0.018092:.0f}X speedup\n", "1st place: Gemini 2.5 Pro: {19.178207/0.013314:.0f}X speedup\n", "\"\"\")" ] }, { "cell_type": "code", "execution_count": null, "id": "8d58753b", "metadata": {}, "outputs": [], "source": [ "\n" ] }, { "cell_type": "code", "execution_count": null, "id": "7202e513", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.9" } }, "nbformat": 4, "nbformat_minor": 5 }