{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "31cd209d-39a4-4e0b-b34a-ced26db5b7d3",
   "metadata": {},
   "source": "# i.MX 9x with eIQ Neutron NPU"
  },
  {
   "cell_type": "markdown",
   "id": "38219ac5",
   "metadata": {},
   "source": [
    "This guide walks you through the process of profiling a TensorFlow Lite model with **eIQ Neutron NP** using the **eIQ AI Toolkit** simulated profiling feature. It covers the essential steps required to register model, and highlights useful endpoints for profiling tasks.\n",
    "\n",
    "Simulated profiling does not require access to a physical board with **eIQ Neutron NPU**. Instead, it provides an estimated performance profile based on representative hardware characteristics.\n",
    "\n",
    "What you will learn:\n",
    "- How to upload and register a TF Lite model for profiling\n",
    "- Key **eIQ AI Toolkit** API endpoints relevant to model profiling\n",
    "\n",
    "*Note: **eIQ Neutron NPU** profiling supports only TF Lite models. Other model formats require conversion to TF Lite.*\n",
    "\n",
    "*Note: This guide was developed and run using Python 3.11.*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8ecad83",
   "metadata": {},
   "source": [
    "This guide requires the **eIQ AI Toolkit** backend to be running.\n",
    "If you haven't set it up yet, please refer to the following tutorial:\n",
    "[eIQ AI Toolkit setup & launch](../../tools/aiToolkit/installRun.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "id": "77e6ac86",
   "metadata": {},
   "source": [
    "import requests\n",
    "from pathlib import Path\n",
    "\n",
    "# Set your eIQ AI Toolkit url:\n",
    "AI_TOOLKIT_BACKEND_URL = \"http://localhost:8000\""
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "12237d11",
   "metadata": {},
   "source": [
    "## Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6eb9300b",
   "metadata": {},
   "source": [
    "If you already have a trained model ready, simply update the path to point to its location.\n",
    "If you don’t have a trained model yet, set the path to a location where the model should be saved.\n",
    "(See the following sections for instructions on how to download a sample model.)"
   ]
  },
  {
   "cell_type": "code",
   "id": "0e663efa",
   "metadata": {},
   "source": [
    "model_path = Path(\"your_model_path.tflite\")"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "1fe5e7c1",
   "metadata": {},
   "source": [
    "Use the following command or script to download the example model:\n",
    "\n",
    "*Note: Skip this step if you already have your own model.*"
   ]
  },
  {
   "cell_type": "code",
   "id": "c9dafc76",
   "metadata": {},
   "source": [
    "example_model_url = \"https://eiq.nxp.com/training-materials/_misc/models/mobilenet_v3-small_224_1.0_uint8.tflite\"\n",
    "\n",
    "with open(model_path, \"wb\") as f:\n",
    "    response = requests.get(\n",
    "        url=example_model_url\n",
    "    )\n",
    "    f.write(response.content)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "05c21bfc",
   "metadata": {},
   "source": "### Upload model to eIQ AI Toolkit"
  },
  {
   "cell_type": "markdown",
   "id": "8468b80b",
   "metadata": {},
   "source": [
    "Uploading a model to **eIQ AI Toolkit** consists of two steps:\n",
    "\n",
    "1. Upload Metadata\n",
    "This includes information such as the model name, format (e.g., TF Lite), input/output shapes, and other relevant attributes.\n",
    "\n",
    "2. Upload Model File\n",
    "After the metadata is registered, the actual model file (e.g., .tflite) is uploaded to the platform."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfbe6606",
   "metadata": {},
   "source": [
    "Submit the metadata:"
   ]
  },
  {
   "cell_type": "code",
   "id": "4a43b0c0",
   "metadata": {},
   "source": [
    "response = requests.post(\n",
    "    url=f\"{AI_TOOLKIT_BACKEND_URL}/models\",\n",
    "    params={\n",
    "        \"model_name\": \"your_custom_model_name\",\n",
    "    },\n",
    "    json={\n",
    "        \"model_type\": \"tflite\"\n",
    "    }\n",
    ")\n",
    "\n",
    "data = response.json()\n",
    "print(data)\n",
    "model_uuid = data[\"data\"][\"model\"][\"uuid\"] # Assigned model identifier"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "5f641f4b",
   "metadata": {},
   "source": [
    "Upload the model file:"
   ]
  },
  {
   "cell_type": "code",
   "id": "270b407b",
   "metadata": {},
   "source": [
    "with open(model_path, \"rb\") as model_file:\n",
    "    response = requests.post(\n",
    "        url=f\"{AI_TOOLKIT_BACKEND_URL}/models/{model_uuid}\", # Model identifier is part of the request URL\n",
    "        files={\n",
    "            \"model_file\": model_file,\n",
    "        }\n",
    "    )\n",
    "\n",
    "print(response.json())"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "da9a1423",
   "metadata": {},
   "source": [
    "After uploading the model metadata and file, you can verify the model's registration and readiness status using the following endpoint:"
   ]
  },
  {
   "cell_type": "code",
   "id": "d3bc198f",
   "metadata": {},
   "source": [
    "response = requests.get(f\"{AI_TOOLKIT_BACKEND_URL}/models/7409abf3-9b6a-4351-b9d7-02480cc1cd3e\")\n",
    "data = response.json() \n",
    "print(f'Model status: {data[\"data\"][\"model\"][\"status\"]}')\n",
    "print(f'Model status description: {data[\"data\"][\"model\"][\"status_description\"]}')"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "5c120cd6",
   "metadata": {},
   "source": [
    "The model can be used for profiling once its status is reported as `ready`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ab77c85",
   "metadata": {},
   "source": [
    "## Profiling"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a82bbdd4",
   "metadata": {},
   "source": [
    "To start simulated profiling, invoke the endpoint `/profiling/run_simulated`.\n",
    "\n",
    "You will need the following parameters:\n",
    "- Model identifier – the unique ID of the model you uploaded\n",
    "- Engine – the engine running the simulation. In case of **eIQ Neutron NPU** the engine is `neutron`\n",
    "- Parameters - in order to profile on **eIQ Neutron NPU**, simulation target must be specified\n",
    "- Run name (optional) – a custom name for the profiling session, useful for tracking and organizing results\n",
    "\n",
    "The `/profiling/supported_simulated_types` endpoint returns information about the simulated profiling parameters:"
   ]
  },
  {
   "cell_type": "code",
   "id": "d8510de9",
   "metadata": {},
   "source": [
    "response = requests.get(\"http://localhost:8000/profiling/supported_simulated_types\")\n",
    "\n",
    "data = response.json()\n",
    "print(data)\n",
    "\n",
    "types = data[\"data\"][\"types\"]\n",
    "supported_imx95 = [x for x in types if x[\"engine\"] == \"neutron\"]\n",
    "print(f\"Neutron: {supported_imx95[0]}\")"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "b3cda808",
   "metadata": {},
   "source": [
    "Set the parameters used for the profiling request:"
   ]
  },
  {
   "cell_type": "code",
   "id": "cdbc9cad",
   "metadata": {},
   "source": [
    "profiling_engine = \"neutron\" # The engine simulating eIQ Neturon NPU\n",
    "profiling_target = \"imxrt700\" # See available target options above\n",
    "profiling_run_name = \"example_imx93_profiling_tflite_model\" \n",
    "\n",
    "print(f\"Model identifier: {model_uuid}\")\n",
    "print(f\"Engine: {profiling_engine}\")\n",
    "print(f\"Target: {profiling_engine}\")\n",
    "print(f\"Custom profiling name run: {profiling_run_name}\")"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "716cfcbf",
   "metadata": {},
   "source": [
    "Request the profiling:"
   ]
  },
  {
   "cell_type": "code",
   "id": "488c2ce9",
   "metadata": {},
   "source": [
    "response = requests.post(\n",
    "    url=f\"{AI_TOOLKIT_BACKEND_URL}/profiling/run_simulated\",\n",
    "    json={\n",
    "        \"model_uuid\": model_uuid,\n",
    "        \"engine\": profiling_engine,\n",
    "        \"parameters\": {\n",
    "            \"target\": profiling_target\n",
    "        },\n",
    "        \"name\": profiling_run_name,\n",
    "    }\n",
    ")\n",
    "\n",
    "data = response.json()\n",
    "print(data)\n",
    "profiling_uuid = data[\"data\"][\"profiling\"][\"uuid\"] # Assigned identifier of the requested profiling run"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "263a8740",
   "metadata": {},
   "source": [
    "After initiating the profiling job, you can monitor its progress using the following API call:"
   ]
  },
  {
   "cell_type": "code",
   "id": "d92d97e9",
   "metadata": {},
   "source": [
    "response = requests.get(\n",
    "    url=f\"{AI_TOOLKIT_BACKEND_URL}/profiling/{profiling_uuid}\" # Profiling run identifier is part of the request URL\n",
    ")\n",
    "\n",
    "data = response.json()\n",
    "print(data)\n",
    "print(f'Profiling status: {data[\"data\"][\"profiling\"][\"status\"]}')\n",
    "print(f'Profiling status description: {data[\"data\"][\"profiling\"][\"status_description\"]}')"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "734724c2",
   "metadata": {},
   "source": [
    "Once the profiling status is marked as `success` you can proceed to analyze the results. If the status is still `in_progress`, re-run the status check (cell above) until the profiling completes."
   ]
  },
  {
   "cell_type": "code",
   "id": "7ed76028",
   "metadata": {},
   "source": [
    "profiling_data = data[\"data\"][\"profiling\"]\n",
    "print(profiling_data)"
   ],
   "outputs": [],
   "execution_count": null
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "sphinx-env (3.11.11)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}