{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "31cd209d-39a4-4e0b-b34a-ced26db5b7d3",
   "metadata": {},
   "source": "# Converting for eIQ Neutron NPU"
  },
  {
   "cell_type": "markdown",
   "id": "c06c3b4b-01c8-4b99-9b30-7c8bd7a676cb",
   "metadata": {},
   "source": [
    "To use a model on any device with NPU acceleration supported by **eIQ AI Toolkit**, the model must first be converted to a **quantized TF Lite** format. Depending on the specific platform, an additional conversion step may be required.\n",
    "\n",
    "Several devices, such as MCX-N, i.MX RT700, i.MX95, and i.MX943, use the Neutron NPU. To run a model on these devices with Neutron acceleration, you need to further convert the quantized TFLite model into a format specific to Neutron.\n",
    "\n",
    "**eIQ AI Toolkit** performs conversions using the **Olive** (https://microsoft.github.io/Olive/) framework.\n",
    "Olive uses **passes**, where each pass represents a single transformation of the model.\n",
    "Passes can also be chained together to apply multiple transformations in sequence.\n",
    "\n",
    "This guide will show you how to:\n",
    "- Load a quantized TF Lite model into **eIQ AI Toolkit**\n",
    "- Convert the quantized TF Lite model to a Neutron NPU-compatible TF Lite model\n",
    "- Download the converted model for use in an application on your device\n",
    "\n",
    "This guide requires the **eIQ AI Toolkit** backend to be running.\n",
    "If you haven't set it up yet, please refer to the following tutorial:\n",
    "[eIQ AI Toolkit setup & launch](../tools/aiToolkit/installRun.ipynb)"
   ]
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "import requests\n",
    "from pathlib import Path\n",
    "\n",
    "# Set your eIQ AI Toolkit url:\n",
    "AI_TOOLKIT_BACKEND_URL = \"http://localhost:8000\""
   ],
   "id": "c1cdcc830d05bae9",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Load quantized TF Lite model",
   "id": "72e2056e3b68e05a"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "Loading any type of model into an application involves two steps:\n",
    "1. Specifying the model metadata\n",
    "2. Uploading the model file\n",
    "\n",
    "The metadata must always include the type of model you are uploading. For example, if you are uploading a PyTorch model, set the type to `pytorch`; for an ONNX model, use `onnx`, and so on.\n",
    "\n",
    "For a TFLite model, no additional metadata is required."
   ],
   "id": "ff3cad58b5da5453"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### 1. Prepare quantized TF Lite model",
   "id": "4f1cfe542437cdf8"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "If you already have a model prepared, update the path to point to its location.\n",
    "If you do not yet have a model, set the path to the location where the model should be saved.\n",
    "(Refer to the following sections for instructions on downloading a sample model.)"
   ],
   "id": "912eb3b425776023"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "# Modify the path to your TFLite model\n",
    "model_path = Path(\"path_to_quantized_tflite_model.tflite\")"
   ],
   "id": "18113f7f481a0fb1",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "Use the following script to download the example model:\n",
    "\n",
    "*Note: Skip this step if you already have your own model.*"
   ],
   "id": "74ff62bb20ee58dc"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "example_model_url = \"https://eiq.nxp.com/training-materials/_misc/models/mobilenet_v3-small_224_1.0_uint8.tflite\"\n",
    "\n",
    "with open(model_path, \"wb\") as f:\n",
    "    response = requests.get(\n",
    "        url=example_model_url\n",
    "    )\n",
    "    f.write(response.content)"
   ],
   "id": "d9d591572c8bd116",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### 2. Specify metadata",
   "id": "9a0ac22d42ec3be0"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "In this section, we will upload the model metadata to **eIQ AI Toolkit**. This step only requires specifying the type of model being uploaded.",
   "id": "9e8a5ad8600cb95d"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "MODELS_API_URL = f\"{AI_TOOLKIT_BACKEND_URL}/models\"\n",
    "\n",
    "model_metadata = {\n",
    "        \"model_type\": \"tflite\",\n",
    "    }\n",
    "\n",
    "response = requests.post(MODELS_API_URL, json=model_metadata)\n",
    "response_data = response.json()\n",
    "model_uuid = response_data[\"data\"][\"model\"][\"uuid\"]"
   ],
   "id": "5977f78d6f6cf749",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### 3. Upload model",
   "id": "24221006d298aec0"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Now we can upload the model file.",
   "id": "f2698eae9416151b"
  },
  {
   "cell_type": "code",
   "id": "d14f4f5a-98b4-48dc-876b-099af681f85f",
   "metadata": {},
   "source": [
    "with open(model_path, \"rb\") as model_file:\n",
    "    response = requests.post(\n",
    "        url=f\"{AI_TOOLKIT_BACKEND_URL}/models/{model_uuid}\", # Model identifier is part of the request URL\n",
    "        files={\n",
    "            \"model_file\": model_file,\n",
    "        }\n",
    "    )\n",
    "\n",
    "print(response.json())"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "After uploading the model metadata and file, you can verify the model’s registration and readiness status using the following endpoint. If the status remains `in_progress`, call the endpoint repeatedly until it changes to `ready`.",
   "id": "ecd438137c74d178"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "response = requests.get(f\"{AI_TOOLKIT_BACKEND_URL}/models/{model_uuid}\")\n",
    "data = response.json()\n",
    "print(f'Model status: {data[\"data\"][\"model\"][\"status\"]}')\n",
    "print(f'Model status description: {data[\"data\"][\"model\"][\"status_description\"]}')"
   ],
   "id": "d8f6251357aa8a0a",
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "a5ef8bd0-e2a7-4bc5-a5a7-456b5bc9ff36",
   "metadata": {},
   "source": "## Conversion to Neutron NPU specific format"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "Before running the conversion, two important parameters must be configured:\n",
    "1. Neutron target\n",
    "2. Neutron flavor (version)\n",
    "\n",
    "The code below displays all available options for these parameters.\n",
    "**Neutron target** should be set to the device where you plan to deploy the converted model.\n",
    "**Neutron flavor** should match the exact version of the SDK (for MCUs) or Yocto (for MPUs) you are using.\n",
    "\n",
    "If you are following this guide to convert and use a model on your device, make sure to adjust these parameters accordingly."
   ],
   "id": "b1528831812d39d9"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "available_passes_response = requests.get(f\"{AI_TOOLKIT_BACKEND_URL}/optimizations/passes\")\n",
    "available_passes = available_passes_response.json()\n",
    "\n",
    "# This prints configuration parameters only for NeutronConversion pass. Feel free to change it and explore\n",
    "# other passes as well.\n",
    "neutron_conversion_pass_config = next(_pass for _pass in available_passes[\"data\"][\"passes\"] if _pass[\"type\"] == \"NeutronConversion\")\n",
    "print(neutron_conversion_pass_config)"
   ],
   "id": "6ef0d126531ae618",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "OPTIMIZATIONS_API_URL = f\"{AI_TOOLKIT_BACKEND_URL}/optimizations\"\n",
    "RUN_OPTIMIZATION_API_URL = f\"{OPTIMIZATIONS_API_URL}/run\"\n",
    "\n",
    "pass_config = {\n",
    "    \"model_uuid\": model_uuid,\n",
    "    \"passes\": [\n",
    "        {\n",
    "            \"type\": \"NeutronConversion\",\n",
    "            \"config\": {\n",
    "                \"target\": \"imxrt700\", # Change this to match your specific Neutron NPU target\n",
    "                \"flavor\": \"MCUXpresso SDK 25.09\" # Change this to match your MCU SDK or Yocto version\n",
    "            }\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "\n",
    "optimization_response = requests.post(RUN_OPTIMIZATION_API_URL, json=pass_config)\n",
    "optimization_response_data = optimization_response.json()\n",
    "optimization_uuid = optimization_response_data[\"data\"][\"optimization\"][\"uuid\"]"
   ],
   "id": "a219d410d6209334",
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "The conversion process is now running. You can check its status by calling this endpoint repeatedly until the status changes to `success`.",
   "id": "5513cb88e2cb7b08"
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "response = requests.get(f\"{OPTIMIZATIONS_API_URL}/{optimization_uuid}\")\n",
    "data = response.json()\n",
    "status = data[\"data\"][\"optimization\"][\"status\"]\n",
    "print(f\"Conversion status: {status}\")\n",
    "\n",
    "if status == \"success\":\n",
    "    artifact_id = data[\"data\"][\"optimization\"][\"artifacts\"][0][\"artifact_id\"]"
   ],
   "id": "f65cd828984cfbe0",
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "ee5aa45d-2904-4ce7-a366-7fcdd73214dc",
   "metadata": {},
   "source": "## Download converted model"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Once the conversion is complete and successful, you can download the resulting model and use it on your Neutron device.",
   "id": "7d6a7916057217b9"
  },
  {
   "cell_type": "code",
   "id": "df900a49-6fd1-4b51-a5ea-ea5351fb7f9d",
   "metadata": {},
   "source": [
    "# Change model path to your location\n",
    "dest_model_path = Path(\"neutron_converted_model.tflite\")"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "metadata": {},
   "cell_type": "code",
   "source": [
    "download_response = requests.get(f\"{AI_TOOLKIT_BACKEND_URL}/optimizations/{optimization_uuid}/resources/{artifact_id}\")\n",
    "with dest_model_path.open(\"wb\") as f:\n",
    "    f.write(download_response.content)"
   ],
   "id": "8d8be440f9afd5d7",
   "outputs": [],
   "execution_count": null
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}