{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c9cec4654fd41cd6",
   "metadata": {},
   "source": [
    "# Converting for i.MX 93"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd04d08994132548",
   "metadata": {},
   "source": [
    "To run models on devices with NPU acceleration supported by **eIQ AI Toolkit**, the model must first be converted to **quantized TF Lite** format. Depending on the specific platform, an additional conversion step may be required.\n",
    "\n",
    "For example, when deploying a model on **i.MX 93** devices with NPU acceleration, the quantized TF Lite model must be converted into a format optimized for execution on the i.MX 93 NPU. This process modifies the original quantized TFLite model to ensure compatibility and performance.\n",
    "\n",
    "**eIQ AI Toolkit** uses the **Olive** (https://microsoft.github.io/Olive/) framework for these conversions. Olive applies **passes**, where each pass represents a single transformation of a model. Passes can also be chained together to perform multiple transformations in sequence.\n",
    "\n",
    "This guide demonstrates how to:\n",
    "- Load a quantized TF Lite model into **eIQ AI Toolkit**\n",
    "- Convert the quantized TF Lite model into an i.MX 93 NPU-compatible format\n",
    "- Download the converted model for use in an i.MX 93 application\n",
    "\n",
    "This guide requires the **eIQ AI Toolkit** backend to be running.\n",
    "If you haven’t set it up yet, refer to the following tutorial:\n",
    "[eIQ AI Toolkit setup & launch](../tools/aiToolkit/installRun.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "261283befc683ed8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "from pathlib import Path\n",
    "\n",
    "# Set your eIQ AI Toolkit url:\n",
    "AI_TOOLKIT_BACKEND_URL = \"http://localhost:8000\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a772d551e21290c",
   "metadata": {},
   "source": [
    "## Load quantized TFLite model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c140c5b8f1cea66a",
   "metadata": {},
   "source": [
    "Loading any type of model into an application involves two steps:\n",
    "1. Specify the model metadata\n",
    "2. Upload the model file\n",
    "\n",
    "The metadata must always include the type of model being uploaded. For example, if you are uploading a PyTorch model, set the type to `pytorch`; for an ONNX model, use `onnx`, and so on.\n",
    "\n",
    "For a TFLite model, no additional metadata is required."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62f966c3b6cd5a00",
   "metadata": {},
   "source": [
    "### 1. Prepare quantized TFLite model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bd1ef4cda9f0a99",
   "metadata": {},
   "source": [
    "If you already have a model prepared, simply update the path to point to its location.\n",
    "If you do not yet have a model, set the path to the location where the model should be saved.\n",
    "(Refer to the following sections for instructions on downloading a sample model.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "58814241cf0c7f16",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Modify the path to your TFLite model\n",
    "model_path = Path(\"path_to_quantized_tflite_model.tflite\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b467dee3f3c3511c",
   "metadata": {},
   "source": [
    "Use the following script to download the example model:\n",
    "\n",
    "*Note: Skip this step if you already have your own model.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d5143cad7e24d02e",
   "metadata": {},
   "outputs": [],
   "source": [
    "example_model_url = \"https://eiq.nxp.com/training-materials/_misc/models/mobilenet_v3-small_224_1.0_uint8.tflite\"\n",
    "with open(model_path, \"wb\") as f:\n",
    "    response = requests.get(\n",
    "        url=example_model_url\n",
    "    )\n",
    "    f.write(response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3263081b154d4388",
   "metadata": {},
   "source": [
    "### 2. Specify metadata"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22193c4767509522",
   "metadata": {},
   "source": [
    "In this section, we will upload the model metadata to the eIQ AI Toolkit. This step involves specifying only the type of model being uploaded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f6ca602d14660b1",
   "metadata": {},
   "outputs": [],
   "source": [
    "MODELS_API_URL = f\"{AI_TOOLKIT_BACKEND_URL}/models\"\n",
    "\n",
    "model_metadata = {\n",
    "        \"model_type\": \"tflite\",\n",
    "    }\n",
    "\n",
    "response = requests.post(MODELS_API_URL, json=model_metadata)\n",
    "response_data = response.json()\n",
    "model_uuid = response_data[\"data\"][\"model\"][\"uuid\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa97086f1d9cb100",
   "metadata": {},
   "source": [
    "### 3. Upload model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6db8f9efdb522d6",
   "metadata": {},
   "source": [
    "Now we can upload the model file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33d138830ded1078",
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(model_path, \"rb\") as model_file:\n",
    "    response = requests.post(\n",
    "        url=f\"{AI_TOOLKIT_BACKEND_URL}/models/{model_uuid}\", # Model identifier is part of the request URL\n",
    "        files={\n",
    "            \"model_file\": model_file,\n",
    "        }\n",
    "    )\n",
    "\n",
    "print(response.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa8f3e05de3f618e",
   "metadata": {},
   "source": [
    "After uploading the model metadata and file, you can verify the model’s registration and readiness status using the following endpoint. If the status remains `in_progress`, call the endpoint repeatedly until it changes to `ready`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc456aa3f1bd743d",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(f\"{AI_TOOLKIT_BACKEND_URL}/models/{model_uuid}\")\n",
    "data = response.json()\n",
    "print(f'Model status: {data[\"data\"][\"model\"][\"status\"]}')\n",
    "print(f'Model status description: {data[\"data\"][\"model\"][\"status_description\"]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7a3ae44356d5b01",
   "metadata": {},
   "source": [
    "## Conversion to i.MX 93 specific format"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "648ae989372efda6",
   "metadata": {},
   "source": [
    "Now we can initiate the conversion by calling this endpoint, which executes the `VelaConversion` pass."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23649dcd80d9b931",
   "metadata": {},
   "outputs": [],
   "source": [
    "OPTIMIZATIONS_API_URL = f\"{AI_TOOLKIT_BACKEND_URL}/optimizations\"\n",
    "RUN_OPTIMIZATION_API_URL = f\"{OPTIMIZATIONS_API_URL}/run\"\n",
    "\n",
    "pass_config = {\n",
    "    \"model_uuid\": model_uuid,\n",
    "    \"passes\": [\n",
    "        {\n",
    "            \"type\": \"VelaConversion\",\n",
    "            \"config\": {\n",
    "                # VelaConversion does not require any configuration parameters\n",
    "            }\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "\n",
    "optimization_response = requests.post(RUN_OPTIMIZATION_API_URL, json=pass_config)\n",
    "optimization_response_data = optimization_response.json()\n",
    "optimization_uuid = optimization_response_data['data']['optimization']['uuid']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2aaf0efcee9dad7",
   "metadata": {},
   "source": [
    "The conversion process is now running. You can check its status by calling this endpoint repeatedly until the status changes to `success`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ffe6671034f9ce17",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(f\"{OPTIMIZATIONS_API_URL}/{optimization_uuid}\")\n",
    "data = response.json()\n",
    "status = data[\"data\"][\"optimization\"][\"status\"]\n",
    "print(f\"Conversion status: {status}\")\n",
    "\n",
    "if status == \"success\":\n",
    "    artifact_id = data[\"data\"][\"optimization\"][\"artifacts\"][0][\"artifact_id\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50c2a12260b6a4b8",
   "metadata": {},
   "source": [
    "## Download converted model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f147b5e6cb7fb0f8",
   "metadata": {},
   "source": [
    "Once the conversion is complete and successful, you can download the resulting model and use it in your i.MX 93 applications."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c318f981-d271-4da2-b653-95ebb7369b07",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Change model path to your location\n",
    "dest_model_path = Path(\"imx_converted_model.tflite\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "88b579b6e8a00bb8",
   "metadata": {},
   "outputs": [],
   "source": [
    "download_response = requests.get(f\"{AI_TOOLKIT_BACKEND_URL}/optimizations/{optimization_uuid}/resources/{artifact_id}\")\n",
    "with dest_model_path.open(\"wb\") as f:\n",
    "    f.write(download_response.content)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}