Download notebook

Microcontrollers

This guide walks you through the process of profiling a TensorFlow Lite model on an MCU device using the eIQ AI Toolkit MCU profiling feature.

You’ll Learn key eIQ AI Toolkit API endpoints relevant to model profiling

Note: The MCU profiling supports only TF Lite models. Other model formats require conversion to TF Lite.

Note: This guide was developed and run using Python 3.11.

This guide requires the eIQ AI Toolkit backend to be running. If you haven’t set it up yet, please refer to the following tutorial: eIQ AI Toolkit setup & launch

import requests
from pathlib import Path

# Set your eIQ AI Toolkit url:
AI_TOOLKIT_BACKEND_URL = "http://localhost:8000"

Model

If you already have a trained model ready, simply update the path to point to its location. If you don’t have a trained model yet, set the path to a location where the model should be saved. (See the following sections for instructions on how to download a sample model.)

model_path = Path("your_model_path.tflite")

Use the following command or script to download the example model:

Note: Skip this step if you already have your own model.

example_model_url = "https://eiq.nxp.com/training-materials/_misc/models/mobilenet_v3-small_224_1.0_uint8.tflite"

with open(model_path, "wb") as f:
    response = requests.get(
        url=example_model_url
    )
    f.write(response.content)

Model info

MCU profiling requires profiling content in JSON format. The profiling content is generated on the MCU device using ModelRunner utility . Here’s a sample JSON schema for MCU profiling:

{
    "timing": 75000,
    "ktensor_arena_size": 1240,
    "inputs": [
        {
            "name": "input_name",
            "scale": 0,
            "zero_points": 0,
            "datatype": "FLOAT32",
            "shape": [
                1,
                3,
                3
            ]
        }
    ],
    "outputs": [
        {
            "name": "output_name",
            "scale": 0,
            "zero_points": 0,
            "datatype": "FLOAT32",
            "shape": [
                1,
                1
            ]
        }
    ],
    "layer_count": 8,
    "layers": [
        {
            "name": "layer_name",
            "type": "layer_type",
            "avg_timing": 7000,
            "tensor": {
                "timing": 7000
            }
        }
    ]
}

To generate the JSON schema:

1. Flash the MCU with eIQ example tflm_modelrunner.

2. With tflm_modelrunner example running on the MCU, follow the example guide to generate the Model Info JSON file.

Save the Model Info JSON on your local machine and set the file path. Set arbitrary profiling run name:

profiling_file_path=Path("model_info.json")
profiling_run_name="example_mcu_profiling_run"

Profiling

With the profiling content file invoke the eIQ AI Toolkit endpoint /profiling/run_mcu_profiling:

with open(profiling_file_path, "rb") as profiling_file:
    response = requests.post(
        url=f"{AI_TOOLKIT_BACKEND_URL}/profiling/run_mcu_profiling",
        files = {
            "profiling_file": (profiling_file_path.name, profiling_file, "application/json"),
            "name": (None, profiling_run_name),
        }
    )

data = response.json()
print(data)
profiling_uuid = data["data"]["profiling"]["uuid"] # Assigned identifier of the requested profiling run

After initiating the profiling job, you can monitor its progress using the following API call:

response = requests.get(
    url=f"{AI_TOOLKIT_BACKEND_URL}/profiling/{profiling_uuid}" # Profiling run identifier is part of the request URL
)

data = response.json()
print(data)
print(f'Profiling status: {data["data"]["profiling"]["status"]}')
print(f'Profiling status description: {data["data"]["profiling"]["status_description"]}')

Once the profiling status is marked as success you can proceed to analyze the results. If the status is still in_progress, re-run the status check (cell above) until the profiling completes.

profiling_data = data["data"]["profiling"]
print(profiling_data)