# TF Lite Profiler

This tool profiles a FLOAT (FP16/FP32) model by computing the dynamic range [min, max] for each FLOAT tensor and writing
the information to a file called **profile** with the following format:

  ```
  <tensor_index_1>,<min_value_1>,<max_value_1>,<freq_tensor_1_bin_1>,<freq_tensor_1_bin_2>,..,<freq_tensor_1_bin_n>,
  <tensor_index_2>,<min_value_2>,<max_value_2>,<freq_tensor_2_bin_1>,<freq_tensor_2_bin_2>,..,<freq_tensor_2_bin_n>,
  ...
  ```

where:

- `<tensor*_index>` is the tensor index used as tensor identifier (the `location` attribute that you can view in the
  Netron visualizer).
- `<tensor*_min>` is the minimum FLOAT value recorded for this tensor across all dataset samples.
- `<tensor*_max>` is the maximum FLOAT value recorded for this tensor across all dataset samples.
- `<tensor*_histogram_bin*>` is the frequency count for each histogram bin of the tensor's value distribution, where the
  first `*` represents the tensor index and the second `*` represents the bin number.

To be noted that the profile file only contains the information related to variable tensors (activations).
The constant tensors (weights) are NOT profiled because the data statistics is fixed (independent on dataset) and
already part of the model.

## Prerequisites

- Model must be a FLOAT TensorFlow Lite model. To obtain a tflite floating point model there are multiple conversion
  strategies:
    1. Use TensorFlow's model conversion API, see example script in *scripts/tensorflow_conversion.py*. More details
       here: https://www.tensorflow.org/api_docs/python/tf/lite/TFLiteConverter
    2. Use AI Edge Torch from Pytorch to TFLite conversion: see more details
       here: https://github.com/google-ai-edge/ai-edge-torch
- Dataset inputs must be in floating point binary format, having the input range required by the model training (e.g.,
  for image classification there are models trained with input 0..1 or models trained on input normalized to Imagenet
  which is approximately -2..2).
- A representative subset of training samples should be used for profiling/calibration purposes, with typically 200-1000
  samples being sufficient to capture the dynamic tensor value distributions. In order to validate the
  profiling/quantization results, it is recommended to use a validation dataset separate from the calibration dataset to
  measure accuracy.
- The representative dataset should be provided in a floating-point binary format with input ranges matching the
  original model's training configuration. An example script is provided in *./scripts/serialize_image.py*.
- The representative dataset should provide diverse coverage of the model's expected outputs, with samples distributed
  across different output categories to ensure comprehensive profiling of activation ranges.

## Usage

The **tflite-profiler** has the following syntax:

```bash
tflite-profiler --input <input_model_path> --dataset <input1>,<dataset1>,<input2>,<dataset2>,... [OPTIONS]
```

## Required Parameters

- `--input` - The path to the input FLOAT TensorFlowLite model.

- `--dataset` - Input dataset description which provides the dataset for each graph input tensor in a comma-separated
  list with the form: `<input1>,<dataset1>,<input2>,<dataset2>,...`.
    - Inputs `input<N>` are identified by the tensor name or the tensor index (location) in the model.
    - If `input<N>` is not specified and only `dataset<N>` is present, the profiler will automatically assign the tensor
      name based on the order of the input datasets for models with multiple inputs.
    - Datasets `dataset<N>` are identified by a `directory` path containing all the files.

## Optional Parameters

- `--output` - The path for the output profile file in CSV format. If not provided, the profile file will be written in
  the same directory as the input model with the suffix **_profile** added to the filename.

- `--dataset` - Input dataset description which provides the dataset for each graph input tensor in a comma-separated
  list with the form: `<input1>,<dataset1>,<input2>,<dataset2>,...`.
    - Inputs `input<N>` are identified by the tensor name or the tensor index (location) in the model.
    - If `input<N>` is not specified and only `dataset<N>` is present, the profiler will automatically assign the tensor
      name based on the order of the input datasets for models with multiple inputs.
    - Datasets `dataset<N>` are identified by a `directory` path containing all the files.

- `--graph-name` - For models with multiple graphs, this specifies the name of the graph to profile. This parameter is
  ignored for single-graph models.

- `--dump-instrumented-model` - Flag to save the instrumented model. Default is `false`. When `true`, use
  `--dump-instrumented-model-name` to specify the output filename.

- `--dump-instrumented-model-name` - Output filename for the instrumented model. Default is the input filename with **
  _instrumented** suffix.

- `--histogram-num-bins` - Number of bins used to calculate the histogram for each variable tensor. Default is `256`.

## Example

```bash
tflite-profiler --input model.tflite --dataset input_data_dir --output model_profile.csv
```