Neutron Runner¶

The eIQ Neutron Runner is a validation and execution tool for TensorFlow Lite models that integrates Neutron’s golden kernels into the TFLite runtime. It enables bit-exact validation of models compiled with the Neutron converter by executing them using reference implementations of Neutron operators.

Overview¶

Neutron Runner operates in two modes:

Neutron Mode: Executes Neutron-converted models using bit-exact golden kernel implementations integrated into the TFLite runtime
Vanilla Mode: Executes standard TFLite models using the original TFLite operators

This tool is primarily used for:

Bit-exactness validation: Verify that Neutron-converted models produce identical outputs to the golden reference
Functional testing: Validate model correctness before deployment
Debugging: Compare Neutron operator behavior against reference implementations

Operator Fallback Behavior¶

When running in Neutron Mode, if an operator is not supported by the Neutron golden kernels, the tool automatically falls back to the standard TFLite Vanilla implementation for that operator. This ensures that models containing a mix of supported and unsupported operators can still be executed.

After execution completes, any unsupported kernels that fell back to TFLite Vanilla will be displayed in the console output, allowing you to identify which operators were not executed using Neutron golden kernels.

Prerequisites¶

For Neutron Mode¶

Model must be converted using neutron-converter using --keep-graphs option in order to keep all the subgraphs.
Using above option, the model will contain NeutronGraph and NeutronOperator.
Supported targets: imx95 and imxrt700

For Vanilla Mode¶

Any standard TensorFlow Lite model (FP32, FP16, INT8)
No conversion required

Usage¶

The neutron-runner has the following syntax:

neutron-runner --input <model_path> --dataset <dataset_path> [OPTIONS]

Required Parameters¶

--input - Path to the input TensorFlow Lite model file. Can be either:
- A Neutron-converted model (for Neutron mode)
- A standard TFLite model (for Vanilla mode)
--dataset - Path to the input dataset. Supports multiple formats:
- Single directory: All input tensors’ data files in one directory
- Single file: One binary file for single-input models
- Multiple paths: Comma-separated directories or files (one per input tensor)
See Dataset Format section for detailed information.

Optional Parameters¶

Output Configuration¶

--output-results - Path for the output results file in binary format. If not provided, results will be written to the same directory as the input model with the suffix _results.bin added to the filename.

Model Configuration¶

--graph-name - For models with multiple subgraphs, specifies the name of the graph to execute. This parameter is ignored for single-graph models.
--graph-index - For models with multiple subgraphs, specifies the index (0-based) of the graph to execute. Use -1 to auto-select the main graph. Default is -1.

Runtime Configuration¶

--use-neutron-runtime - Selects the execution mode:
- true (default): Use Neutron Runtime with golden kernel implementations. Requires a Neutron-converted model. Unsupported operators automatically fall back to TFLite Vanilla.
- false: Use vanilla TFLite runtime with original operators. Works with any standard TFLite model.
--target - Specifies the Neutron hardware target platform. Options: imx95, imxrt700. Only used when --use-neutron-runtime=true. Default is imx95.

Debugging Options¶

--verbose - Enables detailed console output including:
- Model structure information
- Dataset mode detection
- Operator execution details
- Input/output tensor shapes and data types
- Execution timing information
Options: true, false. Default is false.

Execution Modes¶

Neutron Mode (Bit-Exact Validation)¶

This mode executes Neutron-converted models using golden kernel implementations integrated into the TFLite runtime.

Use case: Validate that your Neutron-converted model produces bit-exact results compared to the reference implementation.

neutron-runner \
  --input model_neutron.tflite \
  --dataset ./test_inputs \
  --use-neutron-runtime=true \
  --target imx95 \
  --verbose=true

Requirements:

Model must be converted with neutron-converter
Model contains NeutronOperator custom operators
Golden kernels are available for the target platform

Note: If any operator is not supported by Neutron golden kernels, it will automatically fall back to the TFLite Vanilla implementation. Unsupported kernels will be listed at the end of execution.

Vanilla Mode (Standard TFLite Execution)¶

This mode executes standard TFLite models using the original TFLite operator implementations.

Use case: Run inference on standard TFLite models for comparison or baseline validation.

neutron-runner \
  --input model_standard.tflite \
  --dataset ./test_inputs \
  --use-neutron-runtime=false \
  --verbose=true

Requirements:

Any valid TensorFlow Lite model
No conversion needed

Dataset Format¶

The --dataset parameter supports multiple input formats to accommodate different model architectures and testing scenarios.

Dataset Modes¶

Mode	Format	Use Case
Single directory	`--dataset <directory>`	Single-input models, multiple datasets
Single file	`--dataset <file.bin>`	Single-input models, single dataset
Multiple paths (auto-mapped)	`--dataset <path1>,<path2>,...`	Multi-input models
Explicit mapping	`--dataset <name1>,<path1>,<name2>,<path2>,...`	Multi-input models with explicit tensor mapping

Single Directory Input¶

When --dataset points to a directory, each file represents one inference run. This mode only works for models with a single input tensor.

test_inputs/
├── input_0.bin    # First inference
├── input_1.bin    # Second inference
└── input_2.bin    # Third inference

neutron-runner --input model.tflite --dataset ./test_inputs

Single File Input¶

When --dataset points to a single file, it should contain raw binary tensor data. This mode only works for models with a single input tensor.

# For a model with input shape [1, 224, 224, 3] and float32 type
# File size should be: 1 * 224 * 224 * 3 * 4 bytes = 602,112 bytes
neutron-runner --input model.tflite --dataset input.bin

Multiple Paths (Auto-Mapped)¶

For models with multiple input tensors, you can provide comma-separated paths (directories or files). The paths are automatically mapped to input tensors in order.

Multiple Directories¶

Each directory corresponds to one input tensor. Files within each directory represent different inference runs:

input_tensor_0/
├── sample_0.bin
├── sample_1.bin
└── sample_2.bin

input_tensor_1/
├── sample_0.bin
├── sample_1.bin
└── sample_2.bin

neutron-runner --input model.tflite --dataset ./input_tensor_0,./input_tensor_1

Multiple Files¶

For a single inference with multiple input tensors, provide comma-separated file paths:

neutron-runner --input model.tflite --dataset input0.bin,input1.bin,input2.bin

Note: The number of paths must match the number of model input tensors.

Explicit Input Mapping¶

For precise control over which data maps to which input tensor, use explicit mapping with tensor names:

neutron-runner --input model.tflite --dataset input_1,./dir1,input_2,./dir2

Format: <input_name1>,<path1>,<input_name2>,<path2>,...

This is useful when:

Input tensor order is not obvious
You want to ensure correct mapping regardless of model input order
Debugging multi-input models

Binary Data Format¶

Binary files should contain raw tensor data in the model’s expected input format:

Data Type	Bytes per Element
float32	4
float16	2
int8	1
uint8	1
int32	4

File size calculation: batch_size × height × width × channels × bytes_per_element

Example for a [1, 224, 224, 3] float32 input:

1 × 224 × 224 × 3 × 4 = 602,112 bytes

Output Format¶

Results are written in binary format to the specified output file (or <input_model>_results.bin by default).

Output Modes¶

Directory input mode: Results are saved to individual files with pattern <output_path>_<index>.bin
Single file input mode: All results are saved to a single file

The output file contains:

Raw output tensor data in the model’s output format
For multiple outputs, data is concatenated in the order of model output tensors
For multiple inferences (directory input), results are saved to separate files

Examples¶

Example 1: Validate Neutron-Converted Model¶

neutron-runner \
  --input mobilenet_v2_neutron.tflite \
  --dataset ./validation_images \
  --output-results ./results/mobilenet_validation.bin \
  --use-neutron-runtime=true \
  --graph-name "inference_graph" \
  --target imx95 \
  --verbose=true

This command:

Executes a Neutron-converted MobileNetV2 model
Uses golden Neutron kernels for bit-exact validation
Processes all images in the validation directory
Saves results to individual files: mobilenet_validation_0.bin, mobilenet_validation_1.bin, etc.
Prints detailed execution information
Displays any unsupported kernels at the end
Choose the main graph for execution inference_graph.

Example 2: Multi-Input Model with Multiple Directories¶

neutron-runner \
  --input multi_input_model.tflite \
  --dataset ./image_inputs,./metadata_inputs \
  --use-neutron-runtime=true \
  --target imx95

This command:

Executes a model with two input tensors
Maps ./image_inputs to the first input tensor
Maps ./metadata_inputs to the second input tensor
Processes all matching files across directories

Example 3: Explicit Input Tensor Mapping¶

neutron-runner \
  --input complex_model.tflite \
  --dataset image_input,./images,mask_input,./masks \
  --use-neutron-runtime=true \
  --target imx95 \
  --verbose=true

This command:

Explicitly maps directories to named input tensors
./images directory → image_input tensor
./masks directory → mask_input tensor

Typical Workflow¶

Bit-Exactness Validation Workflow¶

Convert model using neutron-converter:

neutron-converter --input model.tflite --output model_neutron.tflite --target imx95 --keep-graphs

Run with Neutron runtime (golden kernels):

neutron-runner --input model_neutron.tflite --dataset inputs/ --use-neutron-runtime=true --target imx95

Run with vanilla runtime (reference):

neutron-runner --input model.tflite --dataset inputs/ --use-neutron-runtime=false

Compare outputs to validate bit-exactness:

diff model_neutron_results_0.bin model_results_0.bin

Review unsupported kernels (displayed after Neutron mode execution) to understand which operators fell back to TFLite Vanilla.

Supported Kernels¶

The following operators have Neutron golden kernel implementations:

Operator	Description
Conv2D	All Conv2D flavours
DepthwiseConv2D	All DepthwiseConv2D flavours
GlobalAveragePool	Global average pooling operation
BatchNorm	Batch normalization
Add	Element-wise addition

Note: Operators not in this list will automatically fall back to TFLite Vanilla implementation when running in Neutron mode. The list of unsupported kernels encountered during execution will be displayed at the end of the run.

Supported Targets¶

Target	Description
`imx95`	i.MX 95 platform (default)
`imxrt700`	i.MX RT700 platform

Unsupported Kernel Reporting¶

When running in Neutron mode (--use-neutron-runtime=true), the tool will:

Execute supported operators using Neutron golden kernels
Automatically fall back to TFLite Vanilla for unsupported operators
Display a summary of unsupported kernels at the end of execution

Example output:

Running Neutron model...
Neutron Runner completed successfully!

Unsupported kernels (executed with TFLite Vanilla):
  - Softmax
  - Reshape
  - Squeeze

This information helps you understand which parts of your model are not using Neutron golden kernels for bit-exact validation.

Troubleshooting¶

Error: “Model does not contain Neutron operators”¶

Cause: Using --use-neutron-runtime=true with a non-converted model.

Solution: Either convert the model with neutron-converter or use --use-neutron-runtime=false.

Error: “Input data size does not match expected tensor size”¶

Cause: Input file size doesn’t match expected tensor dimensions.

Solution: Verify input file size matches: batch_size × height × width × channels × sizeof(datatype) (applicable for most vision models)

Error: “Number of input files does not match number of model inputs”¶

Cause: When using comma-separated paths, the number of paths doesn’t match the model’s input count.

Solution: Ensure you provide exactly one path per input tensor. Use --verbose=true to see the number of model inputs.

Error: “Single file mode requires a model with exactly 1 input”¶

Cause: Attempting to use a single file dataset with a multi-input model.

Solution: Use comma-separated files or directories for multi-input models:

neutron-runner --input model.tflite --dataset input0.bin,input1.bin

Error: “Graph not found”¶

Cause: Specified --graph-name or --graph-index doesn’t exist in the model.

Solution: Use --verbose=true to list available graphs, or use -1 for auto-selection.

Error: “Dataset path does not exist”¶

Cause: One or more paths in the dataset parameter don’t exist.

Solution: Verify all paths exist and are accessible. Check for typos in path names.

Notes¶

Neutron Runner is a validation tool, not a deployment runtime. It uses golden kernel implementations for bit-exact verification.
For production deployment, use the optimized Neutron hardware runtime.
Output results are in raw binary format for precise numerical comparison.
When using directory mode, results are saved to individual files for easier comparison.
Unsupported operators automatically fall back to TFLite Vanilla, ensuring model execution completes even with partial Neutron support.
Always check the unsupported kernel report to understand which operators were not validated with Neutron golden kernels.