TF Lite Optimizer

This tool is used to apply to the model a set of elementary transformations called passes. Each pass transforms the model graph in a particular way. This tool and the associated graph transformations are generic (Neutron independent).

You can check all the available passes that the tool provides using the show option:

tflite-optimizer --show

For example, here are the first lines shown by the show command:

Available TFlite graph passes:
[ 0] DeadCodeElimination                - Eliminate dead (unused) operators and the associated output tensors.
[ 1] ConstantPropagation                - Execute operators with constant inputs and replace the operator results in the graph.
                                          Options:
                                            --constant-propagation-constant-tensors-as-input
[ 2] ShapeInference                     - Infer all the shapes of the tensors from the graph and convert them from dynamic to static.
                                          Options:
                                            --shape-inference-input-shapes
                                            --shape-inference-ignore-custom-operators
                                          Post hooks:
                                            [15] ConvertShapeToConstant
                                            [34] RemoveUnusedReshapeShape
[ 3] RemoveDuplicateConstants           - Remove duplicate constant tensors which have same data and keep a single copy by reusing it in the graph.
[ 4] InsertDuplicateConstants           - Insert duplicate constant tensors for constants with multiple consumers such that the resulting graph has constants with 
                                          only one consumer. This pass increases the model size but is useful in some cases where we need a graph with such a property.
[ 5] KeepConstantTensors                - Keep all the constant tensors from the graph by exposing them as output placeholders. This is useful for debugging.
[ 6] KeepVariableTensors                - Keep all the variable tensors from the graph by exposing them as output placeholders. This is useful for debugging.
........................................................................................................................................................................

Each pass is identified with a number (index) or a name. For example 6 and KeepVariableTensors both identify a pass which keeps the variable tensors from the graph. It is recommended though to use the pass name and not the index since the pass index could change for various versions of the tool but the name should remain fixed. The purpose of this documentation is not to describe each transformation in particular but to describe the usage of this tool.

The tflite-optimizer has the following options:

tflite-optimizer --input <input_model_path> --output <output_model_path> --run <graph_passes>
tflite-optimizer --input <input_model_path> --output <output_model_path> --optimize

where:

  • input - The path of the input TensorFlowLite model (mandatory).

  • output - The path of the output converted TensorFlowLite model (optional). If not provided the output model will be written in the same directory as the input model with the suffix _converted added to the name.

  • run - List of comma separated names or indices for the graph passes (transformations) you want to run (mandatory). This option cannot be used with the --optimize command.

  • optimize - Run only a predefined list of graph passes that optimize the graph using default options. This option cannot be used with the --run command.

Other options:

  • run-post-hooks - Option to run the post hooks after each graph pass recursively. You can see the associated post hooks for each pass in the --show command output. Default is true.

  • run-after-optimize - Option to check the model after optimization by dry running it using the TFLite interpreter with dummy input data. Default is false.

For example you can run the following command to keep all the variable tensors from the graph by exposing them as graph output placeholders:

tflite-optimizer --input model.tflite --run KeepVariableTensors

You can perform multiple transformations by providing a comma separated list of passes. The transformations will be applied once for each entry and in the same order as provided:

tflite-optimizer --input model.tflite --run DeadCodeElimination,KeepConstantTensors,KeepVariableTensors

Each graph pass can have additional command line options to control the behavior of the transformation. The graphs pass specific command line options are normally prefixed with the pass name.

For example the ConvertUint8ToInt8 pass has the following command line options:

  • --convert-uint8-to-int8-convert-inputs

  • --convert-uint8-to-int8-convert-outputs

The extra command line options associated with each graph pass are listed in the --show command and detailed in the --help command.

You can check all the command line options using the help option:

tflite-optimizer --help