yjy_tflite_flutter

Description:

yjy tflite flutter

Overview #
TensorFlow Lite Flutter plugin provides a flexible and fast solution for accessing TensorFlow Lite interpreter and performing inference. The API is similar to the TFLite Java and Swift APIs. It directly binds to TFLite C API making it efficient (low-latency). Offers acceleration support using NNAPI, GPU delegates on Android, and Metal delegate on iOS.
Key Features #

Flexibility to use any TFLite Model.
Acceleration using multi-threading and delegate support.
Similar structure as TensorFlow Lite Java API.
Inference speeds close to native Android Apps built using the Java API.
You can choose to use any TensorFlow version by building binaries locally.
Run inference in different isolates to prevent jank in UI thread.

(Important) Initial setup : Add dynamic libraries to your app #
Android #


Place the script install.sh (Linux/Mac) or install.bat (Windows) at the root of your project.


Execute sh install.sh (Linux) / install.bat (Windows) at the root of your project to automatically download and place binaries at appropriate folders.
Note: The binaries installed will not include support for GpuDelegateV2 and NnApiDelegate however InterpreterOptions().useNnApiForAndroid can still be used.


Use sh install.sh -d (Linux) or install.bat -d (Windows) instead if you wish to use these GpuDelegateV2 and NnApiDelegate.


These scripts install pre-built binaries based on latest stable tensorflow release. For info about using other tensorflow versions refer to this part of readme.
TFLite Flutter Helper Library #
A dedicated library with simple architecture for processing and manipulating input and output of TFLite Models. API design and documentation is identical to the TensorFlow Lite Android Support Library. Strongly recommended to be used with tflite_flutter_plugin. Learn more.
Examples #



Title
Code
Demo
Blog




Text Classification App
Code

Blog/Tutorial


Image Classification App
Code

-


Object Detection App
Code

Blog/Tutorial


Reinforcement Learning App
Code

Blog/Tutorial



Import #
import 'package:tflite_flutter/tflite_flutter.dart';
copied to clipboard
Usage instructions #
Creating the Interpreter #


From asset
Place your_model.tflite in assets directory. Make sure to include assets in pubspec.yaml.
final interpreter = await tfl.Interpreter.fromAsset('your_model.tflite');
copied to clipboard


Refer to the documentation for info on creating interpreter from buffer or file.
Performing inference #
See TFLite Flutter Helper Library for easy processing of input and output.


For single input and output
Use void run(Object input, Object output).
// For ex: if input tensor shape [1,5] and type is float32
var input = [[1.23, 6.54, 7.81. 3.21, 2.22]];

// if output tensor shape [1,2] and type is float32
var output = List.filled(1*2, 0).reshape([1,2]);

// inference
interpreter.run(input, output);

// print the output
print(output);
copied to clipboard


For multiple inputs and outputs
Use void runForMultipleInputs(List<Object> inputs, Map<int, Object> outputs).
var input0 = [1.23];
var input1 = [2.43];

// input: List<Object>
var inputs = [input0, input1, input0, input1];

var output0 = List<double>.filled(1, 0);
var output1 = List<double>.filled(1, 0);

// output: Map<int, Object>
var outputs = {0: output0, 1: output1};

// inference
interpreter.runForMultipleInputs(inputs, outputs);

// print outputs
print(outputs)
copied to clipboard


Closing the interpreter #
interpreter.close();
copied to clipboard
Improve performance using delegate support #
Note: This feature is under testing and could be unstable with some builds and on some devices.
copied to clipboard


NNAPI delegate for Android
var interpreterOptions = InterpreterOptions()..useNnApiForAndroid = true;
final interpreter = await Interpreter.fromAsset('your_model.tflite',
options: interpreterOptions);

copied to clipboard
or
var interpreterOptions = InterpreterOptions()..addDelegate(NnApiDelegate());
final interpreter = await Interpreter.fromAsset('your_model.tflite',
options: interpreterOptions);

copied to clipboard


GPU delegate for Android and iOS


Android GpuDelegateV2
final gpuDelegateV2 = GpuDelegateV2(
options: GpuDelegateOptionsV2(
false,
TfLiteGpuInferenceUsage.fastSingleAnswer,
TfLiteGpuInferencePriority.minLatency,
TfLiteGpuInferencePriority.auto,
TfLiteGpuInferencePriority.auto,
));

var interpreterOptions = InterpreterOptions()..addDelegate(gpuDelegateV2);
final interpreter = await Interpreter.fromAsset('your_model.tflite',
options: interpreterOptions);
copied to clipboard


iOS Metal Delegate (GpuDelegate)
final gpuDelegate = GpuDelegate(
options: GpuDelegateOptions(true, TFLGpuDelegateWaitType.active),
);
var interpreterOptions = InterpreterOptions()..addDelegate(gpuDelegate);
final interpreter = await Interpreter.fromAsset('your_model.tflite',
options: interpreterOptions);
copied to clipboard




Refer Tests to see more example code for each method.
Use the plugin with any tensorflow version
The pre-built binaries are updated with each stable tensorflow release. However, you many want to use latest unstable tf releases or older tf versions, for that proceed to build locally, if you are unable to find the required version in release assets.
Make sure you have required version of bazel installed. (Check TF_MIN_BAZEL_VERSION, TF_MAX_BAZEL_VERSION in configure.py)

Android

Configure your workspace for android builds as per these instructions.
For TensorFlow >= v2.2
bazel build -c opt --cxxopt=--std=c++11 --config=android_arm //tensorflow/lite/c:tensorflowlite_c

// similarily for arm64 use --config=android_arm64
copied to clipboard
For TensorFlow <= v2.1
bazel build -c opt --cxxopt=--std=c++11 --config=android_arm //tensorflow/lite/experimental/c:libtensorflowlite_c.so

// similarily for arm64 use --config=android_arm64
copied to clipboard

iOS

Refer instructions on TensorFlow Lite website to build locally for iOS.
Note: You must use macOS for building iOS.
More info on dynamic linking
tflite_flutter dynamically links to C APIs which are supplied in the form of libtensorflowlite_c.so on Android and TensorFlowLiteC.framework on iOS.
For Android, We need to manually download these binaries from release assets and place the libtensorflowlite_c.so files in the <root>/android/app/src/main/jniLibs/ directory for each arm, arm64, x86, x86_64 architecture as done here in the example app.
Future Work #

Enabling support for Flutter Desktop Applications.
Better and more precise error handling.

Credits #

Tian LIN, Jared Duke, Andrew Selle, YoungSeok Yoon, Shuangfeng Li from the TensorFlow Lite Team for their invaluable guidance.
Authors of dart-lang/tflite_native.

License

For personal and professional use. You cannot resell or redistribute these repositories in their original state.

Share

• Share on Facebook
• Share on Twitter