How to build a Pokedex with Flutter and Tflite

About

We are going to build a Pokédex with flutter, but the special thing is that it will use machine learning models to identify Pokémons based on their pictures. For learning purposes and ease for training the model, the pokédex will only recognize between Pickachu and Squirtle. You can download the final result.

Initial Configuration

Since we are going to load images and then recognize them with tflite, we are going to need some plugins. So the first step is to create a new Flutter application and then add the required plugins. If you don't know how to create a Flutter application please review the official documentation.

Add the following plugins:

1. image_picker
2. tflite

Please take your time and follow the correct installation of these plugins. They require permissions and some specific configurations, and if you don't follow them correctly the app will break.

Create the Model

The easiest way I found to train a machine learning model is using Teachable Machine with Google. There, we can name classes and add the pictures needed to train the model. Creating an "Other" class will help the model to be more accurate. You can find the pokémon image dataset in this link.

Now we have to train the model and export it in a Tensoflow Lite format. Please check that the model conversion type is selected as Floating point. The tflite plugin for flutter only works with this conversion type.

This will generate two files: labels.txt and model.tflite. We have to add these files as assets in our application. I suggest to add them in /assets/tflite/ path. So just create a directory named "tflite" inside assets folder and then add the generated files there.

Make it work

The logic for this is to be able to select a picture and then give that file to our model for it to recognize it. I will not enter in detail how to build the UI. I just want to explain the logic to make the model work. If you want to review the full code, review the repository

//Required imports
import 'package:tflite/tflite.dart';
 
Future<void> identifyImage(File file) async {
  await Tflite.loadModel(
    model: "assets/tflite/model.tflite",
    labels: "assets/tflite/labels.txt",
    numThreads: 1
  );
 
  List<dynamic> recognitions = await Tflite.runModelOnImage(
    path: file.path,
    numResults: 1,
    imageMean: 128,
    imageStd: 128,
  );
 
  if(recognitions.isNotEmpty){
    dynamic result = recognitions.first;
  }
}

This is the code to select a picture. You can trigger it from an button or whatever widget you like. Notice how we are picking an image from the device gallery and then sending it to the indentifyImage function.

//Required imports
import 'package:image_picker/image_picker.dart';
 
void pickImage() async {
  final ImagePicker _imagePicker = ImagePicker();
  PickedFile picked = await _imagePicker.getImage(source: ImageSource.gallery);
  if(picked != null) {
    File image = File(picked.path);
    identifyImage(image);
  }
}

And that's it, the variable result will store the output of the model in a json format. Then, we can do whatever we like with that result and show it in the UI.

{
  index: 0,
  label: "Pikachu",
  confidence: 0.899
}

Build your UI

The final step is to build your UI. This is my final result. I didn't explain how to build this UI because if I do, this tutorial will be very large and I think there is plenty of other posts of how to build beautiful UIs in Flutter. You can always find the full code.

Other sources for UI:

1. Flutter Pokedex
2. Flutter Layout