Setting Up an Android AI Development Environment

The following parts of the series will guide you through creating the Android application, preparing a pre-trained neural network for use in TensorFlow Lite, and considerations for improving performance and the steps that would be needed to train your own visual recognizer.

What are Deep Neural Networks?

Deep neural networks (DNN) are awesome at image classification and other types of automatic classifications. Results that took millions of dollars in research and computing power a decade ago are now available to anyone with a half-decent GPU. But there is a downside that one may encounter trying to run DNNs on mobile devices. When implemented with conventional code, DNNs may be processor intensive and slow on mobile devices. Fortunately, TensorFlow Lite offers a solution for scaling down a DNN so it can run more smoothly on a mobile device.

DNNs are one of many ways in which artificial intelligence can be implemented. Neural networks are one of the AI approaches based in biomimicry (designing a technology and process based on biological entities). DNNs are based on neurons. Biological neurons are interconnected, may activate in response to external stimuli from the environment or from other neurons, and stimulate other neurons that they are connected to. Software neurons are similar, with their external stimuli coming from either saved data or from sensing devices like cameras and microphones.

The way in which software neural networks are implemented and connections are formed is worthy of a discussion of its own. If you would like to learn about it in more detail, check out AI: Neural Network for beginners.

In a nutshell, a single software neuron (or node) has a number of inputs accepting some numerical value, and for each input, the value is multiplied by some number and has a base value added to it. The results of all of these inputs are added together and passed through an activation function to decide what value the neuron will output. These various neurons are connected together to pass information to each other. The collective configuration of these neurons is known as a model. For a single neuron, there are a number of mathematical operations involved. For a complete neural network model, there may be thousands of interconnected neurons involved.

A typical computer processor can perform sequential operations quickly. But even a multi-core CPU is extremely limited on the number of parallel operations that it can do compared to a GPU. GPUs were designed for performing math on groups of vectors at once. This is necessary for both high-performance 3D rendering and training neural networks. CPUs are extremely fast, but not at the right thing that we need for 3D rendering or training neural networks.

GPUs are designed to perform a massive number of parallel operations so that they can quickly process and display the shapes and pixels that make up a visual scene. GPUs don’t care about the ultimate purpose of the data that they process though, and it was found that they could also be used for more general purpose computing. Some GPUs are now also available with native APIs for performing general purpose computation. On mobile devices, making clever use of the GPU can speed up the processing of a neural network significantly. In addition to the GPU, there are now mobile devices available with hardware that was specifically designed for processing neural networks. Using TensorFlow lite, your code can take advantage of the available hardware acceleration.

Before diving into neural networks on Android, you will need to set up a development environment. You can develop from Windows, macOS, or Linux. The software you will need is available for all three platforms:

• For phone development and deployment, you will need Android Studio.
• For training and converting networks you will need Python 3.7+.
• Netron, while not absolutely necessary, can be of great utility at times when you need to visually examine a saved Neural Network model.

Once you have all of these installed, you can start with the configuration. Since Python will be used from the command line, you will want to add its install location to your PATH variable so that you do not need to type its full path to use it.

Also, there are additional Python libraries that you will need to install to your environment. These libraries are easily installed from the command line with the PIP utility. The libraries that you will want to install include the following:

• NumPy - numerical computing library for working with arrays, linear algebra, and more.
  pip3 install numpy
• ONNX-TF - for converting network models from the Open Neural Network Exchange format.
  pip3 install git+https://github.com/onnx/onnx-tensorflow.git
• TensorFlow - for creating or working with neural network models.
  For Windows and Linux with GPU support: pip3 install tensorflow
  For Windows, Linux, and maxOS: pip3 install tensorflow-gpu
• Tensorflow_addons
  pip3 install tensorflow_addons

For Tensorflow, there are multiple installation options. By default, a compatible version of tensorflow will install. The compatible version, while not the fastest, will work on the widest range of hardware. If your computer has a processor that supports instructions for certain vector operations then TensorFlow will print a message informing you that the computer supports options that could be used to improve performance.

If your computer has a supported graphics adapter, you can also install a version of TensorFlow with GPU support. At the time of this writing, NVIDIA’s CUDA enabled GPUs are the family of graphics cards that are supported. If you have a compatible video card and want to use the GPU accelerated version of TensorFlow, you could install it with the following command:

pip3 install tensorflow-cpu

Android Studio must also be configured to perform native code compilation with the NDK. You will not need to write any native code yourself, but the TensorFlow Lite Android libraries are dependent on this. To install the NDK, open the Tools menu and select SDK Manager. Under the SDK Tools tab is an item labeled NDK (Side by side). Selecting this will cause the most recent version of the NDK to install. TensorFlow Lite is updated frequently. To install the version I used (21.0.6113669), select the Show Package Details checkbox in the lower-right. Once it’s selected, expand NDK (Side by Side) to see the NDK versions.

After selecting the SDK version and clicking on OK, Android Studio will download and install the components. When the download is complete, test that the correct version of the NDK is present by making a new Android project and adding a TensorFlow reference to it. Within Android Studio, open the File menu and select New Project. Select Empty Activity and accept the defaults for this project. Click Finish.

In the project’s build.gradle file in the dependencies section, add the following line:

implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly'

Then build the project. If all goes well then the project will compile. If the NDK requirements have changed you should receive an error noting that the required NDK version isn’t found that also provides the version that is needed. If you encounter this, open the SDK manager and select the NDK version that the error identified.

As a test for the Python and TensorFlow installation, let’s make a quick application that uses TensorFlow. For this application, TensorFlow will get a list of numbers and associated outputs and will have to figure out the relationship between them. For this test, the numbers will have a linear relationship to each other. The input values will be

2, 5, 10, 12, 24, 32, 66, 100

The output values will be

5, 14, 29, 35, 71, 95, 197, 299

Discovering relationships between number sets like this is known as linear regression. The relationships between these values can be expressed with the equation y = 3*x-1. In this code, a simple network is created to learn the relationship and then calculate an output from the test value 7:

import tensorflow as tf
  
print(tf.__version__)

#This is the training data
training_x = [2, 5, 10, 12, 24, 32, 66, 100]
training_y = [5, 14, 29, 35, 71, 95, 197, 299]

#Building the network that will be trained
layer0 = tf.keras.layers.Dense(units=1, input_shape=[1])
model = tf.keras.Sequential([layer0])

#Compiling the network
model.compile(loss='mean_squared_error', optimizer=tf.keras.optimizers.Adam(1))

#Training
history=model.fit(training_x, training_y, epochs=1000, verbose=False)

#Show the network weights
weights = layer0.get_weights()
print('weight: {} bias: {}'.format(weights[0], weights[1]))

#Try prediction
print(model.predict([7]))

Save this program in a file named regression.py. To run the program, from the terminal run the following command.

python3 regression.py

When this runs, TensorFlow will run it’s cycles of training before printing the output for the input value of 7.

weight: [[3.]] bias: [-1.000001]
[[20.]]

If the code runs and prints output similar to the above then you have a working TensorFlow environment. Linear regression is a common data analysis and prediction tool. But we want to do something more complex with visual recognition.

Next Steps

We’re now set up and ready for Android and TensorFlow development. In the next article, we’ll learn how to take a pre-trained neural network and adapt it for use in TensorFlow Lite.