Object Detection using Tensorflow: bee and butterfly Part I

homeMachine Learning >Image Processing

Object Detection using Tensorflow: bee and butterflies

  1. Part 1: set up tensorflow in a virtual environment
  2. adhoc functions
  3. Part 2: preparing annotation in PASCAL VOC format
  4. Part 3: preparing tfrecord files
  5. more scripts
  6. Part 4: start training our machine learning algorithm!
  7. COCO API for Windows
  8. Part 5: perform object detection

First preparation

Our objective here is to try using tensorflow object detection API on Windows machine. We will train our model to recognise butterflies and bees. See the following detection on some images that we obtain at the end of a 4-hour training.

predict

We assume no prerequisite knowledge and will go through step by step as much as possible. We use python 3.6. Find a compatible version from the official site here . Follow the instruction and download accordingly.

We will use command line cmd.exe for no special reason, though any other command lines are okay. Try type python in the command line and see if we get into python mode. If the command is not recognizable, set the environment variables properly. In Windows, edit or add the Variable Name to PYTHONPATH and Variable Value the location python 3.6 is installed. Typically this is just C:\Python36. And then, to the variable Path, add both %PYTHONPATH% and %PYTHONPATH%\Scripts.

Let us do this in a virtual environment. We will create a folder that serves as the virtual environment within the Desktop. Create a folder adhoc in the Desktop. Copy the directory path to adhoc (in windows 10 this can simply be found on the top-left of windows explorer) using copy path. In cmd.exe, move into this directory by typing

cd "C:\Users\acer\Desktop\adhoc"

and the above refers to my directory path to this folder adhoc. Install a python package for virtual environment. The second line creates a virtual environment and the third line activates the it. The fourth line just moves us into myproject.

pip install virtualenv
virtualenv myproject
myproject\Scripts\activate
cd myproject

Install the following dependencies in order to use tensorflow. We use tensorflow that will make use of gpu. But otherwise, if we just want to test out with cpu or if your machine does not have a gpu, replace the last line with pip install tensorflow.

pip install Cython
pip install contextlib2
pip install jupyter
pip install matplotlib
pip install pillow
pip install lxml
pip install pandas
pip install tensorflow-gpu==1.10.0

Remark: You can type deactivate to exit the virtual environment.

Another remark: We successfully used tensorflow-gpu version 1.10.0 for this series of tutorial. However, when trying it with version 1.11.0 (latest version at the moment of writing), our machine does not recognize the gpu. Test if GPU is used using this code.

Install CUDA which is required by tensorflow as well.  We used CUDA 9.0 from here. Furthermore, we also need cudnn. Download it from here.  Unzip cudnn and copy everything into where CUDA is installed (or follow the instructions in the website). In my case, it is in “C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0”.

When installed, go into python and do

import tensorflow as tf

if there is no problem, skip to the next section “More preparations”. We tested pip install tensorflow on Windows 10 on Virtual Box. When we import tensorflow, we get error asking us to install Microsoft Visual Studio 2015 redistributable. We downloaded it from the given link, but still error occurred. Then we try installing Microsoft Visual Studio Community version. Using Microsoft VS Installer, we installed VS community 2017 with the following packages 1. .NET desktop development and 2. Desktop development with C++ under Windows category. Once completed, tensorflow works.


More preparations

Create the following directories.

C:\Users\acer\Desktop\adhoc\myproject
+ ...
+ data
  - butterfly_bee_label_map.pbtxt
+ models
  - model
    - faster_rcnn_resnet101_coco.config # See next section

The following is the label map.

butterfly_bee_label_map.pbtxt

item {
id: 1
name: 'butterfly'
}

item {
id: 2
name: 'bee'
}

Getting tensorflow object detection API

Download or clone the folder models from tensorflow object detection API here. Copy paste it into the tensorflow folder, so that, in my case, we will have the directory

adhoc\myproject\Lib\site-packages\tensorflow\models
+ official
+ research
+ ...

To prevent possible error during training later, we might need the following step (see the reference here): go to ~/tensorflow/models/research/object_detection, find model_lib.py. At around line 390, edit category_index.values() to list( category_index.values() ).

The file faster_rcnn_resnet101_coco.config is obtained from inside the models directory

"C:\Users\acer\Desktop\adhoc\myproject\Lib\site-packages\tensorflow\models\research\object_detection\samples\configs\faster_rcnn_resnet101_coco.config"

We need to configure all PATH_TO_BE_CONFIGURED in the config file; we will do this later. See part 4.

Also, we need to add the following to our environment variables. If you have PYTHONPATH variable and set path to include PYTHONPATH as instructed earlier, then add to PYTHONPATH the paths to research and research\slim.  In my computer, they are

C:\Python36\Lib\site-packages\tensorflow\models\research
C:\Python36\Lib\site-packages\tensorflow\models\research\slim

Now we want to convert protos files into python files in the directory tensorflow/models/objection_detection/protos.

Download the protocol buffer from here. The page may have been updated, so find it by clicking on the Next page in the given link till you find the older release that goes by the name protoc-3.4.0-win32. Try here. Note that we are downloading version 3.4, which is an older release, since newer versions do give problems. Move protoc.exe into some directory you like. In this tutorial I move it to

C:\protoc-3.4\bin\protoc.exe

Go into the command line cmd.exe, then move into the tensorflow/models/research folder using (do adjust the paths accordingly)

cd C:\Users\acer\Desktop\adhoc\myproject\Lib\site-packages\tensorflow\models\research

and then run

"C:\protoc-3.4\bin\protoc.exe" object_detection/protos/*.proto --python_out=.

Notice that now, in tensorflow/models/research/objection_detection/protos, python files (.py) have been generated from .protos files.


Still in the \tensorflow\models\research directory, to check that things are doing fine, do

python object_detection/builders/model_builder_test.py

You will see OK if everything is fine. We will proceed to process our butterflies and bees data in the next part, part II.

Tips: If you encounter error such as No module named ‘absl’, probably you have already exited the virtual environment, i.e. the machine no longer sees the Python installed in the virtual environment. Just make sure to go back into the virtual environment every time you are working on this project, so that all the packages are well managed. From cmd.exe you can cd to myproject, and run Scripts\activate. To deactivate, just type deactivate and click Enter.