Arduino Machine Learning Vision demo

Google has created an interesting web-based teachable machine which recognizes objects captured in front of the camera and reacts accordingly via animated gif, sound or text-to-speech. It is an excellent web tool to introduce children to the basis of machine learning.

To add physical interactivity to the web-based teachable machine, we adapted the demo to allow hardware interaction as one of its output.

There are 2 main components to this project:

1. A Google Chrome extension is created to wrap around Google’s teachable machine to send its output to a “Thing” via Serial protocol over Bluetooth module. Google Chrome extension is the winning option as one can simply install it to any machine with a working Chrome Browser. It works across multiple OS seamlessly without any code changes.

2. The Bluetooth UART module (HC-05) delivers the String input to the “Thing”, an Arduino UNO. The default String of the demo are green, purple and orange and these will be processed by the sketch and light up the NeoPixels LEDs with the correct shade of colors.

 

SCHEMATIC

ARDUINO CODE


#include &ltAdafruit_NeoPixel.h&gt    // Using Adafruit Neo Pixel library to drive the LED Strip
#define PIN            6  // Define the Arduino pin used for driving the LED strip
#define NUMPIXELS      5  // Define the number of Pixel on the Neo Pixel Strip

// Define the Red Green Blue intensity for the different color pattern
#define greenLight  0,200,0
#define redLight    200,0,0
#define blueLight   0,0,200
#define purpleLight 128,0,128
#define orangeLight 200,130, 0
#define whiteLight  128,128,128
#define blackLight  0,0,0

// Initialize the Neopixel and Serial variables
Adafruit_NeoPixel pixels = Adafruit_NeoPixel(NUMPIXELS, PIN, NEO_GRB + NEO_KHZ800);
String inputString = ""; 

// Helper function to set All NeoPixels to the same Color
void setNeopixel(int Red, int Green, int Blue) {
    for(int i=0; i < NUMPIXELS; i++){
        pixels.setPixelColor(i, pixels.Color(Red,Green,Blue)); 
        pixels.show();
    }
}

void setup() {
    Serial.begin(9600);   // The Sketch file assume the HC-05 or UART connection to be
    pixels.begin();       // Set up with a baud rate of 9600
    setNeopixel(blackLight);  // Initialize the Neo Pixels to Black(Off)
}

void loop() {
    while (Serial.available()) {
      
        char inChar = (char)Serial.read();  // Read the Input from the Serial(UART) and 
        // add it to the inputString:
        if ((inChar == '\n') || (inChar == ' ')) {  // Look for the new Line character
            if (inputString.equalsIgnoreCase("green")) {  // Compare the String received 
                setNeopixel(greenLight);                  // via Serial and set the NeoPixel
            }                                             // Color accordingly.
            else if (inputString.equalsIgnoreCase("red")) {
                setNeopixel(redLight);
            }
            else if (inputString.equalsIgnoreCase("blue")) {
                setNeopixel(blueLight);
            }
            else if (inputString.equalsIgnoreCase("purple")) {
                setNeopixel(purpleLight);
            }
            else if (inputString.equalsIgnoreCase("orange")) {
                setNeopixel(orangeLight);
            }
            else if (inputString.equalsIgnoreCase("white")) {
                setNeopixel(whiteLight);
            }
            else{
                setNeopixel(blackLight);
            }         
            inputString = ""; 
        }
        else {
            inputString += inChar;  
        }
    }
}

 

USING THE APP

Step 1 : Download and install the Chrome extension “Minoyo Teachable Machine.crx” from here. (This works on all major platform, Mac, Linux and Win-blows). For instruction on how to install Chrome extension, please refer to this article by How-to Geek.

Step 2: Pair up the Bluetooth module “HC-05” with your PC, and identify the COM Port via Device manager, under “Ports (COM & LPT)”.

Step 3: Start the Chrome application “Minoyo Teachable-Machine” via chrome://apps/

Step 4: Select the correct COM port from drop down box, and click on “Start Vision Machine Learning”

Step 5: Have fun training!

You can also watch the live demonstration here:

AFTER THOUGHT

Possibilities for the kids with such simple tool are endless. Imagine attaching it to a Servo motor. It could be a pointer to different object it can identify. Attach to a door, it could be a simple “Face” id system. Albeit 3 person at the moment.

Please share with us your fun experiment via the comment below.

Leave A Comment?