Charmed Labs: Pixy2 CMUcam5 Camera Image Sensor

• Charmed Labs Pixy 2 CMUcam5 Image Sensor
• Smaller and faster (60 frames per second) than the previous version (Pixy1)
• New “line following” mode with custom algorithms for detecting and tracking lines
• Improved PixyMon software (PC/Mac/Linux application)

The Charmed Labs Pixy 2 CMUcam5 Image Sensor is smaller, faster and more capable than the original Pixy.  Like its predecessor, Pixy2 can learn to detect objects that you teach it, just by pressing a button.  Additionally, Pixy2 has new algorithms that detect and track lines for use with line-following robots.  The new algorithms can detect intersections and “road signs” as well. The road signs can tell your robot what to do, such as turn left, turn right, slow down, etc.

If you want your robot to perform a task such as picking up an object, chasing a ball, locating a charging station, etc., and you want a single sensor to help accomplish all of these tasks, then vision is your sensor. Vision (image) sensors are useful because they are so flexible. With the right algorithm, an image sensor can sense or detect practically anything. But there are two drawbacks with image sensors: 1) they output lots of data, dozens of megabytes per second, and 2) processing this amount of data can overwhelm many processors. And if the processor can keep up with the data, much of its processing power won’t be available for other tasks.

Pixy2 addresses these problems by pairing a powerful dedicated processor with the image sensor. Pixy2 processes images from the image sensor and only sends the useful information (e.g. purple dinosaur detected at x=54, y=103) to your microcontroller. And it does this at frame rate (60 Hz). The information is available through one of several interfaces: UART serial, SPI, I2C, USB, or digital/analog output. So your Arduino or other microcontroller can talk easily with Pixy2 and still have plenty of CPU available for other tasks.

60 frames per second
What does “60 frames per second” mean? In short, it means Pixy2 is fast. Pixy2 processes an entire image frame every 1/60th of a second (16.7 milliseconds). This means that you get a complete update of all detected objects’ positions every 16.7 ms. At this rate, tracking the path of falling/bouncing ball is possible. (A ball traveling at 40 mph moves less than a foot in 16.7 ms.) If your robot is performing line following, your robot will typically move a small fraction of an inch between frames.

Purple dinosaurs (and other things)
Pixy2 uses a color-based filtering algorithm to detect objects called the Color Connected Components (CCC) algorithm. Color-based filtering methods are popular because they are fast, efficient, and relatively robust. Most of us are familiar with RGB (red, green, and blue) to represent colors. Pixy2 calculates the color (hue) and saturation of each RGB pixel from the image sensor and uses these as the primary filtering parameters. The hue of an object remains largely unchanged with changes in lighting and exposure. Changes in lighting and exposure can have a frustrating effect on color filtering algorithms, causing them to break. Pixy2’s filtering algorithm is robust when it comes to lighting and exposure changes.

Seven color signatures
Pixy2's CCC algorithm remembers up to 7 different color signatures, which means that if you have 7 different objects with unique colors, Pixy2’s color filtering algorithm will have no problem identifying them. If you need more than seven, you can use color codes (see below).

PixyMon lets you see what Pixy sees
PixyMon is an application that runs on Windows, MacOs and Linux. It allows you to see what Pixy sees, either as raw or processed video. It also allows you to configure your Pixy, set the output port and manage color signatures. PixyMon communicates with Pixy over a standard mini USB cable. PixyMon is great for debugging your application.

Controller support
Pixy can easily connect to lots of different controllers because it supports several interface options (UART serial, SPI, I2C, USB, or digital/analog output), but Pixy began its life talking to Arduinos. Over the last several months support for Arduino Due, Raspberry Pi and BeagleBone Black have been added.