How to use AI to remotely control a Raspberry PI camera

After installing the latest 12MP camera from raspberry I spent a sunny afternoon in the garden experimenting with "PI photography". I planed to place a headless PI with camera and POE module close to a bird bath and capture the action. The first task turned out to be quite formidable - simply focusing the lens. After several failures, the best solution appeared to be the RPi-Cam-Web-Interface from Silvan Melchior and Robert Tidey as it is fast enough to focus accurately when streaming to browser. The next task proved to be much more tedious. I wanted to capture videos of birds in our bird bath triggering the recording via motion detection. After the first attempts I ended up with hundreds of short videos as the bushes in the background moved in the wind and the bright sun changed the reflections in the water. There seemed to be absolutely no perfect setting. Then I remembered a AI tutorial on pyimagesearch.com - a short example demonstrating fast inference on a PC. AI appeared to offer the perfect solution to ignore the background and detect the birds. But how do you get the image from the PI to the inference PC and trigger the PI to start recording? Well it took the excellent documentation from the authors of the RPi Cam Web Interface, and a surprisingly short program extracted from the pyimagesearch.com. It runs on the PC and basically works like this:

• grab a frame from the PI using a http call 
• start the inference (single shot detection)
• if birds are detected start recording on the PI using a ssh call
• if birds are not detected stop the recording using a ssh call

The http call to grab a frame from the PI is a simple GET to http://<IP_of_PI>/html/get_pic.php. The video recording can be triggered by sending a short string to a pipe monitored by the raspimjpeg process installed with the RPi Cam Web Interface (ssh echo "vi 1" >/var/www/html/FIFO11).

No modifications to the standard installation are required PI side but this has one drawback. There is currently no known way to annotate the detected birds with boxes and detection scores within the PI stream. This can only be done on the inference PC as the image above demonstrates. The image above is not of a high quality as it has been reduced in size to speed things up and we do not require the higher resolution of the original stream image to detect birds. The only option known to us is to use the pipe communication to change the annotation of the stream (echo an "birds found" >/var/www/html/FIFO11) does the trick! 

The short "empty" motion detected videos were eliminated and replaced by bird videos. The code to trigger the PI using inference can be downloaded at https://github.com/tubematics/AIMotionDetection.