Raspberry Pi HD surveillance camera with motion and Dropbox

Background

I wanted to build a surveillance camera pointing out of my house windows, using a Raspberry Pi.

I was inspired to build on top of d0fl's implementation.

What I wanted to improve on specifically is the captured image management. d0fl's solution stores each captured image on the SD card itself, and runs a web server to show the last 6 images.

I wanted to solve for the case where the Raspberry Pi gets disconnected or stolen along with the memory card where the images are getting stored, and I wanted the ability to see more than just the last 6 images. Enter Dropbox!

Summary

My solution has the following additional functionality:

• Wake up every 1 minute (via a cron job)
• Enumerate all the files that have been captured since the last run (picam-dropbox-backup.sh)
• Upload them to dropbox (dropbox_uploader.sh)
• Move the uploaded ones to a local backup folder, if the upload was successful. (dropbox_uploader.sh)
• Purge old file from by the /picam folder if we're running out of disk space (a modified picam.py that also looks in the backups directory)

The Implementation Details

If you're starting from a brand new Raspberry Pi

• Connect the camera module. On the Raspberry Pi B-mode, use the black connector right next to the Ethernet port. The metals pins of the cable should be facing away from the Ethernet port. If in doubt, watch this video.
• I started with a new Raspbian installation (NOOBS) and found that running "sudo apt-get update" at this point to update my package references avoids running into problems installing certain packages in later steps.
• Run raspi-config, and enable the Pi camera. You'll have to reboot after that.

Detecting and capturing motion: picam.py

• d0fl did a great job describing how to install picam.py. Follow his instructions for that.
• Run picam (python picam.py), and play with the configuration values until you get one that works for your setup. Here are the settings that worked best for me for daytime detection:

threshold = 40 # default value was 10
sensitivity = 500 # default value was 20
forceCapture = True
forceCaptureTime = 60 * 60 # Once an hour
filepath = "/home/pi/picam"
filenamePrefix = "capture"
diskSpaceToReserve = 40 * 1024 * 1024 # Keep 40 mb free on disk
cameraSettings = "-vf -hf" # to ensure the correct orientation of the camera. 

picam.py also deletes the old image captures , you need to modify the following section to delete the captures from the backups folder that will be created later. Modify the following part in picam.py:

# Keep free space above given level
def keepDiskSpaceFree(bytesToReserve): 
 

    if (getFreeSpace() < bytesToReserve):
        for filename in sorted(os.listdir(filepath + "backups/")):
            if filename.startswith(filenamePrefix) and filename.endswith(".jpg"):
                os.remove(filepath + "/" + filename)
                print "Deleted %s/%s to avoid filling disk" % (filepath,filename)
                if (getFreeSpace() > bytesToReserve):
                    return 
        for filename in sorted(os.listdir(filepath + "/")):
            if filename.startswith(filenamePrefix) and filename.endswith(".jpg"):
                os.remove(filepath + "/" + filename)
                print "Deleted %s/%s to avoid filling disk" % (filepath,filename)
                if (getFreeSpace() > bytesToReserve):
                    return 

Single upload: dropbox_uploader.sh

This part assumes you have a Dropbox account. If you don't have one already, go create one first.

• The dropbox_uploader.sh described at http://alexbelezjaks.com/?p=50 was my starting point. It is based on https://github.com/andreafabrizi/Dropbox-Uploader (I took a snapshot of it here http://pastebin.com/raw.php?i=THe22DeX just in case)
• The script can upload individual files on demand which is exactly what I needed. The first time you run it asks you to enter the API key and secret provided to you by dropbox.com. The script then creates a config file .dropbox_upoader in the local directory to store those values, and a private folder will be created on dropbox.com under /apps.
• If you're having issues running the scripts that you download (in particular an error complaining about $'\r': command not found on files downloaded using wget), it could be because of the newline character (the old Windows vs Unix/Linux, CR vs CR/LF problem). To solve that use dos2unix to fix the newline characters ("sudo apt-get dos2unix" and run dos2unix on each of the problematic files).

Bulk upload wrapper: picam-dropbox-backup.sh

• This is a simple script I wrote to enumerate all the files in the picam folder that haven't been uploaded yet, and upload them one at a time (using the dropbox_uploader.sh)
• If an image is successfully updated, it will move it to a "backups" folder, where it will be purged from - eventually.
• Here is the script:

#!/bin/bash
FILES=/home/pi/picam/capture*.jpg
BACKUP_FOLDER=/home/pi/picam/backups
exec &>/tmp/my.log
if [ -d $BACKUP_FOLDER ];
then
        echo "Found $BACKUP_FOLDER."
else
        mkdir $BACKUP_FOLDER
fi
echo "Looking for files..."
for f in $FILES
do
        if [ -f $f ];
        then
          echo "Processing $f file..."
          # take action on each file. $f store current file name
          /home/pi/dropbox_uploader.sh upload $f /
          echo "Dropbox upload finished with exit code = $?"
          if [ $? -eq 0 ]
          then
             mv $f $BACKUP_FOLDER/
          else
             echo "Could not upload $f"
          fi
        fi
done
echo "Done uploading."  

Run the bulk upload wrapper regularly

• I used a cron job to look for new images every minute
• A problem that my cron config avoids is the case where a cron instance is not done uploading when the next one kicks off. I solved that by adding a lock.
• Run

crontab -e 

on the command line and append the following to the file:

* * * * * root  /usr/bin/flock -xn /var/lock/script.lock -c '/bin/bash
path-to-picam-dropbox-backup.sh'

Learned from <http://stackoverflow.com/questions/13176569/cron-doesnt-run-bash-script>

• Verify that all the path references (in all the scripts) were absolute paths. That fixed some issues I had running scripts in cron.

Et voilà!

Things to Improve On

The following are features I want to get to at some point, but you implement them yourself before I do, I would appreciate leaving a comment on this article to point me to your implementation :)

1. Overlay each image with with the local date and time.
2. An alerting mechanism when there is motion captured and uploaded. http://IFTTT.com is a place you can look at.
3. A mechanism to arm and disarm the system on demand

Points of Interest

This was an interesting weekend project, and the following were new things I learned doing it.

• Python 101: white space is important in Python, and indentation is used to determine the scope :-)
• How to create locks in Bash to prevent two processes from overlapping each other.
• And last but not least, the client for the Dropbox REST interface.

What did you learn?