Simulating and controlling GE Color Effects Lights with Arduino

• Download source - 669.1 KB
• Download Arduino compiler, etc.
• Watch the actual lights on YouTube

Overview

This project builds a Christmas lights controller for the GE Color Effects lights allowing programmed control of up to 8 sets of Christmas lights. Furthermore, it provides a function specific language for programming patterns for these lights and an emulation environment for testing the programs requiring the lights to be built or hung.

Pre-reading

Arduino - If you have never heard of an Arduino, then I suggest you check out their site. They design and sell an open-source microprocessor platform which is great for building programmable electronic gadgets for those with limited electronics experience.

GE Color Effects - These have to be the coolest Christmas lights ever for the hacker. Each bulb contains a red, green and blue LED and can be controlled individually or as a group by sending instructions down a single data line. For a description of what they are and how they work, checkout this article which also provides basic control information that Robert Quattlebaum (darco) reverse engineered. While you are at it, do some Youtube searching for some very cool demos of what can be done with a microprocessor and a set of these lights (The ten string Christmas tree has to be seen to be believed).

Introduction

This article is made up of a hardware project and 3 programs:

• The hardware project uses an Arduino Mega 2560 to control up to eight sets of lights.
• The first program is the firmware which runs on the Arduino Mega to control the lights. It does this by reading a set of instructions which are expressed in the form of an array of short integers for each string of lights which tell the firmware what patterns to generate with the lights. The firmware can also be cross compiled and run on the PC where they can do the same job but control a lights emulator. On the PC, I use the express version of VC++ to compile it.
• The second program is the lights emulator. This will emulate one or more sets of lights arranging them on screen according to your desired layout. When you run the firmware on the PC, this firmware will send the instructions to the emulator which will display the lights as they would appear. I use the express version of VC# to compile it.
• The final program is a compiler (or more correctly an assembler) which takes a higher level instruction set and generates the required array of short integers the firmware uses to tell it what to do with the lights. I use the express version of VC# to compile it.

This diagram gives you and idea of how everything works together ...

To try this out, you don't need an Arduino or to build any hardware ... the compiler, the firmware (built for the PC) and the Emulator all run on your PC.

Hardware

I have significantly simplified the hardware design by removing the isolation circutry and going with a common ground configuration across multiple adapters. I was concerned about this causing problems but it seems to work ok.

In my first prototype, I used an Arduino Pro. It is cheap and works well but I quickly ran into memory issues as it only has 2K of RAM. Even with several refactorings to minimise memory use, I found I was running out of memory controlling 1 to 2 sets of lights. So I upgraded to the Arduino Mega 2560 with 8K of RAM. Still paltry but enough if you are very frugal with your memory use and you put all the static data such as the instructions into the more generous 256K of flash memory.

The Color Effects light comes with a 3A 5V adapter which is powerful enough to run 2 strings of lights. With 8 strings this requires 4 adapters and a 7.5V adapter to run the arduino. The circuit also includes some key elements which make it easy to use/test:

• A reset button which resets the program ... as it does not cut power to the lights it does not reset any bulb addressing issues so sometimes you still need to cut the power
• 2 debug buttons. These can be held down on power up either singly or together to put to unit in one of 3 debug modes.
• A power LED that shows the arduino has power.
• An error LED that is used to flag errors and other conditions
• A progress LED which flashes as the program runs showing that all is good.
• Here is the full wiring diagram.

  

  This is everything put together in a nice box. The Arduino is visible bottom left.

  

  It is also worth making a few comments about the lights themselves. Each string of lights is 50 bulbs long. Bulb addresses are established during initialisation which using my firmware means they will be always numbered 0 ... 49 in order from the controller to the end (but they don't have to be). Each bulb can be individually controlled in terms of color and brightness. Brightness can also be controlled using a broadcast command to the mythical bulb 63. The bulb control protocol involves sending a pattern of low and high voltages down the data wire. Each high/low lasts for 10 microseconds and a full single bulb command takes 820 microseconds.

  The Compiler

  As mentioned above, the compiler is really more of an assembler as apart from some variable naming and jump measuring is basically substitutes some symbols for some numbers and emits a data file and a .h file for use in the firmware.

  The instruction set is very simple and explained in detail in the TidlyWiki HTML file included in the download, so I will cover the basics here only.

  The instructions are simple instructions relevant to a string of bulbs.

  <span id="ArticleContent">// set bulbs 0..9 red and bright
  SetColour
  0
  9
  red
  bright
  
  // set bulbs 10 to yellow and 19 to blue and graduate bulbs 11..18
  SetFadeColour
  10
  19
  yellow
  blue
  bright
  
  donothing
  1</span>

  Instructions are executed in cycles. All instructions are executed in the same cycle up until a donothing or a loop instruction is encountered.

  Some instructions run across multiple cycles. These commands display an effect over multiple cycles such as gradually changing colour or brightness or even shifting bulbs around.

  <span id="ArticleContent">// rotate bulbs 10..19 1 bulb to the right 50 times
  RotateBulbs
  10
  19
  1
  50
  1
  1</span>

  When controlling multiple strings of lights, you define a separate program for each string. When executing the cycle progression is centrally controlled, i.e., all strings complete their cycle 1 commands before the program proceeds to cycle 2. With careful authoring, you can then synchronise patterns across multiple strings of bulbs. Some of the pre-built examples included show how this is done. The secret is making sure you have the same number of cycles in each program.

  The compiler emits a .dat file for each program. These files are used by the firmware when it is running on the PC allowing you to test programs without having to recompile the firmware for each test.

  The compiler also emits a single .h file. This .h file contains up to 8 programs and is compiled into the firmware before deploying to the arudino. You only really need to do this at the very end when you are happy with how the program looks in the emulator.

  To compile, just pass in 1..8 program files to the compiler. The first passed is deemed program 0 in the .h file.

  The compiler also understands some common concepts and has keywords to represent them to make programs more readable. Keywords such as NOCOLOR, LASTBULB, YES, NO. These are all covered in the TiddlyWiki file.

  The Emulator

  The emulator emulates up to eight sets of lights. It does this by displaying bulbs as small circles on the screen which it then colours according to the instructions received from the firmware when it is running on the PC. The firmware talks to the emulator by using a TCP/IP connection. The address used is kept in the tcpip.txt file. It is by default the loopback adapter 127.0.0.1 but can be another machine on the network.

  The emulator can also get its input direct from an arduino connected over a Serial link. You can set this using the menu on the simulator and by including the appropriate #defines in the gloabl.h file. This slows down the program signficantly and should not be necessary except when trying to debug my firmware. When running in this mode you can even monitor the free memory on you ardunio.

  The emulator can lay out the bulbs to represent how they might look when strung up on your house. To do this, just edit the house.xlsm file and generate an output.xml file which can replace the GELightsSimulator.xml file with your own layout.

  The Firmware ... When Running on the PC

  The core of the firmware runs the same on the PC and the Arduino. However, when running on the PC, memory is obviously not such a concern and so the PC version does a lot more error checking and has extensive trace messaging which can help you iron out the kinks in your programs. This is done through the liberal use of #ifdef PC precompiler instructions in the code. It also logs everything to a LOG file which can be examined for timing issues.

  When running on the PC, the firmware will try to find a running instance of the emulator. If not found, it will start one up from the current folder. The firmware will also look for one or more .dat files passed to it as command line parameters and treat these as the program it is to run. If none are found, then any internally compiled program.h will be run.

  The Firmware ... When Running on the Arduino

  When running on the Arduino, the firmware runs in its stripped down version. Memory usage and consequently validation, etc. is minimal. Pin 11 of the arduino is the only error message you will get and this will just be set high and the program will stop if a problem is encountered. Many problems won't even be detected ... the program just won't work as expected. I strongly suggest testing on the PC first.

  Pin 13 on the arduino will alternately go high/low as the program progresses through the program cycles. This shows the program is running.

  I must admit I cheated a bit with getting the code setup and running in the Arduino SDK. In too much of a hurry to work out how it all worked, I just put all the classes into a library and the rest into a folder in the examples directory. I used a batch file to copy it from the folder I was doing PC development in into the Arduino compilers folders.

  Minimising Memory Usage

  When I first wrote the firmware, it would not even run a simple command sequence without running out of memory. Having gigabytes of RAM on a PC really makes you lazy when it comes to memory utilisation. In my first iteration everything was an integer, the program was kept in memory in a list all the time and commands like rotate bulbs would keep their own working copy of the bulbs.

  To shrink memory usage, the following changes were made:

  • Static data was moved into flash memory ... no point using ram for data that never changes. Arduino requires special declarations and access protocols to make this work.
  • Bytes and shorts were used instead of integers to minimise wasted bytes.
  • Unused bits in the bulb colour were used to hide status flags to avoid having to add bytes.
  • Instructions were constructed just in time and destroyed as soon as they were done.
  • Error checking and validation was largely removed.
  • Commands requiring temporary copies were minimised and allocated for just as long as required.
  • Pointer arithmatic was used instead of arrays of pointers when managing the bulbs. This saved lots of memory.

  This increased the code size and complexity and slows down the program slightly but the benefits in reduced memory usage made this worthwhile.

  The most memory hungry component of the program is tracking the state of the 50 bulbs in each string. Unfortunately this is an evil that cannot be avoided as it is impossible to query a physical bulbs state. Instructions often depend on their ability to understand the state of the bulbs before deciding the new state so these have to be tracked.

  Timing

  Interfacing the 16MHz Arudino with a device so sensitive to signal timing was a big challenge. The Color Effects is looking for a high/low 10 microseconds long. Anything more than a microsecond out and you may get spurious behaviour from the lights.

  In my first prototype on the Arudino pro, a sleep of 7 microseconds after each output was perfect. When I moved to the Arudino Mega, this had to be reduced to 4 microseconds and even then I would get the occasional wrong bulb lighting up ... a further refinement of 3 micrososeconds if the signal being sent was the same as the last signal had to be introduced to avoid accumulated signal delay issues.

  The only way to troubleshoot this was with a logic analyser.

  The 2012 version adds a parallel output option but this is not turned on by default as I ran into timing issues which i did not have time to resolve. These may be resolved in a fututre version.

  The Instruction Set

  Deciding what to include or exclude from the instruction set is always a challenge. The basic instructions are pretty obvious and in theory any pattern can be built just using the SetColour and DoNothing instructions but more complex patterns would be excessively long to develop and almost impossible to debug due to their size. Adding the more complex instructions makes some pretty cool effects possible without unweildy programs but each of these instructions are in themselves quite large and often memory expensive. Every time I came up with an idea for a pattern that I wanted to display, it was always a case of do I need a new instruction or is there a way to make the existing set do what I want. On balance, I think it is about right but I would not be surprised if you decided it was easier to add an instruction for a complex pattern rather than trying to make the existing instruction set do what you want.

  Adding a new instruction requires changes to the Compiler so the new instruction is recognised. This is usually just a matter of adding a row to the commands array and some keywords to the decodetable. You will also need to add a new class to the instructions.cpp file inheriting from the Instruction class and implement the required override functions. Basically the Construct method initialises the instruction and the Execute method executes a cycle of the instruction. You will need to add member variables to track the state of the instruction between cycles. The emulator should not need to change as it is oblivious to the instructions ... it just knows how to display bulbs.

  Conclusion

  What can I say programming these devices takes me back to my Z80 assembler programming days ... almost no memory, slow CPUs but amazing what could be done with them.

  History

  • v1.0 November 2011
  • v1.1 November 2011 - Fixed memory leak in Loop instruction
  • v1.2 December 2011 - Fixed 6N137 Pin labeling
  • v2.0 November 2012 - Major rewrite including:
    • Cross string variables added - see Manual.html in download file for details.
    • Parallel output of bulb messages added but disabled by default due to timing issues.
    • Subroutines added to minimise code size.
    • Memory utilisation improvements.
    • Raspberry Pi support added but is problematic due to lack of a RTOS Linux image which impacts signal timing issues.
    • Simulator messaging converted to TCP/IP and Serial Port allowing remote debugging including Arduino memory usage monitoring.
    • Code refactored to put most optional compilation settings in Global.h.
    • Code refactored to put most platform specific code in a platform class.

  Improvement Opportunities

  Fix parallel messaging.

  Fix Raspberry Pi implementation.