This is a showcase review for our sponsors at The Code Project. These reviews are intended to provide you with information on products and services that we consider useful and of value to developers.
If Only We'd Used ANTS Profiler Earlier... We Would Have Had a Shot at the $2 Million Cash Prize!
My name is Bryan Cattle. I'm a graduate in Electrical Engineering from Princeton University and a member of the DARPA Grand Challenge team. The DARPA Grand Challenge program is a series of races sponsored by the US government to encourage research into autonomous vehicles. Driverless cars attempt to traverse a course without human intervention. The top prize is $2 million.
Driverless Truck Running on 10,000 lines of C# Code
We were participants in the 2005 Grand Challenge, where our team advanced all the way to the national finals in Las Vegas. The final race was to navigate a 138-mile course through the Mojave desert.
Our cars used an asynchronous event-based code stack written from scratch by us. At the lowest level, it consisted of device drivers and low-level controllers for basic functions such as steering wheel position and GPS. Above that was obstacle detection. We were unique among the teams in the finals in that we used stereo vision, as opposed to scanning lasers, to detect and range obstacles. All in all, we wrote 10,000 lines of C# code to drive the cars.
Bug in the Obstacle-detection Code
In the finals, we ran for 9 miles before succumbing to a memory leak in the obstacle-detection code. Actually, most of our code is written in garbage-collected C#, so it wasn't a memory leak per se, but it wasn't until two weeks later that we discovered the true problem.
It was the closest thing to a memory leak that you can have in a "managed" language. C# manages your memory for you by watching the objects you create. When your code no longer maintains any reference to the object, it automatically gets flagged for deletion without the programmer needing to manually free the memory, as they would need to do in C or C++. Hence, in order to be allocating memory that is never freed in C#, you need to somehow be referencing an object in a way that you don't know about. Our actual bug occurred in the obstacle detection code. As we detect obstacles in each frame, we store the ones we detect. As the car moves, we call an update function on each of the obstacles that we know about, to update their position in relation to the car. Obviously, once we pass an obstacle, we don't need keep it in memory, so everything 10 feet behind the car got deleted.
Or so we thought. We kept noticing that the computer would begin to bog down after extended periods of driving. This problem was pernicious because it only showed up after 40 minutes to an hour of driving around and collecting obstacles. The computer performance would just gradually slow down until the car just simply stopped responding, usually with the gas pedal down, and would just drive off into the bush until we pulled the plug. We looked through the code on paper, literally line by line, and just couldn't for the life of us imagine what the problem was. It couldn't be the list of obstacles: right there was the line where the old obstacles got deleted. Sitting in a McDonald's the night before the competition, we still didn't know why the computer kept dying a slow death. Because we didn't know why this problem kept appearing at 40 minutes, we decided to set a timer. After 40 minutes, we would stop the car and reboot the computer to restore the performance.
On race day, we set the timer and off she went for a brilliant 9.8 mile drive. Unfortunately, our system was seeing and cataloging every bit of tumbleweed and scrub that it could find along the side of the road. Seeing far more obstacles than we'd ever seen in our controlled tests, the list blew up faster than expected and the computers died only 28 minutes in, ending our run.
ANTS Profiler Revealed the Problem
We had vacations coming up a few weeks after the race, so we left the cars in Vegas and returned, two weeks later, to investigate the problem. One of our team members downloaded the 14-day trial of ANTS Profiler and we ran it on our car's guidance code. We profiled the memory usage and saw the obstacle list blowing up. How could this be? We called "delete" on those old obstacles! To our amazement, it was only minutes before we realized that our list of detected obstacles was never getting garbage collected. Though we thought we had cleared all references to old entries in the list, because the objects were still registered as subscribers to an event, they were never getting deleted.
If Only We Had Used It Earlier...
We added one line of code to remove the event subscription and, over the next three days, we successfully ran the car for 300 miles through the Mojave desert.
ANTS Profiler helped us fix a problem in minutes that would have taken us weeks to track down. If only we'd thought of it before the competition, we would most likely have finished the entire race and had a chance at the top prize money.
Bryan Cattle, DARPA Grand Challenge team member at Princeton University
For more on the Princeton team, visit here. For more information on ANTS Profiler, visit the Red Gate website.
