Introduction
While browsing my usual daily collection of news / gadget and hardware sites, I stumbled across a post for the Blitzortung network. I even posted a link to it on the Lounge pages on the day because I found it that fascinating.
It allows you to watch live lightning strike data via your browser in near real-time (delay of typically 3 to 5 seconds due to processing). And it is quite impressive when there are some storms sitting over Europe or USA and the amount of strikes is quite something else!
I don't know about you, but lightning storms have always fascinated me. I also have an interest in knowing where the lightning is occurring as it can impact my travel plans to/from the offshore platforms via helicopter! If there is active lighting in the area or on the flight path, the helicopters will not fly.
As a community project that anyone can participate in, the necessary hardware can be purchased via the site creators (approximately €200 at the time) and any participant who provides data can get access to all the other raw data.
I thought that this would make another interesting project and allow me to get my soldering iron out once more!
Lightning Detection Methodology
There is a really good write up on how the system works in detail and all the science behind it (see the project description at http://www.blitzortung.org/Webpages/index.php?lang=en&page=3), but suffice to say that the lightning generates very low frequency (3kHz to 30kHz) electromagnetic waves which can travel for thousands of kilometeres.
These waves can be detected by using either a E-Field (Electric) or H-Field (Magnetic element) antenna and the signals filtered and processed to determine a lightning strike.
The system uses the GPS network to get accurate timings of each strike detected by each detector and the detectors location in the network, then by using Time-Of-Arrival principles can give a very accurate location for each strike.
I would recommend however that the detailed explanation of it all is worth downloading, the pdf from the link above has all the pretty pictures and explanations.
Hardware Build
I elected to purchase an H-Field complete system, as this uses small ferrite rods (200mm x 10mm) as they have more flexibile requirements on positioning than using larger co-axial loop antenna or E-Field antennas.
The kit came as a collection of parts as can be seen in the photo below.
Within the kit there are;
- 2x Ferrite Rod Antenna
- 2x custom PCBs (one controller and one amplifier)
- 1x (122x132 pixel) Graphic LCD
- 1x STM32F4 Discovery Board (Arm Cortex M4)
- 1x ATMega 8-16 microcontroller IC
- 1x Ethernet controller ENC28J60 IC
- 1x GlobalTop PA6H GPS Module
- 1x GPS Antenna
- 8x Various Surface Mount Op-Amps
- Numerous LEDs, Resistors, Capacitors, Diodes, Terminals, RJ45 Sockets, Buzzer, Switches etc.
The first thing to do was start with the smallest components and build up to the largest. The hardest parts to attach where the surface mount operational amplifier integrated circuits. I found the easiest way to attach these to the pads on the PCB was to pre-solder the pad in one corner, then slide the IC into position while applying heat and gently square up to align the other pads. Solder the opposite corner and then fill in the remaining pads. These can seen on the amplifier board below;
The second hardest component was the GPS as it didn't really have any terminals as such. It featured small semi-cular mounts for dropping into a socket so had to really rely on getting a good heat transfer and capillary action to achieve a healthy solder joint onto the tiny surface of the contact point. I used the same method as the SMDs, but really could have done with a smaller tipped soldering iron. It maybe isnt pretty, but at least it worked!
I also had noticed at somepoint during the reading before the kit arrived to remember and solder the large capacitors onto the board on their side(i.e. folded over) as the discovery board had to mount down onto the main controller board over the top of them.
The only other couple of minor problems were;
1) one of the SMD pads came off during soldering, having checked out the data sheet for the chip, found that this was a no-connection.
2) some of the resistors supplied were 47Ohms instead of the specification of 49.9Ohms, having checked with the website all was ok with these components.
After a few hours of building up both boards, I didn't have time to do any testing before calling it a day and having to go and pack my bag and get some sleep before heading back to work for a couple of weeks. The next step would just have to wait until I get home! The completed boards;
On return, it was time to get out the instructions and review the initial hardware tests.
The amplifier board was a relatively easy test. This comprised of applying a 5V source to the mini-usb socket and the red power LED should come on and the 4 gain indicator LEDS should blink 4 times. The board then would display the current amplifier gain using the 4 LEDs in a pattern. Adjusting the gain pot would adjust the LED pattern. Success #1!
The video of this initial test can be seen here: http://youtu.be/4HK5K_8OxRU
The controller board had an even easier test of simply plugging in the 5V and checking a couple of LEDs came on and the GPS LED started to flash. The second test involved inserting the discovery board and powering up the system again and checking that the in-built default program ran the LEDS on the discovery board.
The video of this default program test of the discovery board can be seen here: http://youtu.be/Otu352H0KJ8
After downloading the ST-Link software and grabbing a copy of the latest firmware, it was a really easy job of uploading this to the discovery board.
After a power up the GPS started searching and locked onto the satellites and the LCD displayed some information on network IP Address(assigned from my home router DHCP) and also the satellite time (UTC)/fix count/ visible count and diagnostic information like CPU temperature and voltage and also the used amplifier gains amplied in automatic mode. With an ethernet cable connecting the controller and amplifier boards the amplifier board powered up successfully and the gain LEDs were not illuminated as this was being controlled automatically by the controller board.
The board also displayed a CPU-ID for registering with your account details on Blitzortung. This effectively linked this station to your account. After requesting an account from Blitzortung, it was a simple case of logging in and adding this ID to you user settings.
Antenna Build
The ferrite rods need to be enclosed in a plastic conduit (if you want to protect/weatherproof) and subject to interference levels in area, possibly need to be shielded. The rods are mounted in the horizontal plane at 90degrees to each other. This is because we are concentrating on Cloud to Ground strikes, and the electric field is generated in parallel to the current flow and the magnetic field is at 90' to this.
I elected just to build the antenna with the shielding in place from the start, rather than have to potentially go revisiting later due to high interference levels. The ferrite rods were mounted inside 20mm plastic conduit. The shielding was provided by using adhesive aluminium foil. The important thing with the shielding is that there must be a gap between the shield down the length of conduit, otherwise it turns this into a mini single wind transformer and stops things working. You can see in the image below the gap running down the conduit.
To provide the link between the shielding and the ground, I attempted to solder a wire onto the adhesive foil, but this was a failure, couldn't get it to work. In the end I used some coaxial wire (from standard TV aerial cable) and having removed the outer sheath, shielding and inner insulator, I placed this down the length of the shielding and held in place at both ends with tightly wrapped self-amalgamating tape. Having tested I had a good connection between the shields and the ground wires I then applied a heat shrink tube and heat shrink end caps to seal the whole lot up and also provide additional pressure between the shield connecting wires and the shield.
The location where I was mounting the antenna was visible from one of the roof windows, I therefore wanted to make sure I could see the LEDs. I purchased two clear lid weatherproof enclosures to mount both the PCBs in. These were purchased from a retailer on E-Bay and were Gewiss GW44426 and Gewiss GW44427. To provide the seals for the cable entries, I used some standard plastic 12mm and 20mm Waterproof cable stuffing glands, again purchased on E-Bay.
Using some white plastic rod from the local DIY shop I made some PCB stand-offs and a hot glue gun to secure to the boxes. The height of the stand-offs made sure I could see the LEDs by bringing the PCB forward to the clear area. You can see a typical standoff below;
For the antenna you can now see the LEDs mounted in the clear area of the box lid in the picture below;
With both PCBs now nicely secured in weatherproof enclosure, it was time to move onto locating them in the final positions. The controller board and antenna boards in their respective enclosures;
Installing the Boards / Antenna
I had an old aerial mount left on my chimney stack and felt that would be a good place to install. This was furthest away from any electrical noise in my loft (Solar inverter, Computers, Network equipment, shower pumps, central heating pumps etc.).
On the antenna box I orginally had used some Araldite® glue to bond some of the solid plastic rods to the back of the enclosure and then used a jubilee clip to secure onto the old aerial mount. I wanted to do this so I didn't need to put any holes into the enclusure that were not properly weather tight. Sadly, this approach failed as the bonds didn't take to the enclosure, one pulled off during tightening, and I didn't trust the other one would last in high winds. I reverted to plan B and used an aerial u-bolt and drilled two holes in the back of the enclosure and then made a waterproofing seal using some building waterproofing membrane I had lying in my garage.
The box was mounted and cables ran back into the house through the existing roof penetration that is in place for the solar hot water system pipes. You can see the ferrite rod conduits are now heat shrink wrapped and also the shielding wires coming off each arm.
The controller box was mounted inside my hot water cupboard, where the cable from the antenna was hooked up, as well as the GPS antenna which was fed out the same roof penetration and sat secured onto the solar hot water system pipes insulation, Finally a USB power feed was taken from a nearby socket in the cupboard and a network cable back to the main switch was run in to complete the install.
The Software Side
The controller provides a web interface for configuring things and I had already set up the networking etc. as part of the intial hardware tests earlier.
Connecting to the controllers web interface via the browser and switching to the Signals tab you can see the data that is coming in from the antenna. By activating the Noise floor check box you can see all the background noise and occassional spikes of signal data coming in. You can also see the Thresholds and Noise Floor numbers dynamically adjusting the horizontal dashed lines.
Real Strike Data
Now the system was stable, collecting and processing data, I opened both the live signal view and the main live maps on the Blitzortung website and waited.....patiently! Finger was poised over the print screen button ready to capture that first strike that showed my station on the map and the corresponding spike on the data. Fortunately there were a few storms rumbling away over Europe, so it shouldn't be that long I hoped. Sure enough after only a couple of minutes...Bang! Got You, the keyboard was hammered and I got that moment perfectly, a strike in the North of Denmark.
In the picture above you can see my station (Red Arrow pointing). The white circle actually moves out to represent the sound wave propogation from the strike. The lines can be either Green or Blue. The green lines go to the participating station that detected the strike. The blue lines are the 12 closest stations that were used to calculate the postion of the strike.
In the picture above you can also see there are other Red/Orange/YellowWhite markers, these are previous recent strikes and the legend at the top left shows the 'minutes' for each colour, and graph at the bottom left shows the strike count for each colour.
More Data To Explore
Below are some links to the various data anyone can see;
Blizortung - The main Blitzortung web page
Lightning Maps - Another website using the Blitzortung Data
Station 1147 - My Aberdeen, UK Station Data Summary (on Lightning Maps) (Updated every 15 minutes from Blitzortung)
Station 1365 - My Doha, Qatar Station Data Summary (on Lightning Maps) (Updated every 15 minutes from Blitzortung)
The station data page at the link above also includes some detailed information relating to the specific installation. These include statistics such as scroke counts and ratios to network, but also graphs relating to the data.
The latest sroke data captured by the station and broadcast to the system is displayed as is the frequency analysis of the stroke. These two graphs are shown below:
One of the other useful graphs displayed is the analysis of the stroke count and the distance from the station of the various strokes.
Long Time Station Data
The Lightning Maps site also provides some interesting views and statistics on the long time data as they call it, i.e. pretty much the stats from the start of the station going live.
My Aberdeen Station has detected 2,263,748 strokes so far (364 day station uptime), the closest being 1.1Km and the furthest being 9399km!
My Doha Station has detected 109,606 strokes so far (46 day station uptime), the closest being 312Km and the furthest being 7896Km.
Also, the system provides distances between all the stations using the GPS system, so I now know it is 5503Km between my two stations......not that I really need to know that!
Particpant Only Data
As I mentioned earlier, particpants who provide station data can access all the raw data from the Blitzortung system. There are conditions for use of this data, i,e. no commercial services and no proxying the data of to others.
As a participant you can download the raw logs quite easily. It is case of simply making a http call to the server in the format
http://<Username:Password>@<TheServer>.blitzortung.org/<Region>/Protected/Strokes/<YYYY>/<MM>/<DD>/<HH>/<mm>.log
Each file contains the 10 minute block of strike data. If I enter the date/time for the start of the data block for the strike I capured above (on the map image), you can see the raw text file that is returned, complete with the entry;
2014-08-24 13:52:01.846953381 pos;57.315008;11.488181;0 str;0.00 dev;4564 sta;11;134;545,1012,591,740,589,813,1092,590,585,781,
702,466,869,739,767,598,979,588,741,592,
30,130,584,594,1014,830,641,246,1109,997,
986,469,785,944,125,1121,1086,43,891,111,
699,715,969,1032,1052,893,896,1028,688,526,
859,370,1107,627,535,752,711,764,866,735,
833,889,845,779,994,694,1049,826,693,847,
879,907,283,1043,685,304,573,768,674,912,
1147,1063,414,733,988,1011,894,1048,538,777,
1116,51,957,1075,1095,703,687,774,856,1006,
434,829,784,811,76,843,878,824,923,198,
922,959,890,780,747,546,766,698,810,193,
838,786,933,1071,938,887,846,1083,1131,559,
800,839,1104,678
This information can be parsed out as: Date Time Position Strength(subject to availability) Deviation(nanoseconds) and the Stations List (first 11 used in Calcs, 134 detected strikes, 1147=mine)
There is also the Station Participant list available which details locations, board types, amplifier settings etc.
Both the station list and the raw log files are updated every minute.
Station Private Data
Logging into LightningMaps site also provides the station owner with extended data. This includes various stroke analysis over longer time and some key metrics such as those shown below;
- Record longtime data since:2014-08-19 15:19:11 UTC
- Strokes detected:149,663
- Station active:85.4 days
- Station inactive:1.5 days (0.3 hours without GPS)
- Max stroke count per hour:938
- Minimum distance:86.5km
- Maximum distance:3,821.5km
- Signals detected:1,655,781
- Stroke ratio:12.2%
- Locating ratio:9.0%
- Max signal count per hour:8,737
Network Data;
- Begin of data collection:2011-05-06 10:10:52 UTC
- Lightning data imports:1,239,290
- Traffic to Blitzortung.org:3,129.3 MB/Day
- Traffic - Strokes:9.2 MB/Day
- Traffic - Stations:4.5 MB/Day
- Traffic - Total:874,943 MB
Controller Web Interface - More Info
The controller web interface also provides more information on the system some of the key parts of the interface are;
Firmware and Hardware Versions as well as CPU Temperature, Voltages and Processor Load
Details about the amplifier communications as well as current trigger level settings and gain values used in the operational amplifiers. Details of the Noise floor levels can also be seen.
There is quite a bit of GPS information including the actual satellites in view and their signal levels as well as the UTC time and current/smoothed positions. Taking the smoothed position and popping it into Google Maps and then zooming into the Earth view, it showed that the GPS was about 4 feet off where the actual antenna was located.
The tracker details the number of received signals by the network and well as the number of signals generated, not all signals are considered good and sent to the network.
Other Views
Below is a recording of 10seconds of lightning activity of North America;
Lightning Density Map
The system can also generate a global overview of the lightning density, this gives a pretty good overvall view of where the current hot spots of weather activity are happening, below is a snapshot of map taken.
Conclusions
At interesting community project. No doubt about it! Surprised at the boards coming up first time, so my soldering skills can't be that bad.
Will have to see how the unit performs over time and what sort of data can be pulled. I am thinking about how I can monitor my strike detection and possibly see the over time the furthest strike I detect, currently I am just short of 9399km for the furthest!
I am sure the guys at Blitzortung will continue to update the firmware for the controllers as well as the website with some APIs or more functionality.
However, more participants are needed, with only ~1200 stations present, many more are needed to improve the coverage and accuracy of the system globally.
So, the answer to the question........does lightning strike twice? Well, maybe you will just have to join in to find out :)
History
- 28th August 2015 - Add Qatar Detector, Density Map, Update Stats
- 24th August 2014 - First Published.
