Decoding raw data


#1

Hi there

I plan to install a couple of Pi boxes with the software soon. I am willing to try to archive the data in some storage (yet to be defined). I have seen some attempts that do modify (fa)dump1090 to access MySQL for example. My preference would be to be non intrusive and avoid any modification of the software. My thought was therefore to probably connect the 2 boxes using nc as explained on the github homepage and to connect my external component to store the data through port 30002. I have therefore a couple of questions:
1- Is it a good idea to go with the RAW data ? If yes how to decode it back to the physical data ?
2- Anyone aware of something already existing :wink: ?
3- Is there any easy way to encrypt the data (between the box) other than tunnelling using ssh ?

Thanks for your help

Guillaume


#2

Depending on exactly what info you need it may be simpler to use the SBS mode output on port 30003, this is essentially a CSV format containing decoded data.

I used this at one point to archive just position reports - basically nc + grep + gzip - the pi has enough CPU to deal with that. The compressed output was a few mb per day.

If you do go with raw output be aware that a) there are no timestamps and b) the volume can be high - 30kb/sec or more in busy areas. To decode it later you will either need to write your own mode-s decoder, or feed it back through e.g. dump1090 in --net-only mode.


#3

Looks better indeed.
Where can I get the proper documentation. I had a look to this page homepages.mcb.net/bones/SBS/ … t_Data.htm but there seems to be some subtil differences.

For example I got:

MSG,6,111,11111,AA9830,111111,2014/11/19,21:21:18.415,2014/11/19,21:21:18.381,CKS978 ,4414,0,0,0,0

where CKS978 is apparently the Callsign of AA9830 (HexIdent) but according to the link above MSG,3 is not supposed to return a callsign.

My interest goes to aircraft information i.e. Altitude, Callsign, Latitude, Longitude, GroundSpeed, Track, VerticalRate and IsOnGround

It seems that not all this will be in a single MSG … so slightly more complicated than a nc | grep | gzip :wink:

Also I am a bit puzzled by the 2 timestamps (generated vs logged) - I believed one was related to the A/C and the other to the Pi - but that does not make any sense because they might not be synchronized !

Cheers
Guillaume


#4

dump1090 does maintain some per-aircraft state and that’s why you might see more data than in theory is in the message.
(In particular it has to maintain state to do global position decoding, as you need at least 2 messages to do that)

The two timestamps in theory distinguish between “time the message was received” versus “time the output line was emitted”.
I believe early versions of the SBS firmware which originally generated this format would impose a 5 min delay on data.
It’s not so important with dump1090, the timestamps should always be close together.
There’s no timestamp info in the message data itself, it’s all on the receiver side.

In terms of docs, basically that page you already found is it - plus whatever you can find in the dump1090 code. There’s no official docs, this is a reverse-engineered format more than anything.
(+ only enough is implemented to satisfy the common cases where you’re feeding other software that just wants messages - e.g. dump1090 never generates anything except MSG messages, you’ll never see an ID or AIR message)