Get data from dump1090 via command line/python

FlightAware ADS-B Flight Tracking
1

Hi…me again,

I’m trying to get the number of aircraft with positions from dump1090 via the command line (or even better python). I’ve done some searches both here and github but I can’t seem to find where dump1090 is installed, let alone talk to it. I’m running the v3.5.3 sd card image in a Pi2. When I ssh in and list directories I get nothing - any tips on where I should be looking for the dump1090 install or how to get data from it ?

The endgame is a to drive some ws2812b Leds (which are already working) to give a visualisation of the volume of traffic.

Thanks for reading, you have a nice helpful community here.

2

This is not a direct solution, but may help

Exploring Port 30003 and 30106

3

Dump1090 is installed in the /usr/bin directory.
command to find program is “which dump1090-fa”

If you just want to have the plane count you can read the json files made by dump1090. When dump1090 is running it will output json files to /var/run/dump1090-fa . You want to read the aircraft.json file or stats.json file depending on the exact data you want to find.

The output ports are another way. You will need to parse them and this way is a bit more difficult. The parser for the output ports can be on any networked computer while the json parser will need to run directly on the raspberry pi.

2 Likes
4

Dump1090 MySQL Database Script with Alert and Filter

GitHub - TomMuc1/Dump1090-MySQL-Alert-Filter: a simple php script that makes use of Dump1090-mutability aircraft.json and filters/writes to MySQL

2 Likes
5

@Spacer
The two threads created by @TomMuc (see his post above) will greatly heip you in achieving your target.

6

Thanks for all of the replies - I have some reading to do about the SQL scripts. One thought - the skyview is already calculating and displaying the data I want - no. of aircraft with positions. Would it be possible to querry the skyview to get the data ?

David.baker - thanks for the directory listings. I was being slow and forgot I was in the home directory when I ssh’d in. All I had to do was jump up a couple of levels and the familiar pi directory structure was there.

7 
pi@raspberrypi:~ $ cat /var/run/dump1090-fa/aircraft.json
{ "now" : 1529421688.9,
  "messages" : 12207076,
  "aircraft" : [
    {"hex":"ac8215","altitude":20100,"mlat":[],"tisb":[],"messages":4,"seen":1.6,"rssi":-28.4},
    {"hex":"a2be96","altitude":35475,"mlat":[],"tisb":[],"messages":12,"seen":2.4,"rssi":-26.9},
    {"hex":"ad9edf","altitude":32000,"mlat":[],"tisb":[],"messages":13,"seen":13.1,"rssi":-27.0},
    {"hex":"ab77d5","altitude":37000,"mlat":[],"tisb":[],"messages":5,"seen":15.8,"rssi":-29.4},
    {"hex":"c03b0b","lat":41.156321,"lon":-78.266976,"nucp":0,"seen_pos":4.7,"altitude":30000,"vert_rate":64,"track":350,"speed":422,"mlat":["lat","lon","track","speed","vert_rate"],"tisb":[],"messages":66,"seen":0.2,"rssi":-26.0},

......
.......

.

.

pi@raspberrypi:~ $ cat /var/run/dump1090-fa/stats.json
{
"latest":{"start":1529421840.2,"end":1529421840.2,"local":{"samples_processed":0,"samples_dropped":0,"modeac":0,"modes":0,"bad":0,"unknown_icao":0,"accepted":[0],"strong_signals":0},"remote":{"modeac":0,"modes":0,"bad":0,"unknown_icao":0,"accepted":[0]},"cpr":{"surface":0,"airborne":0,"global_ok":0,"global_bad":0,"global_range":0,"global_speed":0,"global_skipped":0,"local_ok":0,"local_aircraft_relative":0,"local_receiver_relative":0,"local_skipped":0,"local_range":0,"local_speed":0,"filtered":0},"altitude_suppressed":0,"cpu":{"demod":0,"reader":0,"background":0},"tracks":{"all":0,"single_message":0},"messages":0},
"last1min":{"start":1529421780.2,"end":1529421840.2,"local":{"samples_processed":144048128,"samples_dropped":0,"modeac":0,"modes":892462,"bad":552894,"unknown_icao":330136,"accepted":[9432],"signal":-11.9,"noise":-30.1,"peak_signal":-1.2,"strong_signals":476},"remote":{"modeac":0,"modes":534,"bad":0,"unknown_icao":0,"accepted":[534]},"cpr":{"surface":0,"airborne":970,"global_ok":882,"global_bad":0,"global_range":0,"global_speed":0,"global_skipped":2,"local_ok":71,"local_aircraft_relative":0,"local_receiver_relative":0,"local_skipped":17,"local_range":0,"local_speed":1,"filtered":0},"altitude_suppressed":0,"cpu":{"demod":11391,"reader":5052,"background":860},"tracks":{"all":6,"single_message":0},"messages":9966},
"last5min":{"start":1529421540.2,"end":1529421840.2,"local":{"samples_processed":719978496,"samples_dropped":0,"modeac":0,"modes":4427481,"bad":2735665,"unknown_icao":1636618,"accepted":[55198],"signal":-10.5,"noise":-29.1,"peak_signal":-0.8,"strong_signals":3347},"remote":{"modeac":0,"modes":2583,"bad":0,"unknown_icao":0,"accepted":[2583]},"cpr":{"surface":63,"airborne":5528,"global_ok":5256,"global_bad":0,"global_range":0,"global_speed":0,"global_skipped":6,"local_ok":252,"local_aircraft_relative":0,"local_receiver_relative":0,"local_skipped":83,"local_range":0,"local_speed":2,"filtered":0},"altitude_suppressed":0,"cpu":{"demod":56109,"reader":25377,"background":4199},"tracks":{"all":31,"single_message":0},"messages":57781},
"last15min":{"start":1529420940.2,"end":1529421840.2,"local":{"samples_processed":2160066560,"samples_dropped":0,"modeac":0,"modes":13382700,"bad":8269698,"unknown_icao":4932468,"accepted":[180534],"signal":-11.0,"noise":-29.1,"peak_signal":-0.6,"strong_signals":8785},"remote":{"modeac":0,"modes":6807,"bad":0,"unknown_icao":0,"accepted":[6807]},"cpr":{"surface":64,"airborne":20011,"global_ok":18971,"global_bad":2,"global_range":2,"global_speed":0,"global_skipped":14,"local_ok":833,"local_aircraft_relative":0,"local_receiver_relative":0,"local_skipped":269,"local_range":0,"local_speed":4,"filtered":0},"altitude_suppressed":0,"cpu":{"demod":171233,"reader":76609,"background":12557},"tracks":{"all":86,"single_message":5},"messages":187341},
"total":{"start":1529336640.1,"end":1529421840.2,"local":{"samples_processed":204453969920,"samples_dropped":0,"modeac":0,"modes":1279207996,"bad":788259614,"unknown_icao":479108158,"accepted":[11840224],"signal":-10.2,"noise":-30.1,"peak_signal":-0.6,"strong_signals":620251},"remote":{"modeac":0,"modes":394473,"bad":0,"unknown_icao":0,"accepted":[394473]},"cpr":{"surface":333,"airborne":1227665,"global_ok":1154388,"global_bad":343,"global_range":148,"global_speed":4,"global_skipped":1057,"local_ok":52705,"local_aircraft_relative":0,"local_receiver_relative":0,"local_skipped":20562,"local_range":6,"local_speed":214,"filtered":0},"altitude_suppressed":0,"cpu":{"demod":15820956,"reader":7149473,"background":1088648},"tracks":{"all":6767,"single_message":646},"messages":12234697}
.......
.......
8

Skyview reads the aircraft.json file every second to displays it on the map. You can make a python program to read the JSON file every second and count the number of aircrafts in the JSON. This is actually fairly easy to do in a few lines of code.

Lots of examples online on how to open and read a JSON file.

command to output aircraft.json to console
cat /var/run/dump1090-fa/aircraft.json

If you want full SQL search then Tom’s solution will work. MySQL is kind of overkill for your application but good if you need the data for something else. This solution connects to the data feed from dump1090 and stores it in a SQL database. You can then query the database for your information.

You need to know SQL and how to connect to the database and runs queries on it.
Also fairly easy to do but requires a lot of resources to run the database.

Read the data directly from the 30003 output. You can just count the number of unique aircraft in the feed.
Need to know how to open a local port for reading streaming data.

command to open port and stream data to console.
“nc localhost 30003”

9

if you are just looking for some aggregated data you could also use the second simpler script that works optionally without database:
Daily Log/E-Mail Aircraft Report directly on your RaspberryPi
GitHub - TomMuc1/Dump1090-Daily-Aircraft-Report: short script that reports daily aircrafts from dump1090

2 Likes
10

@Spacer –

I’m curious if you got your data viz (with LEDs) project up and running. I was thinking of doing something similar and welcome any insights you had…

Thanks!!

11

Did you ever get your data to trigger LEDs? I’m looking to do the same but trigger a relay to close an HVAC damper when aircraft are in the area. Let me know how your project came out when you have time.
-Derrick Perrin

12

I realize none of this data output just shows a simple plane quantity count, apparently.

But…

the aircraft.json file has each current plane shown on 1 single line. We can just count the lines very quickly, -efficient, -quite low resource load ( wc -l <path/filename> ).

But, the output also includes the given filename(s) so just grab the first word only and ignore the rest ( awk '{print $1}' ).

In that file, the first 3 lines, and the last 2, are json scaffolding, so just use subtraction after we get the total line count of that file ( count - 5 ).

We took the total output count of that file and use the output arg to pass into the next command so we get the math calculation ( xargs )

wc -l /var/run/dump1090-fa/aircraft.json | awk '{print $1}' | xargs -n 1 bash -c 'echo $(($1 - 5))' args

That all runs in a blazing split second.

I tried to get the subtraction right out of that awk section but couldn’t figure it out.

I suspect that line could be stuck into a script file and run from whatever environment, and python I suppose with whatever command will fire off that shell script.

13 
jq < /run/dump1090-fa/aircraft.json '.aircraft | length'

In python it’s much simpler to just load the json and check the length of the aircraft array.
No need to run shell commands from python for this.

2 Likes
14

Oh! that’s nice! jq, -that’s a great feature.

I appreciate your comment, and have seen years of your offerings on the web, so wanted to say a personal thank you for what you do.

I’m lost with python, and can only claw my way through bash stuff as needed.

I did run a test with that:

zcot@zcot-OP701:~$ time (jq < /run/dump1090-fa/aircraft.json '.aircraft | length')
124

real	0m0.032s
user	0m0.028s
sys	0m0.004s
zcot@zcot-OP701:~$ time (wc -l /var/run/dump1090-fa/aircraft.json | awk '{print $1}' | xargs -n 1 bash -c 'echo $(($1 - 5))' args)
124

real	0m0.005s
user	0m0.003s
sys	0m0.007s
zcot@zcot-OP701:~$ time (wc -l /var/run/dump1090-fa/aircraft.json | awk '{print $1}')
129

real	0m0.004s
user	0m0.001s
sys	0m0.005s

I saw the meaning of the 3 different timing explanations: unix - What do 'real', 'user' and 'sys' mean in the output of time(1)? - Stack Overflow

I’m not sure if the embedded bash -c is included with that timing, for the subtraction math, but I would guess it is since the main script is not returning until after that anyway. And the xargs subtraction method seems clunky to me anyway. It could be removed and then just do the math on the other end as needed.

But, all those methods are screaming fast anyway, even in different leagues. String together a bunch of those and the dot makes a proportional difference.

The OP didn’t mention where the data collection, then production, is occurring. In my case, when I found this aged thread from a search, I have the dump on one system, and the processing is needed on a different point of the network. In my case the “server” is in a closet so I need the data elsewhere. I wanted to share it.

15

The issue with this really is that there is no guarantee every aircraft is on its own line.
Actually parsing the json is gonna be more CPU but it’s also gonna be correct even if dump1090-fa changes the formatting of that file.

Python really isn’t that hard, also google helps with many many python issues you might have.

import json

with open("/run/dump1090-fa/aircraft.json") as file:
    data = json.loads(file)
    
aclist = data.aircraft
print(len(aclist))

for ac in aclist:
    print(f"hex: {ac.hex} callsign: {ac.flight}")

Give that a try.
No guarantees for the above, didn’t test it :slight_smile:

16

Well that’s all kinds of broken :slight_smile:

But easy enough to fix:

import json

with open("/run/dump1090-fa/aircraft.json") as file:
    data = json.load(file)

aclist = data.get('aircraft')
print(f"Number of aircraft: {len(aclist)}")

for ac in aclist:
    icao = ac.get('hex')
    callsign = ac.get('flight')
    print(f"hex: {icao} callsign: {callsign}")
2 Likes
18

wiedehopf, that’s great! certainly an impressive way to deal with some stuff, of course doing anything more complex. That’s a complex set of movements and only using a minimal amount of lines. I like that. Obviously doing that on bash is going to require an actual investment.

To the OP, in my case at least, just looking for the 1 number only, and I end up using using bash on the other end. -well piping into a conky config. -dirty network transfer with nc, cat it off the file of the single number.

I’ll be digging into various topics in the future, antenna tuning, multi-feed, it seems like MLAT is not working for me. Is there any IRC, or shudders Discord, or Matrix groups, or something?

Thank you for your input.

19

That a VM?

USB extension dropping voltage and the SDR not getting enough power is another contender.

FA mlat also needs your station connected to your account and the location set on the fa stats page.