Recording Data To MySQL

Using Debian 11 on Intel with an Airspy Mini

Installed piaware, dump1090-fa, airspy_adsp using scripts. Everything works. Thanks @wiedehopf!

Installed mysql and php using apt-get. All is well.

Installed scripts for writing contents of json to mysql. Got them here:–GitHub - TomMuc1/Dump1090-MySQL-Alert-Filter: a simple php script that makes use of Dump1090-mutability aircraft.json and filters/writes to MySQL Thanks @TomMuc1

Created the DB and the table using the provided script. Modified radar.php per my needs and understanding. All almost works. I am seeing data n my json file when I inspect it directly. It works. The radar.php setting is pointing at it. User ID, Password and other DB settings are correct. No errors being thrown when I run the code directly: php radar.php

The script runs, looks like it checks the json once per second, but no data goes to the database. I think I have turned off all filtering. Not interested in alerts. Just want the data saved for later analysis.

Any ideas of what I might be missing? Thanks for thinking about this and any suggestions.

Here is some output from the terminal when I run radar.php using the php command:

upt(us): 0001536 - 0.998073 loops/s avg - since 0 days 00:08:39 - run 518 @ 0,1 msg/s → 000 dataset(s) =>
upt(us): 0001596 - 0.998077 loops/s avg - since 0 days 00:08:40 - run 519 @ 0,1 msg/s → 000 dataset(s) =>
upt(us): 0001776 - 0.998081 loops/s avg - since 0 days 00:08:41 - run 520 @ 0,1 msg/s → 000 dataset(s) =>
upt(us): 0001594 - 0.998084 loops/s avg - since 0 days 00:08:42 - run 521 @ 0,1 msg/s → 000 dataset(s) =>

And here is the relevant part of my radar.php file:

#!/usr/bin/php

<?php #phpinfo(); #var_dump(ini_get_all()); #ini_set('error_reporting', E_ALL); // below a sample create statement for database table // CREATE TABLE aircrafts (id INT NOT NULL AUTO_INCREMENT, message_date VARCHAR(100), now VARCHAR(100), hex VARCHAR(100), flight VARCHAR(100), distance VARCHAR(100), altitude VARCHAR(100), lat VARCHAR(100), lon VARCHAR(100> // set the rectangle and altitude to store aircraft-data in database - if your lon is negative be aware to use the right values for max and min $user_set_array['max_lat'] = 37.384796; $user_set_array['min_lat'] = 37.563922; $user_set_array['max_alt'] = 50000; $user_set_array['max_lon'] = -122.427054; $user_set_array['min_lon'] = -122.481594; // set the rectangle and altitude to send alert message - if your lon is negative be aware to use the right values for max and min $user_set_array['alert_max_lat'] = 37.384796; $user_set_array['alert_min_lat'] = 37.563922;; $user_set_array['alert_max_alt'] = -5000; $user_set_array['alert_max_lon'] = -122.427054; $user_set_array['alert_min_lon'] = -122.481594; // set lookup-interval default is 1 (must be integer between 1 - 900) this is the frequency the script runs and writes to database or looks for alerts $user_set_array['sleep'] = 1; // set your google email-address for alert-messages - if you want to use mailer.php instead gmail set method 'gmail' to 'webmail' or 'pushover' and a key according to mailer.php $user_set_array['alert_method'] = 'gmail'; $user_set_array['email_address'] = 'YOUR_EMAIL@gmail.com'; $user_set_array['secret_email_key'] = 'YOUR_USER_KEY'; // set parameters for database connection $user_set_array['db_name'] = 'adsb'; $user_set_array['db_host'] = '127.0.0.1'; $user_set_array['db_user'] = 'MYDB'; $user_set_array['db_pass'] = 'NotVerySecure'; // set path to aircraft.json file $user_set_array['url_json'] = '/run/dump1090-fa/'; // set path to your mailer.php file $user_set_array['url_mailer'] = '/home/skygizmo/hmbatc/mailer.php'; // set the absolute limit of alert-messages (default is 1000) this script is allowed to send over its whole runtime $user_set_array['mailer_limit'] = 1000; // set aircraft suspend time (default is 900) - change only if needed - time in seconds an aicraft is suspended from alert-messages after sending an alert-message for this aircraft $user_set_array['aircraft_suspend_time'] = 900; // set this to true if you want alerts and/or database writes from those aircrafts matching your hex_code_array.txt or flight_code_array.txt files within limited area or whole site-range, with or without wildcards $user_set_array['filter_mode_alert'] = false; $user_set_array['filter_mode_alert_limited'] = false; //Tried next three both false and true $user_set_array['filter_mode_database'] = true; $user_set_array['filter_mode_database_limited'] = true; $user_set_array['filter_mode_wildcard'] = true; // set path to your hex_code_array.txt and flight_code_array.txt files $user_set_array['hex_file_path'] = '/home/skygizmo/hmbatc/hex_code_array.txt'; $user_set_array['flight_file_path'] = '/home/skygizmo/hmbatc/flight_code_array.txt'; // set your timezone see http://php.net/manual/en/timezones.php $user_set_array['time_zone'] = 'America/Los_Angeles';

https://github.com/TomMuc1/Dump1090-MySQL-Alert-Filter/pull/4/commits/cb0edfc30d267392d253baa462a19cf48694105e

Thank you for quick response but it looks like the radar.php code I am using has your fix in it already. I assume the fix you have supplied “maps” the fields from the json file to the aircrafts table. I can see there are more fields/columns in the table than the json. As all the columns other than the ID are allowed to be null, I figure who cares. I can clean up table structure later. But no data is going into my db. Lots of data in the json. Has something else changed?

I am trying to compare against the fix you have sent and the code in my radar.php. Kind of get what is going on. But confused. Thanks again for any help. Three chunks of settings and code are included below.

Here is a list of fields from aircraft.json. I have stripped out the data:

hex
type
flight
alt_baro
alt_geom
gs
ias
tas
mach
wd
ws
oat
tat
track
roll
mag_heading

And here is a list of the columns from aircrafts table:
id
message_date
now
hex
flight
distance
altitude
lat
lon
track
speed
vert_rate
seen_pos
seen
rssi
messages
category
squawk
nucp
mlat
tisb
rec_msg_sec

And here is the insert code from my radar.php:

            isset($row['flight']) ? $ac_flight = trim($row['flight']) : $ac_flight = '';
            isset($row['altitude']) ? $ac_altitude = $row['altitude'] : $ac_altitude = '';
            isset($row['alt_baro']) ? $ac_altitude = $row['alt_baro'] : true;
            isset($row['lat']) ? $ac_lat = $row['lat'] : $ac_lat = '';
            isset($row['lon']) ? $ac_lon = $row['lon'] : $ac_lon = '';
            isset($row['track']) ? $ac_track = $row['track'] : $ac_track = '';
            isset($row['speed']) ? $ac_speed = $row['speed'] : $ac_speed = '';
            isset($row['gs']) ? $ac_speed = $row['gs'] : true;
            isset($row['vert_rate']) ? $ac_vert_rate = $row['vert_rate'] : $ac_vert_rate = '';
            isset($row['baro_rate']) ? $ac_vert_rate = $row['baro_rate'] : true;
            isset($row['seen_pos']) ? $ac_seen_pos = $row['seen_pos'] : $ac_seen_pos = '';
            isset($row['seen']) ? $ac_seen = $row['seen'] : $ac_seen = '';
            isset($row['rssi']) ? $ac_rssi = $row['rssi'] : $ac_rssi = '';

Most common issue is bad geographic area or bad altitude range.
If i remember correctly.

Yes, it is. Most of the times it’s a misunderstanding in the given parameter.

But here is another option you can use:

Logging dump1090-fa to local database – Cactus Projects

There are some minor mistakes in the script (e.g. the name of the table is not consistent and needs to be fixed, additionally some dependencies need to be installed manually).

But finally it’s working and writes the results in a SQLiteDB:
I do not have programming skills, but i am sure it can be modified to get it with MySQL

Benefit is that it can count an aircraft twice in case the aircraft is the same but on a different flight (e.g. one inbound, the other outbound) for the same day.

I’ll give Cacti Guy’s script a try. Looks pretty straight forward. Thanks for the heads up.

What is meant by bad geographic or altitude range? The settings in radar.php? Does it always filter based on the geo cube I have set up? If so, that would explain why the code selects nothing. Maybe I have them all backwards?

What would be the right settings for the Western Hemisphere North of the Equator, Below Heaven? :cowboy_hat_face:

I’ve not used this script myself but It looks to me that you have your max_lat and min_lat settings the wrong way round. Your max value is less than the min. The max_lon and min_lon settings look correct, min_lon is more negative than max_lon.

You have to select a rectangle where your receiver is located. I am using typically a map tool (available online) to determine the four corner coordinates and put it in the script.
I do not use the script any longer so i don’t know exactly the filter settings for altitude etc.

Got it working. Column names and json names were not what they should have been. Based on the contents of the aircraft.json file, I was able to deduce what the table structure should be and make the adjustment to the code. I stripped out a lot of stuff related to filtering based on hex code and or flight number. Kept the geo cube. Notice that the lat and lon numbers make no sense. I just set them wide to see what I would catch. It is 3 AM here and no local traffic expected for 6 hours.

#!/usr/bin/php

<?php #phpinfo(); #var_dump(ini_get_all()); #ini_set('error_reporting', E_ALL); // below a sample create statement for database table Note that there are aorund 37 fields in the json file // You can figure out if you want to use them by inspecting your aircraft.json file // CREATE TABLE `aircrafts` ( //`id` int(11) NOT NULL AUTO_INCREMENT, // `now` varchar(100) DEFAULT NULL, // `hex` varchar(100) DEFAULT NULL, // `flight` varchar(100) DEFAULT NULL, // `alt_baro` varchar(100) DEFAULT NULL, // `alt_geom` varchar(100) DEFAULT NULL, // `lat` varchar(100) DEFAULT NULL, // `lon` varchar(100) DEFAULT NULL, // `track` varchar(100) DEFAULT NULL, // `gs` varchar(100) DEFAULT NULL, // `baro_rate` varchar(100) DEFAULT NULL, // `seen_pos` varchar(100) DEFAULT NULL, // `seen` varchar(100) DEFAULT NULL, // `nac_baro` varchar(100) DEFAULT NULL, // `nac_p` varchar(100) DEFAULT NULL, // `nac_v` varchar(100) DEFAULT NULL, // `messages` varchar(100) DEFAULT NULL, // PRIMARY KEY (`id`) //) ENGINE=InnoDB AUTO_INCREMENT=0 DEFAULT CHARSET=latin1; // set the rectangle and altitude to store aircraft-data in database - if your lon is negative be aware to use the right values for max and min // Or just leave them like I have them to catch every thing that is in range. $user_set_array['max_lat'] = 90.000000; $user_set_array['min_lat'] = 0.000000; $user_set_array['max_alt'] = 50000; $user_set_array['max_lon'] = 0.000000; $user_set_array['min_lon'] = -180.000000; // set lookup-interval default is 1 (must be integer between 1 - 900) this is the frequency the script runs and writes to database or looks for alerts $user_set_array['sleep'] = 1; // set parameters for database connection $user_set_array['db_name'] = 'MyDB; $user_set_array['db_host'] = '127.0.0.1'; $user_set_array['db_user'] = 'ME'; $user_set_array['db_pass'] = '[DumbPassword]'; // set path to aircraft.json If you don't know wher it is, try searching: #sudo find / -name aircraft.json // It depens on which package you installed If you got this far, you should know what I am talking about $user_set_array['url_json'] = '/run/readsb/'; // set your timezone see http://php.net/manual/en/timezones.php $user_set_array['time_zone'] = 'America/Los_Angeles'; $i = 0; $start_time = time(); date_default_timezone_set($user_set_array['time_zone']); // fetch receiver.json and read receiver latitude and longitude $json_receiver_location = json_decode(file_get_contents($user_set_array['url_json'] . 'receiver.json'), true); isset($json_receiver_location['lat']) ? $rec_lat = $json_receiver_location['lat'] : $rec_lat = 0; isset($json_receiver_location['lon']) ? $rec_lon = $json_receiver_location['lon'] : $rec_lon = 0; while (true) { $x = 0; $sql = ''; $start_loop_microtime = microtime(true); // fetch aircraft.json and read timestamp and overall message number $json_data_array = json_decode(file_get_contents($user_set_array['url_json'] . 'aircraft.json'), true); isset($json_data_array['now']) ? $ac_now = $json_data_array['now'] : $ac_now = ''; isset($json_data_array['messages']) ? $ac_messages_total = $json_data_array['messages'] : $ac_messages_total = ''; // loop through aircraft section of aircraft.json file foreach ($json_data_array['aircraft'] as $row) { isset($row['hex']) ? $ac_hex = $row['hex'] : $ac_hex = ''; isset($row['flight']) ? $ac_flight = trim($row['flight']) : $ac_flight = ''; isset($row['alt_baro']) ? $ac_alt_baro = $row['alt_baro'] : $ac_alt_baro = ''; isset($row['alt_geom']) ? $ac_alt_geom = $row['alt_geom'] : $ac_alt_geom = ''; isset($row['lat']) ? $ac_lat = $row['lat'] : $ac_lat = ''; isset($row['lon']) ? $ac_lon = $row['lon'] : $ac_lon = ''; isset($row['track']) ? $ac_track = $row['track'] : $ac_track = ''; isset($row['gs']) ? $ac_gs = $row['gs'] : $ac_gs = ''; isset($row['baro_rate']) ? $ac_vert_rate = $row['baro_rate'] : $ac_baro_rate = ''; isset($row['seen_pos']) ? $ac_seen_pos = $row['seen_pos'] : $ac_seen_pos = ''; isset($row['seen']) ? $ac_seen = $row['seen'] : $ac_seen = ''; isset($row['nac_baro']) ? $ac_baro = $row['nac_baro'] : $ac_nac_baro = ''; isset($row['nac_p']) ? $ac_nac_p = $row['nac_p'] : $ac_nac_p = ''; isset($row['nac_v']) ? $ac_nac_v = $row['nac_v'] : $ac_nac_v = ''; isset($row['messages']) ? $ac_messages = $row['messages'] : $ac_messages = ''; // generate sql insert statement per aircraft in range of user set altitude/latitude/longitude if ($ac_altitude != '' && $ac_altitude < $user_set_array['max_alt'] && $ac_lat < $user_set_array['max_lat'] && $ac_lat > $user_set_array['min_lat'] && $ac_lon < $user_set_array['max_lon'] && $ac_lon > $user_set_array['min_lon']) { $sql .= "INSERT INTO aircrafts VALUES (NULL, '" . date("Y-m-d G:i:s l", $ac_now) . "', '$ac_now', '$ac_hex', '$ac_flight', "; $sql .= "'$alt_baro', '$ac_alt_geom', $ac_lat', '$ac_lon', '$ac_track', '$ac_gs', '$ac_baro_rate', '$ac_seen_pos', '$ac_seen', "; $sql .= "'$ac_nac_baro', '$ac_nac_p', '$ac_nac_v', '$ac_messages');"; $sql .= PHP_EOL; $x++; } } // if db connection is ok write selected aircraft data to database try { $db = new PDO('mysql:host=' . $user_set_array['db_host'] . ';dbname=' . $user_set_array['db_name'] . '', $user_set_array['db_user'], $user_set_array['db_pass']); $db_insert = ''; $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); if ($sql) { $db->exec($sql); $db_insert = 'inserted'; } $db = null; } catch (PDOException $db_error) { $db_insert = 'db-error' . PHP_EOL . $db_error->getMessage(); } // generate terminal output and set sleep timer to get minimum a full second until next aircraft.json is ready to get fetched $runtime = (time() - $start_time); $runtime_formatted = sprintf('%d days %02d:%02d:%02d', $runtime/60/60/24,($runtime/60/60)%24,($runtime/60)%60,$runtime%60); ($runtime > 0) ? $loop_clock = number_format(round(($i / $runtime),6),6) : $loop_clock = number_format(1, 6); $process_microtime = (round(1000000 * (microtime(true) - $start_loop_microtime))); print('upt(us): ' . sprintf('%07d', $process_microtime) . ' - ' . $loop_clock . ' loops/s avg - since ' . $runtime_formatted . ' - run ' . $i . ' @ ' . number_format($message_rate, '1', ',', '.') . ' msg/s -> ' . sprintf('%03d', $x) . ' dataset(s) => ' . $db_insert . PHP_EOL); sleep($user_set_array['sleep']); $i++; } ?>
2 Likes

You still need to have max bigger than min numerically.
Otherwise you’ll catch nothing.

Meanwhile i have improved the script and removed the section where the location is verified.
Currently the script logs all aircraft captured by my receiver, ignoring location, altitude, speed…

I left the values in the config file, because otherwise the script complain about it.
However these are not used any longer

In addition to that I have added an aircraft list as table and built a view which shows me the aircraft details depending on teh Mode-S Hexcode, finally it looks like this, a query against both databases:

Next step is adding a table containing the flight routes and checking them against the callsign.
Finally i am experimenting on getting the script moved from SQLite to MySQL, but for this i need to improve my skills.

EDIT:
Just seen that my device caught a B-52 Bomber. Pretty rare over Germany:

This is really cool. I’ve done a similar thing, but with Python writing to a Postgres db. After some searching around I was also able to add aircraft type in. Lots of fun.

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.