Updating local DB

FlightAware ADS-B Flight Tracking
41

i know the exact path. I only mentioned the last part of it :wink:

I backuped the existing files first and then overwritten with the newer versions. After reloading the page it’s working fine.

42

Be careful to leave
/usr/share/dump1090-fa/html/db/aircraft_types/icao_aircraft_types.json
in place.

Otherwise the webinterface won’t be happy.

43

i do not have this file, it’s aircraft_types.json only without the icao_ in front of it.
This file i did not touch, move or delete

Just copied the db files to the tar1090 db directory, works still perfect :slight_smile:

EDIT: Just realized that aircraft_types is a directory. There the icao_ file is in.
All good

44

It will overwrite in case you update, just so that you are aware.

45

good hint, thanks
I will keep a copy or start over from here if necessary

46

Man, this is going to be an uphill struggle.

Now there’s the issue of the incorrect outlines / markers but that is a whole other subject.

Edit: Found and fixed some typos in my markers.js and it seems to be working now.

47

Is my 4Gb basestation.sqb file going to be any use in compiling this new db? It’s still growing, albeit not as quickly as it was.

48

The more the merrier I think. You see a lot more and different traffic to me and I believe that the conversion tools will filter out duplicated entries.

We can certainly give it a try.

49

Can someone give some idea of a practical limit to the size of the json files we are creating?

I don’t suppose it was ever envisaged that every 'plane on the planet would put be in them. Even allowing for removing the ones that can be determined from the hex id, that seems like a lot of data for a Pi to handle.

50

It’s not going to be a problem.

Remember the RPi is just acting as a stupid webserver providing the json files.
Extracting information from the takes place in your browser.

Note i’ve take down the copy of the database, i’m not convinced planebase likes their database being redistributed even if it’s very indirectly.

It’s really a shame there isn’t a central ICAO registry, even updated yearly would be fine.
Stupid system.

1 Like
51

I guess the data is available if you are willing to pay for it but this is a hobby isn’t it.

52

Going back to one of your previous posts

In the UK, the the airframe hex id remains the same even on re-reg in the UK. So some of the Monarch Airbus continued to show as the previous reg after they transferred to Thomas Cook. Confused me for a while. GA and biz aircraft works the same. Presents a problem for updating a database - do you wipe the whole thing and apply the new version or just update unknowns?

53

The current idea is to extract airplane data from as many instances of Virtual Radar Server as possible, merge them together with the FA database and convert them into a format that works in PiAware. The resulting data is then used to replace the existing data.

There are a number of issues:

Firstly, as far as I can see, there is no mechanism for editing the data in the Virtual Radar Server database so what you get is whatever VRS looked up in the planebase databases. Do we know how accurate it is?

Secondly, if there are records in multiple VRS databases that contain the same hex id but different ‘tail numbers’, how do we know which records are the most recent?

Thirdly, for the reason stated in the posts you picked up on, there seems to be a question mark about the deterministic decoding method used in DUMP1090. Would it be better NOT to filter out those records or would DUMP1090 just go ahead and override them by calculating them?

Fourthly, how ‘global’ the data is would depend on the number and location of instances of VRS from which we can extract data. It seems to me that data extracted from my database wouldn’t be a whole lot of use to someone living in New Zealand.

Lastly, as @wiedehopf said, it isn’t clear how happy planebase would be with us providing access to data based on information they provided to VRS.

The whole thing may ultimately be a waste of effort.

1 Like
54

What about a basestation.sqb? I use the basestation.sqb with modesmixer2 and VRS for data.

1 Like
55

If you want a reliable source of hex/registration data, then running a VDL2 decoder and gathering the data directly is possible: Feeding dump1090 aircraft DB with vdlm2dec | TLeconte.github.io

It won’t get you aircraft type, but it worked ok when I tried it.

There is a database editor plugin for VRS: Download - Virtual Radar Server

You need to make sure you get the right version if you are using a preview version of VRS.

2 Likes
56

Skyview only uses a deterministic mapping when one exists. A deterministic mapping means “This hexid 12345 will always be this registration X-YZQQ”, by definition. This is true for e.g. the US, and for a few other countries, but for many countries (e.g. the UK example above) there is no deterministic mapping.

57

Thanks @caius, I read about using VDL2 data to get the registration but discounted it because it doesn’t help with stuff like Aircraft type, description, wake category or number of engines.

I was hoping to expand this project so that I could automatically update that information where it was needed.

I somehow missed the VRS editor though but that immediately brings us to problem #2.

Lets say that user 1 provides data from their instance of VRS and user 2 provides data from their instance of VRS and user 1 had edited their data but user 2 hadn’t.

How would we know which was version was correct?

With the best will in the world, typos and corruption creep in.

edit: I downloaded the database editor from the link you provided and discovered that I already had it. What is happening to my brain?

1 Like
58

Sorry if I’m being dense. I understand that the toolchain removes the information that could be determined but I don’t understand what Skview would do if the data were left in the database. Would it use the data from the database or the mapping it would have used if the data weren’t in the database?

59

If you want to know the exact details, look at the code:

The initial value is the result of the deterministic lookup. If there’s a DB entry available (which may take an AJAX request to discover) then that overrides it when it becomes available.

The toolchain runs exactly the same deterministic lookup code and removes the registration from the database if the deterministic lookup returned the same value. So the only way you will end up with a different result from the data you gave to the toolchain is if you’ve modified the deterministic lookup code since the database was generated.

If the data provided when the database was generated does not agree with the deterministic lookup, then the provided data is retained and will override the deterministic lookup when the AJAX lookup completes. This case will also complain at you when you generate the database; in all the cases I’ve encountered where this has happened, it was the input data that was wrong, not a case where there was an exception to the deterministic lookup.

60

Thanks @obj

:thinking: