Upgraded, now less is being seen??

FlightAware ADS-B Flight Tracking
1

Hi guys,

I upgraded yesterday to 3.1.0, reason for the upgrade was I was seeing 160K at peak and around 150K average, and it was going down (down to average of 120K)
But since the upgrade I am getting a lot less (currently >24 hours = 80K).

Bug maybe? or something else??

Any suggestions in getting this back up and running as it was.

2

Your stats look pretty much the same today as yesterday except for a 4-6 hour outage (10AM to 4PM local time)
flightaware.com/adsb/stats/user/26CT1681

3

I saw a drop off as well (7-10%) moving from 2.4.1 or whatever to 3.1 - running parallel rigs so has nothing to do with daily traffic, comparing against each other, unchanged rig is my watermark/baseline. The changelog is unclear as to what may have changed so far as data aggregation. Upgraded version is running dump1090-mutability as was the previous version - identical config settings, hardware, etc. 3.0.4/5 did not seem to have the same drop off when the other rig was updated. …For what it’s worth. Maybe somebody slipped something into my drink.

4

The positions-reported number is not comparable between PiAware versions, exactly because the reporting parameters change from version to version.

for example: 3.1.0 overhauled the decoder fairly substantially to prefer ADS-B to Mode S data, and to prefer fully-CRC-checked Mode S data to partially-CRC-checked Mode S data.
That is likely to reduce the number of reports somewhat, as “flapping” values where it’s actually just bad data won’t trigger more frequent reports (usually a changing value e.g. altitude triggers a more frequent report).

also: if you’ve got a view of an airport your reports have probably gone up because 3.1.0 increases the reporting frequency for surface positions.

overall: not comparable with 3.0.

5

Since moving to 3.1 I now see less range and possibly therefore less messages. As an example, the stats page for the last 24 hours on 3.1 showed no positions over 200nm, in the last hour back on 3.0.4 I have eight.

I have since returned to 3.0.4 and placed auto-updates to ‘No’. If FlightAware consider 3.1 a commercial move then I can understand, but for ‘enthusiasts’ that want to primarily track aircraft and feed FA secondary then I don’t consider it a forward movement. Mind you, the interface was nicer in 3.1!

6

Thanks for the breakdown. That makes sense now and happy to know my testing parameters haven’t gone belly-up. Regardless of reported position counts, I can only assume you are encouraging everyone to upgrade for the sake of overall system integrity so we don’t need to be taking this recalibration as a reason not to make a switch. Accuracy trumps individual stats and as things mature changes will happen.

Cheers~

7

There were no changes to the core demodulator between 3.0 and 3.1 (which is what is going to affect message rate and range). I suggest you do a proper test. Comparing one range bucket for one day is not a proper test. Ideally you’d run the two systems in parallel off the same antenna, but I know that’s not necessarily practical.

I would also suggest you derive your own stats rather than using FA’s stats for this. FA’s stats reflect data fed to FA; the data fed to FA depends on the data that FA cares about; none of that affects what you see locally, which is presumably what you care about here given your comment above. So measure what you’re receiving locally.

One thing that seems to cause confusion is that there is a distinction between the source of the ADS-B messages - be it dump1090-fa, dump1090-mutability, a Beast, whatever - and the data fed to FA by piaware. I would strongly encourage you to use PiAware 3.1.0 to feed data from the data source of your choice, even if the data source of your choice is not dump1090-fa. If you’re not using the latest-and-greatest PiAware, the data fed to FA is less useful, and given enough time eventually data from old PiAwares will just stop being used.

8

@obj

I have another ‘test system’ in place that is actually a source of data for my online reports (HfAero.com is for sale | HugeDomains). This system takes port 30003 data from dump1090-fa and parses it to a MySQL database using a java script. I’ve had this running off and on for a most years and I really understand what my receiver (which have included PGR, Beast, SBS-1er, FlightRadar, RTL and now FlightStick Pro) should be receiving. Since moving to 3.1 two days ago I started to see some unusual occurrences in my reports such as incorrect flight ID’s associated with positions, Bombardier Dash8 at 234nm at FL200, that sort of thing which indicates some erroneous 30003 data. But mostly I noticed no flights being received beyond about 170nm on those routes where I’d expect to see 200-230nm.

I don’t have hours and hours to spend setting up testing systems and methodologies, I go by what my true and , er hem, tested system has yielded in the last 10 years.

Sorry to be one of the bearers of bad news, but 3.1 is a downgrade for me.

9

I’d like to work out what’s going wrong here. Port 30003 is processed data, so it could well be problems in the guts of the decoder as there were extensive changes there. Can you capture some raw data when you see something going wrong? i.e. the raw messages, not port 30003 - try view1090-fa --show-only and mail me the output. Maybe you could pick an outbound flight that you would expect to track beyond 170nm and capture that. I don’t see problems here, but that could just be different message characteristics in Europe vs NZ.

My point about data source vs. PiAware versions stands, please run piaware 3.1.0 even if you’re not using dump1090-fa 3.1.0

10

Also, the 30003 output code changed, possibly there’s a problem in there (or it produces results in a form that your script is not expecting), can you go back to the original 30003 data when you see a problem and verify that the problem exists in the original data too?

(edit: I spotted a bug in there which would add an extra comma when only GNSS altitude was available, not barometric altitude, I wonder if that explains any of what you see)

11

I just ran a 5 min regression test on port 30003 output between 3.0.5 and 3.1.0 (this is where I feed identical data into the old and new versions and look at the differences in output) and the only differences are not significant:

  • the exact SBS “message types” reported in field 2 have changed (3.1.0 tries to be more accurate, though there’s no real spec for this)
  • the dummy fields (11111 etc) that aren’t used by dump1090 have different dummy values
  • speed and heading are up to 1kt / 1 degree different as 3.1.0 rounds fractional values but 3.0.5 truncates them
  • the on-ground indicator is populated on fewer messages because 3.1.0 does not trust the airground indication in some messages where 3.0.5 does
  • squawks are now always reported as 4 digits (3.0.5 would drop leading zeros)

So I can’t reproduce what you are seeing and I will need more data from you to diagnose the problem.

If the problems are relatively frequent, capture the binary output on port 30005 for a while and send me the (large) results and I can run regression testing on that.

12

That could be significant depending on the decoder interpolation MSG{1…8}, especially if those rooted from DF17 {1…4} are involved in the change(s). Wouldn’t this also effect in the .py aggregation layer as it chooses what to hand off? (I assume this change is at the demodulator level, so would effect all output types?) I could be way off base as I’m still trying to learn and follow your work. I praise thee for trying to keep this cludge straight in the first place as you are right when it comes to published spec (or lack thereof). Besides the extra field (added comma bug) in those more rare occurrences, his parser could be going off the rails on any subtle change in the message type field which could explain the anomalies. Good place to look at either rate.

Hopefully this is viewed as a derailment, I’m still learning and hope this is somewhat on cue.

13

The SBS “message type” field is specific to port 30003 output, it doesn’t affect other output types, and it doesn’t really carry any data itself. Could break the parser on the other side, it’s hard to know without seeing the whole system.

14

I don’t understand!

I was getting 160K+ per day a few months back, I upgraded because the stats were getting lower and lower, but since upgrading I am getting a lot lower.

15

Sounds like it is some combination of:

  1. You have a deteriorating antenna/RF installation, check for loose connections or corrosion.
  2. Maybe the traffic patterns near your site have changed
  3. Stats are not comparable between 3.0 and 3.1 because the reporting intervals changed.
16

I’ll check antenna/connections.
If anything air traffic has increased quite a bit around us, big local discussions (moaners)
I actually didn’t have v3, but v2 something.

Thanks

17

I would expect an upgrade from 2.x to be maybe 25% better as it has a better demodulator. About the only thing I can think of is that it may be worth checking your gain settings as the new demodulator may work better at a different gain setting to 2.x