Pi with poor network connectivity feeding FA

FlightAware ADS-B Flight Tracking
1

A friend of mine has a Raspberry Pi 3B+ running the 8.2 PiAware image. The unit’s current placement puts it right on the edge of wireless connectivity on the local network and, as a result, connectivity to it suffers from a lot of jitter. It’s running through a couple of mesh nodes which are needed to get wifi over to where it is located and they appear to be quite temperamental in the coverage they provide.

As a consequence, ping times from a client (which does have a good connection or is on ethernet) vary from a few ms to 1+ seconds, and the SkyAware front-end frequently reports it has lost the connection, usually 3-4 times per minute.

call_failed

dump1090-fa is running fine on the PiAware, it is just the connection that is the problem.

My questions are these:

  1. Does the jittery network connectivity of the PiAware have any impact on feeding FlightAware? For example positions or aircraft numbers reported? Or is feeding-related data queued up and sent a few seconds later anyway?

  2. Are there any Pi 3B+ or OS tricks to boost wifi performance, eg based on hardware or chipset? It’s using the internet wifi – I guess an external antenna could help.

  3. Are there any useful OS or PiAware-specific commands worth knowing which can assist in diagnosing the state of the connectivity and the status of the PiAware with respect to FlightAware? Ideally in real time. For example something which shows that it missed sending x positions to FlightAware in the last minute, that kind of thing?

  4. The SD card is a few years old but appears functional, albeit not one of the faster newer ones. Might this have any bearing on connectivity? For example if PiAware ends up spending disproportional resource on queuing up card I/O. Seems unlikely but worth asking for completeness.

Thanks for any insights.

2

There’s no buffering; if the connection to the FlightAware servers is down, then positions are not reported and are discarded.

mlat, in particular, will be very disrupted if the connection is frequently down, as the synchronization process has to restart on each reconnection.

There’s no statistics that directly report on this, but you can monitor /run/piaware/status.json which is frequently updated with the current connection status.

piaware should be generating almost no sdcard i/o

3

Thanks for the info. His mesh is a set of Linksys Velop devices. They’ve been working fine up to now but suddenly become very unstable. There are various threads about the same symptoms on Reddit.

We tried connecting his PiAware to his distant broadband router via a weak wifi signal and it connected, and the disconnect problem appears to have gone away. So the mesh seems implicated. Will do further testing to try and get to the bottom of it.

4

You could look at: sudo journalctl -eu piaware

For example the last message count on mine shows:
May 13 11:34:08 raspberrypi piaware[1215]: 1774812 msgs recv'd from dump1090-fa (4105 in last 5m); 1774801 msgs sent to FlightAware

This has been running for about 2 1/2 days and the number of messages sent to FlightAware is 11 less than what was received from dump1090-fa. So there was (probably) 1 disconnect and reconnect with FlightAware during that 2 1/2 days. This is typically what a see over a few days of operation – i.e. one or no disconnects. If you see a big difference in those numbers, or it seems to be growing with time you have a problem.

You can also look at sudo cat /var/log/piaware.log to view the latest piaware log and find out where there were disconnects with FlightAware.

I used to have lots of disconnects/reconnects with FlightAware. Turned out to be an IPV6 problem with my router.

5

Thanks @jimMerk2 that’s a useful comparison of messages sent and the FA count. In fact I was looking at journalctl earlier and it appears that for some reason the wlan0 interface stopped working, taking everything offline. And before that various messages saying that a percentage of “MLAT messages were not sent – network problem?” or words to that effect.

It’s looking like this is also an IPv6 problem – I need to delve into the logs a bit more, but it appears to be some sort of interaction with the way the Velops manage connections and IPv6. Going to try removing the mesh for now and test with the broadband router alone and wifi vs ethernet, as a baseline, and go from there.

6

There is a site you can go to that tests how well your computer is using IPV6:
test-ipv6.com

Right now when I test it I get 9/10 score. However when I was having a problem with IPV6, I got a diagnostic message:

Danger! IPv6 sorta works - however, large packets appear to fail, giving the appearance of a broken website. If a publisher publishes to IPv6, you will believe their web site to be broken. Ask your ISP about MTU issues; possibly with your tunnel. Check your firewall to make sure that ICMPv6 messages are allowed (in particular, Type 2 or Packet Too Big).

I keyed on the part about "large packets appear to fail’ because I was seeing disconnects from the FlightAware servers when there was a lot of traffic.

Anyway, after I changed my WiFi router, I no longer see that diagnostic.

You might want to try that IPV6 test site.

7

You might want to consider configuring and plugging in a wifi booster/extender near to your Pi hardware. The TP-Link AX1500 WiFi Extender Internet Booster(RE500X) is a unit that I have found to provide good performance for wifi devices struggling with wifi signal strength/connectivity. $55 on Amazon. TP-Link has proven itself in the field over quite a few years in the business now. Hard wire ethernet is always best but when that isn’t possible it is best to bring the wifi to your device.

8

Thanks, in fact he only has IPv6 on his local network, not externally via his ISP. In those cases I tend to just disable IPv6 entirely.

9

How do they work in practice? Do they create a second wireless network piggybacked onto the first one?

In the end the problem appeared to be down to these Velop mesh nodes. Although the client was connected via ethernet, it was ethernet into a node which was connected via wifi to its parent mesh node.

Client ---- velop 2 ~~~ velop 1 --- router --- internet
PiAware ~~~ velop 3 ~~~~^

~~~ wifi  --- ethernet

As a result both the client and the PiAware were being periodically stalled, typically every 30 seconds or so. The Velops send out a raft of DNS PTR requests every 30 seconds so this seems implicated. They were purchased for his previous house so they’ve now been removed and everything connected directly to his router. Although the signal is weaker for the PiAware it is stable and the periodic disconnects have stopped happening.

Client ---- router --- internet
PiAware ~~~~^

The Velops will next be tested in bridge mode to support the router network directly. This will create another wifi network on the same IP network and this will be tested for stability or to see if the same problem returns.

We’re also looking at installing Cat 6 cable, from near the router to a couple of remote locations, and adding some switches. This will eliminate the need for wifi entirely except for handheld devices and some IoT stuff. PiAware and the main client will be on ethernet which should make for a nicer experience.

10

Yes you are correct. The wifi booster/extender sees your local network and creates [during a start-up configuration] the same base networks with a “EXT” extension to signify they are extended available networks of your already established routed networks. They can be accessed with the same password established for the base networks in 2G / 5G or whatever flavors your router is supporting. I think your idea of installing a Cat 6 cable infrastructure is the very best idea. Keep those cable runs away from fluorescent lights and any other electrical cables that create “fields” of potential interference in close proximity. Ethernet switches are absolutely the way to build out your connectivity so that each connected device has a dedicated switched connection and is not sharing a “party line” type connection on a basic hub device. Sounds like you are certainly on the right path with those thoughts.

2 Likes
11

If you go the Cat6 route, please make sure that the cable ends are properly terminated with Cat6 connectors as well. The ones with the metallic shroud, otherwise you just have Cat5. Everything from point A to point B must be Cat6 rated. If you do not see the metal shroud in the jack or cable end, it is not Cat6.

If you can, depending on you IT network skill set, I would suggest that you get managed switches, not much more expensive than un-managed. This way you can remote connect to these switches and see what is going on inside on a port basis. You can also setup VLAN’s to separate traffic if you so choose.
Have a look at TP-Link TL-SG1016DE

1 Like
12

Nice one, thanks, it’s handy to know how they work. This sounds the same as how the Velops work in bridge mode, except the Velops are also doing some inter-node related stuff and lots of management traffic behind the scenes in order to hand off clients and communicate with the management app. The TP-Link model sounds simpler and focused on the one job, so that will probably be the way to go if one turns out to be needed later on.

Good point, yes Cat 6 and shielded RJ45 plugs. Edit – I think in our case we’ll be using Cat 6 between parts of the house with a network socket installed in each location, so will make sure the sockets are of suitable spec and quality.

Already got some Netgear GS108E spare, they were actually a bit cheaper than the unmanaged GS108.

13

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.