This is my first post in this forum but have been a long time reader. Up until this point, I usually could find answers to my many questions somewhere in the forum but I cannot find anything about the following Anomaly Report which I am receiving.
Anomaly report for PiAware feeder with a MAC address of xx:xx:xx:xx:xx:xx:
12% of UDP multilateration traffic sent by piaware is not reaching the FlightAware servers. This may indicate a network problem.
Since the upgrade to PiAware 2.1-3, I have been receiving the above anomaly report on an intermittent basis usually beginning in the early afternoon and continuing through late evening.
Here is a part of the piaware log file showing the anomaly message.
10/21/2015 20:42:43 mlat(2323): Receiver status: connected
10/21/2015 20:42:43 mlat(2323): Server status: synchronized with 21 nearby receivers
10/21/2015 20:42:43 mlat(2323): Receiver: 267.4 msg/s received 4.7kB/s from receiver
10/21/2015 20:42:43 mlat(2323): Server: 0.9 kB/s from server 0.0kB/s TCP to server 2.6kB/s UDP to server
10/21/2015 20:42:43 mlat(2323): Results: 397.1 positions/minute
10/21/2015 20:42:43 mlat(2323): Aircraft: 72 of 83 Mode S, 25 of 37 ADS-B used
10/21/2015 20:42:51 204294 msgs recv’d from dump1090 (1585 in last 5m); 204294 msgs sent to FlightAware
10/21/2015 20:47:51 205885 msgs recv’d from dump1090 (1591 in last 5m); 205885 msgs sent to FlightAware
10/21/2015 20:51:31 NOTICE from adept server: 11% of multilateration messages (UDP) are not reaching the server - check your network?
10/21/2015 20:52:31 NOTICE from adept server: 14% of multilateration messages (UDP) are not reaching the server - check your network?
10/21/2015 20:52:51 207512 msgs recv’d from dump1090 (1627 in last 5m); 207512 msgs sent to FlightAware
10/21/2015 20:53:31 NOTICE from adept server: 14% of multilateration messages (UDP) are not reaching the server - check your network?
10/21/2015 20:54:32 NOTICE from adept server: 13% of multilateration messages (UDP) are not reaching the server - check your network?
10/21/2015 20:55:32 NOTICE from adept server: 11% of multilateration messages (UDP) are not reaching the server - check your network?
10/21/2015 20:56:32 NOTICE from adept server: 13% of multilateration messages (UDP) are not reaching the server - check your network?
10/21/2015 20:57:33 NOTICE from adept server: 12% of multilateration messages (UDP) are not reaching the server - check your network?
10/21/2015 20:57:43 mlat(2323): Receiver status: connected
10/21/2015 20:57:43 mlat(2323): Server status: synchronized with 25 nearby receivers
10/21/2015 20:57:43 mlat(2323): Receiver: 295.4 msg/s received 5.1kB/s from receiver
10/21/2015 20:57:43 mlat(2323): Server: 1.1 kB/s from server 0.0kB/s TCP to server 3.0kB/s UDP to server
10/21/2015 20:57:43 mlat(2323): Results: 485.0 positions/minute
10/21/2015 20:57:43 mlat(2323): Aircraft: 63 of 73 Mode S, 24 of 36 ADS-B used
10/21/2015 20:57:51 209046 msgs recv’d from dump1090 (1534 in last 5m); 209046 msgs sent to FlightAware
10/21/2015 20:58:33 NOTICE from adept server: 12% of multilateration messages (UDP) are not reaching the server - check your network?
And this goes on and on and on!
My setup is a Raspbery Pi model 2 running the plain PiAware image file. The Pi is hardwired into my Ethernet hub thus wifi is not the issue. I’ve swapped Ethernet cables and hub ports but still no joy.
I am curious as to if the messages are actually not reaching the server OR are they reaching the server but not in a time to be useful. If they are truly not reaching the server, then maybe I have a buffering issue in my hub or router.
I would appreciate any suggestions as what I should try to determine where the problem lies.
2.1-3 started providing information that allows the server to notice message loss; it was probably also happening in 2.1-2, but without the extra info the server can’t tell.
The packet loss measurement is just a very simple “UDP messages received by server” vs “UDP messages sent by client” (if at least 5 UDP messages are sent in a 60 second interval). It doesn’t look at the time of arrival and it doesn’t try to be clever about noticing e.g. duplicates or messages in flight.
So unfortunately all I can really say here is that only 85-90% of the messages sent by the client are reaching the server - it could be anything in between that’s dropping them.
If it turns out to be a widespread problem I’ll probably raise the threshold for warning about it, currently it’ll warn about loss over 10%.
Hey obj, just so you are aware of my issues too, i randomly get errors that say “33% of UDP multilateration traffic sent by piaware is not reaching the FlightAware servers. This may indicate a network problem.”, or “This feeder is not being used for multilateration because its timing information appears to be unreliable. This can be caused by the site location being incorrect, or because your Pi is running out of free CPU.”
I’ve reset my router, restarted router when this issue comes up, I’ve swapped out RPis from an original RPi to an RPi 2, I’ve changed out wifi cards, tried hardwired ethernet, built new SD cards, and I still can’t seem to get these messages to go away.
Not only that but randomly since upgrading to 2.1.3, the RPi quits feeding to FA and is unresponsive over the network. 2.1.2 had worked fine and as soon as I upgraded to 2.1.3 it seems like a few of these issues had come up, I’ve also seen a few other people report this too.
Thanks obj for your input. You confirmed my suspicions that I have probably always had the problem and that the new release was just able to detect and notify me of what was happening. If the issue is in my network, then it probably is with the router. There are several things I am considering including enabling QoS and giving the UDP a high priority. I may also update the firmware in the router as it is pretty old. Unfortunately given the intermittent nature of the problem, this may take a while to resolve. Thanks again for your assistance.
one thing you guys could do to test if its a router issue is put the IP of the rPI as a DMZ and it should bypass and firewall. Just be aware you will expose that device to the internet without a firewall.
[quote=“daverdfw”]
one thing you guys could do to test if its a router issue is put the IP of the rPI as a DMZ and it should bypass and firewall. Just be aware you will expose that device to the internet without a firewall.
[/quote]
Just tried your DMZ suggestion and it had no effect. The anomalies were still there for the couple of hours that the DMZ was active. I think I am going to try some new firmware as the version of the tomato firmware that I am using no longer supported. Anyway thanks for you help.
