Are there other users out there that have issues connecting their feeds to FR24 ?
One of my feeds got disconnected somehow yesterday afternoon and up till now it is showing online but no data is being sent.
When looking at the logs is shows that it is trying to connect but is unsuccessful in doing so.
…
Jul 30 20:09:43 raspberrypi24tom fr24feed[609]: [I] Stopping ReceiverACSender threads for feed EHAM808
Jul 30 20:09:43 raspberrypi24tom fr24feed[609]: [I] Joined network thread for feed EHAM808
Jul 30 20:09:43 raspberrypi24tom fr24feed[609]: [I] Configured ReceiverACSender: 185.218.24.22:8099,185.218.24.23:8099,185.218.24.24:8099, feed: EHAM808, send_interval: 30s/30s, max age: 15s, send metadata: true, mode: 1, filtering: true
Jul 30 20:09:43 raspberrypi24tom fr24feed[609]: [I] Network thread connecting to 185.218.24.22:8099 for feed EHAM808
Jul 30 20:09:43 raspberrypi24tom fr24feed[609]: [feed][n]EHAM808@185.218.24.22:8099/UDP
Jul 30 20:09:43 raspberrypi24tom fr24feed[609]: [feed][n]connecting
Jul 30 20:10:48 raspberrypi24tom fr24feed[609]: [feed][n]Error: Could not connect feed! (result -1, fd -1)
Jul 30 20:10:48 raspberrypi24tom fr24feed[609]: [I] [feed][n]waiting 5 seconds
Jul 30 20:10:53 raspberrypi24tom fr24feed[609]: [feed][n]EHAM808@185.218.24.23:8099/UDP
Jul 30 20:10:53 raspberrypi24tom fr24feed[609]: [feed][n]connecting
Jul 30 20:10:53 raspberrypi24tom fr24feed[609]: [I] Network thread connecting to 185.218.24.23:8099 for feed EHAM808
Jul 30 20:11:59 raspberrypi24tom fr24feed[609]: [feed][n]Error: Could not connect feed! (result -1, fd -1)
Jul 30 20:11:59 raspberrypi24tom fr24feed[609]: [I] [feed][n]waiting 12 seconds
Jul 30 20:12:11 raspberrypi24tom fr24feed[609]: [feed][n]EHAM808@185.218.24.24:8099/UDP
Jul 30 20:12:11 raspberrypi24tom fr24feed[609]: [feed][n]connecting
Jul 30 20:12:11 raspberrypi24tom fr24feed[609]: [I] Network thread connecting to 185.218.24.24:8099 for feed EHAM808
Jul 30 20:13:16 raspberrypi24tom fr24feed[609]: [feed][n]Error: Could not connect feed! (result -1, fd -1)
Jul 30 20:13:16 raspberrypi24tom fr24feed[609]: [I] [feed][n]waiting 15 seconds
Jul 30 20:13:31 raspberrypi24tom fr24feed[609]: [feed][n]EHAM808@185.218.24.22:8099/UDP
Jul 30 20:13:31 raspberrypi24tom fr24feed[609]: [I] Network thread connecting to 185.218.24.22:8099 for feed EHAM808
Jul 30 20:13:31 raspberrypi24tom fr24feed[609]: [feed][n]connecting
Jul 30 20:14:36 raspberrypi24tom fr24feed[609]: [feed][n]Error: Could not connect feed! (result -1, fd -1)
Jul 30 20:14:36 raspberrypi24tom fr24feed[609]: [I] [feed][n]waiting 22 seconds
Jul 30 20:14:58 raspberrypi24tom fr24feed[609]: [feed][n]EHAM808@185.218.24.23:8099/UDP
Jul 30 20:14:58 raspberrypi24tom fr24feed[609]: [I] Network thread connecting to 185.218.24.23:8099 for feed EHAM808
Jul 30 20:14:58 raspberrypi24tom fr24feed[609]: [feed][n]connecting
Jul 30 20:16:02 raspberrypi24tom fr24feed[609]: [I] [stats]sent 16 bytes
Jul 30 20:16:03 raspberrypi24tom fr24feed[609]: [feed][n]Error: Could not connect feed! (result -1, fd -1)
Jul 30 20:16:03 raspberrypi24tom fr24feed[609]: [I] [feed][n]waiting 25 seconds
Jul 30 20:16:08 raspberrypi24tom fr24feed[609]: [time][i]Synchronizing time via NTP
Jul 30 20:16:08 raspberrypi24tom fr24feed[609]: [time][i]Time synchronized correctly, offset -0.001 seconds, drift -0.000 seconds/minute
Jul 30 20:16:08 raspberrypi24tom fr24feed[609]: [reader][i]Timestamp source changed from SYSTEM-UNCERTAIN to SYSTEM-VALIDATED
Jul 30 20:16:28 raspberrypi24tom fr24feed[609]: [W] Exited network thread for feed EHAM808
Jul 30 20:16:28 raspberrypi24tom fr24feed[609]: [feed][d]Fetching configuration
Jul 30 20:16:28 raspberrypi24tom fr24feed[609]: [feed][i]decoder config not changed
Jul 30 20:16:28 raspberrypi24tom fr24feed[609]: [feed][i]configuration changed
Jul 30 20:16:28 raspberrypi24tom fr24feed[609]: [feed][i]defined 3 servers
Jul 30 20:16:28 raspberrypi24tom fr24feed[609]: [feed][c]Timestamps: optional
Jul 30 20:16:28 raspberrypi24tom fr24feed[609]: [I] Stopping ReceiverACSender threads for feed EHAM808
…
I’ve tried both avr-tcp and beast-tcp with their associated ports but both are to no avail.
All my other feeds are working correctly. this specific station is also connecting normally to FA and reporting data.
Also I’ve purged the software, reinstalled it and tried both protocols but the result is still the same, system stays disconnected in the status screen
Reported a ticket to FR24 support this morning, it seems to me they have a server issue in their system but haven’t received an answer yet.
Anyone else seeing the same issues since yesterday ?
It’s running fine here with the same version you are running.
I did a quick restart to watch the log. I noticed that fr24 gives me a list of 3 server IPs, and that my system is connecting to the .22 one. I see that your failed attempts are going to the .23 one. It looks like possibly an issue at their end with load balancing between servers. I do not know if it is possible to exclude a server or force a single selection.
Disregard. I see now that you are cycling through all the available servers. I noticed the different server first before I posted, but then did a copy/paste showing the exact one I am using.
OT: Personally, I always use TCP despite what they used to recommend. Their setup help always seemed a bit dated, and their recommendation to disable mlat when feeding other sites always sounded a bit odd.
Thanks for the confirmation, I did check if the receiving server could be stated but to no avail.
Stupid thing is that all other sites I have work fine ( 16 out of 17 work fine but just this one isn’t connecting) so I’ve scratched my head a few times already.
I once had the issue with a system not picking up the ini file as it should and there the solution was changing from avr-tcp to beast-tcp. That cured the issue of the other site permanently but this one doesn’t want to establish a correct data connection.
It’s status stays online-no data in the FR24 stats page.
That is a strange one. The only reason I even looked was because yesterday during a primary internet outage, I was uploading to FA and FR24 through a Win-10 VirtualBox server running Linux with backup hotspot data. It worked fine then. Today back on primary internet the Pi is uploading normally to FR24.
I think the only issue I had with FR24 was once when I had disabled their FR24 automatic software updates. Once I was a few versions behind when they e-mailed requesting the update. FR24 for me is generally very stable. I normally only view their traffic through their Android app, which is excellent. The FA Android app is not that useful in my opinion, at least for what I use it for.
Via the status page of the FR24 feeding software : ip adress xxx.xxx.xxx.xxx:8754
Then use show logs
Copy/paste the portion of the log you want to paste into a message
After some days of joint investigation the issue is resolved for now.
We tried a number of things
remove IPv6 from the feeder
adapt DNS to the standard google DNS
forcing the system to use http only.
None of this worked, system stayed offline until 3 hours ago, then the system started reporting again by itself without any manual intervention.
Both the FR24 and myself are stumped by this behavior of the system.
The system itself worked flawlessly towards FA in the same timeframe so a hardware fault was already ruled out.
I agree but FR24 insisted that there was no issue with the servers on their side.
I posted this so others know how to approach this kind of issue in the future
Yep I agree but I wanted to get to the issue behind this to see what was going on. Issue lasted for 3,5 days on a row so that’s something I want to look into.
The issue jumped from the affected station to another one that is already for 3 years in service.
Only thing I can think about is that the number of simultaneous connections towards FR24 is limited somehow, otherwise it wouldn’t jump from station A to station B.
I’ve asked FR24 that question, I guess the answer will be no but I can’t explain it any other way.
Connection to FA is running just fine for the affected station
Their feed software has some flaws, it’s simple to assume the servers do as well. (others might use much harsher language)
This might be a bit technical:
fr24feed will iterate (loop) over all available file descriptors (number of files a program can have open) and do a system call (takes some time / CPU) on each and every one.
So if your linux or container for some reason allows programs to open a lot of files, fr24 will use 100% CPU every few seconds when it iterates over all available file descriptors, even when it uses less than 20 open files.
This obviously is unrelated to your issue, that’s almost always an issue with their servers.
But i ran across this issue years ago and only recently re-discovered it when people were having the issue with containers. https://github.com/sdr-enthusiasts/docker-flightradar24/issues/132
That’s somewhat easy to test by turning off another station (or all other stations).
You got antennas looking in different directions or why do you need multiple fr24feed clients?
I’ve used 2 or 3 just for testing but never more. (and no issues so far)
I can follow you technically so no problem there.
I’ve checked the CPU’s but there no issue there in eating away the capacity.
CPU use hardly exceeds 10% on this system, on high traffic moments it’s 15%.
I’ve tested the limiting of the number of connections, it does follow around, when I restart a system or even better halt it for 10 minutes then that system will have an issue coming back online, so that’s why I suspect a connection limit.
The antenna’s look in different directions ( I live in a multi cornered building with a lot of blind spots you have to work around) and are set to long range or short range reception.
Besides that I have some test sites that i use to play around with.
They have different antenna’s dongles and other stuff to get a spread of hardware that’s out there ( radarbox, fa, rtl-sdr, adsb-exchange as dongles and a mix of FA, Vinnant and other antennas).
It is probably overkill in regard to aircraft in the area but it enables me to test something if there’s an issue on these boards where someone has a problem or to try new stuff without breaking the setup when doing so.
Yep that will probably be the answer in the future.
I’ve played around a bit more this morning and the disconnected status follows exactly to the halted station enabling the other station to come back online.
So this certainly looks like some form of connection limit when you have more than 16 stations (yeah I know, I need some professional help for that ).
I think this will hardly be an issue to the average user (they would never have that many stations at one location).
For now I leave the status like it is on the newest station so they can investigate it ( if they are willing). After that I will remove the fr24feed software from that one again.
I followed up on the CPU eating since you managed to spark my curiosity in looking into that.
With ulimit 1000 which might be the systemd service default for most distros, it really doesn’t show.
For 10k / 60k ulimit it starts to get noticeable.
Then i’ve seen some weird setups that have a ulimit that is 1M or something which means fr24feed will be stuck at 100% CPU.
The practical issue was besides the point, i’m well aware 99.9% of installs aren’t affected in any noticeable way.
Also the connection limit seems reasonable. I’d personally just cancel the ticket but then i don’t have 16 stations!
I get running more stations than necessary to cover let’s say 3 or 4 directions, but why feed the data? That seems like a bit of resource waste to me. But i’ve dealt with server side mlat-server stuff and that clearly colors my opinion.
).