Loss of MLAT due to Server Clock Instablitiy

Having a problem that surfaced when upgrading to PiAware 3.5.3, improving antenna location and number of aircraft seen. If I reboot the Pi it will work for a while until number of aircraft increase. A fan was mounted on the unit and I turned it on this afternoon but still have the same issue.

Here’s a portion of the Log File.

[2017-11-13 15:33 EST] 39264 msgs recv’d from dump1090-fa (4122 in last 5m); 38111 msgs sent to FlightAware
[2017-11-13 15:33 EST] mlat-client(1177): Server status: clock unstable
[2017-11-13 15:33 EST] mlat-client(1177): Receiver: 1139.6 msg/s received 176.0 msg/s processed (15%)

Is this a hardware issue? Has anyone else seen this condition? Is there a software setting that can be made to correct the problem.

Thanks in advance for any suggestions!

Lowell
OSUH8

Yes, I also got “server clock unstable”, when I checked logs after upgrading Piaware from 3.5.1 to 3.5.3. I did not check again. Will check again tonight to find out if it was temporary or is persisting.

OS : Ububtu 16.04.3 x64 installed on Desktop PC.

Piaware upgraded from 3.5.1 to 3.5.3 by building Piaware 3.5.3 package from source code.

All seems OK. “Server clock unstable” no more there. Seems it was a temporary condition related to FA Servers, nothing to do with upgrade to 3.5.3.

Still having the problem today. Just rebooted to restore MLAT. Is the clock issue with the FA servers or something internal to the Pi unit? Anyway to determine that?

I think you were running a PC version so I’m hoping it’s not station related.

Lowell

Your receiver (I assume it’s 39602 you’re having trouble with) looks fine at the moment.

There have been no server side changes recently, so it’s probably hardware problems on your side, perhaps temperature or RF noise related.

I looked at some of the historical logs and your receiver has been flapping in and out of unstable for at least a couple of months. It’s got gradually more frequent over the last couple of weeks. Might be hardware on its way out.

I have had this before and it was solved by changing the power supply. I believe the bad power supply was causing USB issues which would manifest itself in this error.

Anyway, a new power supply, clock issues removed (for me)

1 Like

As mentioned above it is usually power related. Try a new decent power supply.

Thanks for the feedback on the power supplies and possible hardware issue on 39602. It currently has a 2.5 amp supply attached and I think I have a spare that I can try. If not, I purchased an new Pi 3B unit but have to do some tweaking to have it communicate through the corporate firewall.

Tried to swap it out by changing out the SD card but it wouldn’t link up to the wireless. Will have to see what’s causing that if the unit needs changed.

Lowell

OBJ, I changed out the power supply around 17:00 Z yesterday and haven’t seen a loss of MLAT when I’ve checked. Is there a way that I can look at the historical logs you mentioned to verify the problem is fixed? I’m guessing if the clock stability doesn’t show up over the weekend, the power supply was at fault.

Thanks for your help with this issue!

Lowell

1 Like

cat /var/log/piaware.log
This will output log of last few days.

Just wanted to say Thanks to kevdel and ScotsSharer for the power supply recommendation to correct the clock instablitiy. Made the swap on Friday around 17:00 Z and unit had no issues over this past weekend.

Lowell

2 Likes

I wanted to followup on this previous squawk about Server Clock Instability and Loss of MLAT. The Pi 3B unit that was having the problem finally failed on Friday Feb 23rd causing an outage for about 10 hours. I replaced it with an Element14 3B unit and thought it had been the culprit all along but today I’m having Clock Instability on the new unit as well.

Power supplies were changed and upgraded to a 3.0 Amp unit in early January. It appears that once aircraft counts approach 300 the clock starts to become erratic.

Does anyone have other suggestions? I’m thinking about trying a unit from a different Pi manufacturer. Could this be software related?

Lowell

Resolution to Server Clock Instability. After much discussion and interaction with ADS-B Tech Support, the problem was the actual electrical circuit that the RPi was powered from. New ProSticks, Pi units, power supplies, secondary grounding of the antenna lead, different filter options, had no effect.

I happened to notice that we lost sync when a roll-up garage door was opened that’s powered by the same circuit. The opener and an emergency light fixture are the only known units powered by that circuit and the door is seldom used during cold winter temperatures.

I moved the power supply to a different electric circuit and the sync problems have gone away. The hangar is a 39 year old building but the new power source was added when we converted from 208 VAC 3 Phase to 480 VAC 3 Phase about 2 years ago. In hindsight, the problems started shortly after that installation.

Thanks again to all who offered comments during this troubleshooting process!

Lowell

3 Likes

This is an interesting observation. I have been fighting the same thing with one of three systems. I have even moved hardware around trying to solve it. I think the problem system shares a circuit with an air compressor that runs intermittently. When I reboot the Pi MLAT works for a while (a few hours to just under a day) and then looses sync and reports clock instability. I upgraded the power supply to a 3A unit, but did not think about other large loads on the circuit that might be affecting it.