piaware issues today - having trouble with image & dump1090


#1

Been a piaware user a couple of months now. Things have been pretty rock solid - until today.

I’m seeing all sorts of dump1090 and faup1090 disconnects. I couldn’t figure it out. Since my install is from the pre-image days, I wasn’t sure if maybe I had a deb package or something go haywire on me. So I decided to download the official piaware image and start fresh.

After installing the image, piaware can see that dump1090 is available, but it cannot connect. faup1090 is also not connecting.

Thoughts?


pi@piaware ~ $ sudo piaware-status
dump1090 is running.
faup1090 is not running.
piaware is running.
dump1090 is listening for connections on port 30005.
dump1090 is listening for connections on port 10001.
piaware is NOT connected to port 10001.
piaware is connected to FlightAware.


10/23/2014 01:32:39 ****************************************************
10/23/2014 01:32:39 piaware version 1.15 is running, process ID 4284
10/23/2014 01:32:39 your system info is: Linux piaware 3.12.30+ #717 PREEMPT Fri Oct 17 18:46:31 BST 2014 armv6l GNU/Linux
10/23/2014 01:32:39 connecting to FlightAware eyes.flightaware.com/1200
10/23/2014 01:32:40 FlightAware server SSL certificate validated
10/23/2014 01:32:40 encrypted session established with FlightAware
10/23/2014 01:32:45 ADS-B data program 'dump1090' is listening on port 30005, so far so good
10/23/2014 01:32:45 i see dump1090 serving on port 10001
10/23/2014 01:32:45 connecting to dump1090 on port 10001...
10/23/2014 01:34:52 error opening connection to dump1090 : couldn't open socket: connection timed out, retrying in 10s...
10/23/2014 01:34:52 dump1090 is listening for connections on FA-style port 10001
10/23/2014 01:34:52 logged in to FlightAware as user drsprite
10/23/2014 01:34:52 server is sending alive messages; we will expect them
10/23/2014 01:35:02 ADS-B data program 'dump1090' is listening on port 30005, so far so good
10/23/2014 01:35:02 i see dump1090 serving on port 10001
10/23/2014 01:35:02 connecting to dump1090 on port 10001...
10/23/2014 01:37:10 error opening connection to dump1090 : couldn't open socket: connection timed out, retrying in 60s...
10/23/2014 01:37:10 0 msgs recv'd from dump1090; 0 msgs sent to FlightAware
10/23/2014 01:38:10 ADS-B data program 'dump1090' is listening on port 30005, so far so good
10/23/2014 01:38:10 i see dump1090 serving on port 10001
10/23/2014 01:38:10 connecting to dump1090 on port 10001...



#2

The key thing here I am pretty sure is “connection timed out”. I believe this means that dump1090 is hung. Dump1090 is a pretty amazing program and it does a phenomenal amount of stuff just in that one C program. Let’s try restarting dump1090. You said you’re not running the image. Either way…


sudo /etc/init.d/fadump1090.sh restart

…might fix it.

If you’re feeling intrepid you might try to attach the process with gdb (“gdb /usr/bin/dump1090 pid” where pid is the process id of dump1090) and get a backtrace (“bt”) and paste us the output.

You might also check /var/log/messages and /var/log/syslog and see if there’s anything strange in there, particularly mmcblk errors which would indicate corruption of the SD card. You might try rebooting because, you know, sometimes that helps, even if it shouldn’t.

Since you’re running 1.15 then after an hour piaware should attempt to restart dump1090 on its own, but this is brand new code and it may not be working in all cases.

Please let us know what you find out, and thanks for playing!


#3

I am running the image now. After dump1090 died on me, I decided to go to to the image.

I tried running


sudo /etc/init.d/fadump1090.sh restart

Here’s what I have:


pi@piaware ~ $ sudo piaware-status
dump1090 is running.
faup1090 is not running.
piaware is running.
dump1090 is listening for connections on port 30005.
dump1090 is listening for connections on port 10001.
piaware is NOT connected to port 10001.
piaware is connected to FlightAware.

And, more connection timed out errors


10/23/2014 12:04:05 error opening connection to dump1090 : couldn't open socket: connection timed out, retrying in 60s...
10/23/2014 12:05:06 ADS-B data program 'dump1090' is listening on port 30005, so far so good
10/23/2014 12:05:06 i see dump1090 serving on port 10001
10/23/2014 12:05:06 connecting to dump1090 on port 10001...


Only thing in the syslog is DHCP requests, and in /var/log/messages shows nothing new since my reboot 10 hours ago.

Here’s my attempt at the gdb, let me know if I’ve done it incorrectly.


pi@piaware ~ $ sudo gdb /usr/bin/dump1090 8765
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/dump1090...done.
Attaching to program: /usr/bin/dump1090, process 8765
Reading symbols from /usr/lib/arm-linux-gnueabihf/libcofi_rpi.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/arm-linux-gnueabihf/libcofi_rpi.so
Reading symbols from /usr/lib/librtlsdr.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/librtlsdr.so.0
Reading symbols from /lib/arm-linux-gnueabihf/libusb-1.0.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/arm-linux-gnueabihf/libusb-1.0.so.0
Reading symbols from /lib/arm-linux-gnueabihf/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
[New Thread 0xb6cb4470 (LWP 8779)]
Loaded symbols for /lib/arm-linux-gnueabihf/libpthread.so.0
Reading symbols from /lib/arm-linux-gnueabihf/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/arm-linux-gnueabihf/libm.so.6
Reading symbols from /lib/arm-linux-gnueabihf/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/arm-linux-gnueabihf/libc.so.6
Reading symbols from /lib/ld-linux-armhf.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux-armhf.so.3
Reading symbols from /lib/arm-linux-gnueabihf/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/arm-linux-gnueabihf/libgcc_s.so.1
Reading symbols from /lib/arm-linux-gnueabihf/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/arm-linux-gnueabihf/librt.so.1
0x0000f6a0 in detectModeS (m=0x0, mlen=452) at mode_s.c:1583
1583    mode_s.c: No such file or directory.
(gdb) bt
#0  0x0000f6a0 in detectModeS (m=0x0, mlen=452) at mode_s.c:1583
#1  0x0000a554 in main (argc=<optimized out>, argv=<optimized out>) at dump1090.c:883
(gdb)




#4

In an attempt to keep troubleshooting, I received the following error.

  1. I stopped /etc/init.d/fadump1090.sh
  2. I stopped /etc/init.d/piaware
  3. I started /etc/init.d/fadump1090.sh
  4. I started /etc/init.d/piaware
  5. /tmp/piaware.out shows this:

10/23/2014 17:54:51 logged in to FlightAware as user drsprite
can't read "::netstatus(program_30005)": no such element in array
    while executing
"if {$::netstatus(program_30005) == $::netstatus(program_10001)} {
                        set who "$::netstatus(program_30005)"
                } else {
                        set who "$::netstatus(program..."
    (procedure "traffic_report" line 7)
    invoked from within
"traffic_report"
    ("after" script)




#5

A reboot may have cleared that error, however I cannot get piaware to work with the image.


10/23/2014 18:13:18 i see dump1090 serving on port 10001
10/23/2014 18:13:18 connecting to dump1090 on port 10001...
10/23/2014 18:15:26 error opening connection to dump1090 : couldn't open socket: connection timed out, retrying in 10s...
10/23/2014 18:15:26 dump1090 is listening for connections on FA-style port 10001
10/23/2014 18:15:26 logged in to FlightAware as user drsprite
10/23/2014 18:15:26 server is sending alive messages; we will expect them
10/23/2014 18:15:36 ADS-B data program 'dump1090' is listening on port 30005, so far so good
10/23/2014 18:15:36 i see dump1090 serving on port 10001
10/23/2014 18:15:36 connecting to dump1090 on port 10001...



#6

I think I had a bad SD Card. I got home from work, re-imaged a new one and things seem OK for now. I’ll let it sit a little while longer and we’ll see how it goes.


#7

OK, thanks, Dr. Sprite!