PiAware 3.6.2 - *** buffer overflow detected ***


#1

Hi,

I recently updated my RasPi to the latest version (“sudo apt-get update && sudo apt-get upgrade”) and am now getting errors in /var/log/piaware.log:

Sep 8 19:25:18 piaware01 piaware[2230]: piaware version 3.6.2 is running, process ID 2230
Sep 8 19:25:18 piaware01 piaware[2230]: your system info is: Linux piaware01 4.14.62-v7+ #1134 SMP Tue Aug 14 17:10:10 BST 2018 armv7l GNU/Linux
Sep 8 19:25:18 piaware01 piaware[2230]: *** buffer overflow detected ***: /usr/bin/piaware terminated

I tried a re-install with the April 18th version of Raspian-Stretch-Lite, which worked just fine on my hardware until a fresh OS update killed PiAware again.

Any tips to debug and address the issue would be greatly appreciated !


#2

What are the exact steps you took to install the OS and PiAware?


#3

I did the following:

  1. Using Win32DiskImager burned a 32GB micro SD card with the April release of Raspbian lite,
  2. Followed the steps 2-4 on https://flightaware.com/adsb/piaware/install,
  3. After the reboot checked /var/log/piaware.log and saw no errors,
  4. Checked https://flightaware.com/adsb/stats/user/shol and got a green status for my feeder,
  5. Did "sudo apt-get update && sudo apt-get upgrade”, followed by a reboot.

After that PiAware is giving the buffer overflow error…
I’m using a RasPi power supply, a Samsung microSD card with a RasPi 3B+ and a FlightAware Pro Stick.


#4

You mean this image? https://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2018-04-19/
Does it work if you use the latest (27 June) image directly?
I’ll try to reproduce the problem here.


#5

Hm, can’t reproduce it with those steps on 3B+ here. Post-upgrade, piaware is fine:

Sep  8 22:42:21 raspberrypi piaware[498]: creating pidfile /run/piaware/piaware.pid
Sep  8 22:42:21 raspberrypi piaware[498]: ****************************************************
Sep  8 22:42:21 raspberrypi piaware[498]: piaware version 3.6.2 is running, process ID 498
Sep  8 22:42:21 raspberrypi piaware[498]: your system info is: Linux raspberrypi 4.14.62-v7+ #1134 SMP Tue Aug 14 17:10:10 BST 2018 armv7l GNU/Linux
Sep  8 22:42:22 raspberrypi piaware[498]: Connecting to FlightAware adept server at piaware.flightaware.com/1200
Sep  8 22:42:23 raspberrypi piaware[498]: Connection with adept server at piaware.flightaware.com/1200 established
Sep  8 22:42:23 raspberrypi piaware[498]: TLS handshake with adept server at piaware.flightaware.com/1200 completed
Sep  8 22:42:23 raspberrypi piaware[498]: FlightAware server certificate validated
Sep  8 22:42:23 raspberrypi piaware[498]: encrypted session established with FlightAware
Sep  8 22:42:24 raspberrypi piaware[498]: logged in to FlightAware as user guest
[...]

I didn’t see any upgrades to packages that piaware / tcl depend on when doing the upgrade either so I’m not sure what’s going on with your system.

Next step would probably be for you to run piaware under gdb and/or valgrind to see where exactly it’s crashing:

gdb:

$ sudo service piaware stop
$ sudo -s -u piaware gdb /usr/bin/piaware
[...]
(gdb) run
  # wait for it to crash, record the output
(gdb) bt
  # record the output
(gdb) quit

Valgrind:

$ sudo apt-get install valgrind
$ sudo service piaware stop
$ sudo -s -u piaware valgrind /usr/bin/piaware
    # be patient, it's slow, capture the output

#6

Yes, the image dated 2018-04-18.
The 2018-06-27 image gave me the buffer overflow error.

I’ll now try the debugging steps you suggested.
Thanks for your help!


#7

Hi,

Here’s the gdb output:

  1. (gdb) run
    Starting program: /usr/bin/piaware
    [Thread debugging using libthread_db enabled]
    Using host libthread_db library “/lib/arm-linux-gnueabihf/libthread_db.so.1”.
    2018-09-09 05:26:28Z failed to reopen /var/log/piaware.log: couldn’t open “/var/log/piaware.log”: permission denied
    2018-09-09 05:26:28Z ****************************************************
    2018-09-09 05:26:28Z piaware version 3.6.2 is running, process ID 30378
    2018-09-09 05:26:28Z your system info is: Linux piaware01 4.14.62-v7+ #1134 SMP Tue Aug 14 17:10:10 BST 2018 armv7l GNU/Linux
    [New Thread 0x767a5470 (LWP 30389)]
    *** buffer overflow detected ***: /usr/bin/piaware terminated

Thread 1 “piaware” received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at …/sysdeps/unix/sysv/linux/raise.c:51
51 …/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb)

  1. (gdb) bt
    #0 __GI_raise (sig=sig@entry=6) at …/sysdeps/unix/sysv/linux/raise.c:51
    #1 0x76cf6824 in __GI_abort () at abort.c:89
    #2 0x76d2ff78 in __libc_message (do_abort=do_abort@entry=2, fmt=) at …/sysdeps/posix/libc_fatal.c:175
    #3 0x76dac1c4 in __GI___fortify_fail (msg=0x76debc28 “buffer overflow detected”) at fortify_fail.c:30
    #4 0x76daa29c in __GI___chk_fail () at chk_fail.c:28
    #5 0x76dac13c in __fdelt_chk (d=) at fdelt_chk.c:25
    #6 0x76f6b68c in Tcl_CreateFileHandler () from /usr/lib/arm-linux-gnueabihf/libtcl8.6.so
    #7 0x76f68e24 in ?? () from /usr/lib/arm-linux-gnueabihf/libtcl8.6.so
    Backtrace stopped: previous frame identical to this frame (corrupt stack?)
    (gdb)

  2. pi@piaware01:~ $ sudo -s -u piaware valgrind /usr/bin/piaware

==30746== Memcheck, a memory error detector
==30746== Copyright © 2002-2017, and GNU GPL’d, by Julian Seward et al.
==30746== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==30746== Command: /usr/bin/piaware
==30746==
==30746== Use of uninitialised value of size 4
==30746== at 0x401AC38: strcpy (strcpy.S:123)
==30746== by 0x400AEAF: _dl_lookup_symbol_x (dl-lookup.c:876)
==30746== by 0x4B31443: do_sym (dl-sym.c:168)
==30746== by 0x4B318F7: _dl_sym (dl-sym.c:273)
==30746== by 0x4B67CBF: dlsym_doit (dlsym.c:50)
==30746== by 0x4010CDF: _dl_catch_error (dl-error.c:187)
==30746== by 0x4B68293: _dlerror_run (dlerror.c:163)
==30746== by 0x4B67D1F: dlsym (dlsym.c:70)
==30746== by 0x49C6FC3: ??? (in /usr/lib/arm-linux-gnueabihf/libtcl8.6.so)
==30746==
==30746== Use of uninitialised value of size 4
==30746== at 0x401AC44: strcpy (strcpy.S:128)
==30746== by 0x400AEAF: _dl_lookup_symbol_x (dl-lookup.c:876)
==30746== by 0x4B31443: do_sym (dl-sym.c:168)
==30746== by 0x4B318F7: _dl_sym (dl-sym.c:273)
==30746== by 0x4B67CBF: dlsym_doit (dlsym.c:50)
==30746== by 0x4010CDF: _dl_catch_error (dl-error.c:187)
==30746== by 0x4B68293: _dlerror_run (dlerror.c:163)
==30746== by 0x4B67D1F: dlsym (dlsym.c:70)
==30746== by 0x49C6F5B: ??? (in /usr/lib/arm-linux-gnueabihf/libtcl8.6.so)
==30746==
2018-09-09 05:30:25Z failed to reopen /var/log/piaware.log: couldn’t open “/var/log/piaware.log”: permission denied
2018-09-09 05:30:26Z ****************************************************
2018-09-09 05:30:26Z piaware version 3.6.2 is running, process ID 30746
2018-09-09 05:30:26Z your system info is: Linux piaware01 4.14.62-v7+ #1134 SMP Tue Aug 14 17:10:10 BST 2018 armv7l GNU/Linux
*** buffer overflow detected ***: /usr/bin/piaware terminated
==30746==
==30746== Process terminating with default action of signal 6 (SIGABRT)
==30746== at 0x4A5345C: raise (raise.c:51)
==30746== by 0x4A54823: abort (abort.c:89)
==30746== by 0x4A8DF77: __libc_message (libc_fatal.c:175)
==30746== by 0x4B0A1C3: __fortify_fail (fortify_fail.c:30)
==30746== by 0x4B0829B: __chk_fail (chk_fail.c:28)
==30746== by 0x4B0A13B: __fdelt_chk (fdelt_chk.c:25)
==30746== by 0x49B268B: Tcl_CreateFileHandler (in /usr/lib/arm-linux-gnueabihf/libtcl8.6.so)
==30746==
==30746== HEAP SUMMARY:
==30746== in use at exit: 3,183,829 bytes in 237 blocks
==30746== total heap usage: 465 allocs, 228 frees, 6,074,438 bytes allocated
==30746==
==30746== LEAK SUMMARY:
==30746== definitely lost: 0 bytes in 0 blocks
==30746== indirectly lost: 0 bytes in 0 blocks
==30746== possibly lost: 2,752,777 bytes in 171 blocks
==30746== still reachable: 431,052 bytes in 66 blocks
==30746== suppressed: 0 bytes in 0 blocks
==30746== Rerun with --leak-check=full to see details of leaked memory
==30746==
==30746== For counts of detected and suppressed errors, rerun with: -v
==30746== Use --track-origins=yes to see where uninitialised values come from
==30746== ERROR SUMMARY: 5 errors from 2 contexts (suppressed: 18 from 10)
Aborted
pi@piaware01:~ $

Seeing the errors opening the log file, I checked that it is being written to:

pi@piaware01:~ ls -l /var/log/piaware.log -rw-r----- 1 root adm 797854 Sep 9 07:23 /var/log/piaware.log pi@piaware01:~


#8

Yeah, the log file errors are normal when running like that.

The crash is in core tcl, it’s not specific to piaware. It smells familar, see here: problem with piaware 3.1.0

Have you done anything that would disable ipv6?

edit: I can reproduce your crash on current raspbian/piaware if I blacklist ipv6, as mentioned in the topic linked above, so it hasn’t been fixed upstream in the meantime. Workaround is the same: reenable IPv6 or comment out the IPv6 addresses in /etc/hosts.


#9

Ooops. Guilty as charged.

I’d added inserting “ipv6.disable=1” into /boot/cmdline.txt as part of my “standard” setup for a fresh RasPi.
Having reversed this to re-enable ipv6 on both my PiAware boxes, they’re now running smoothly…

Thanks for the tremendous support.


#10

In case you had that for fr24 because they can’t get their ipv6 problem fixed, i have just hardwired it in /etc/hosts:
104.20.1.101 feed.flightradar24.com
(check ping -4 feed.flightradar24.com if the ip is the same for you).
Of course that means you will need to change it if the IP should change.

But i prefer it over disabling it because they can’t fix their feeder.


#11

Thanks for the note.
Fr24 was working all the while w/o IPv6, so I disabled IPv6 again and followed obj’s 2nd option of commenting out IPv6 addresses in /etc/hosts. Now I have PiAware, Fr24 and Pf working fine w/o IPv6, which suits me fine :wink:


#12

For FR24 IPv6 problem, I use following method. This method does NOT totally block IPv6, but changes preference to IPv4.

sudo nano /etc/gai.conf 

In this file scroll down to find following:

....
....
#    For sites which prefer IPv4 connections change the last line to
#
#precedence ::ffff:0:0/96 100
....
....

Uncomment the line
#precedence ::ffff:0:0/96 100
so it looks like:
precedence ::ffff:0:0/96 100

Now Pi will prefer IPv4

Restart fr24feed

sudo systemctl restart fr24feed

#13

Hi,
I also changed the gai.conf file as described, works very well