Debian Bookworm kills piaware.service minutes after starting

jokessler · January 3, 2023, 6:24pm

Hi there,

I am having trouble with piaware 8.2 after the update it gets killed for some reason.
Dump1090 works fine and I have a nice map with planes locally, but I am unable to feed to FlightAware for some reason.

Systemctl Staus output is below and I am on arm64 Debian Bookworm

● piaware.service - FlightAware ADS-B uploader
     Loaded: loaded (/lib/systemd/system/piaware.service; enabled; preset: enabled)
     Active: active (running) since Tue 2023-01-03 18:05:29 GMT; 2s ago
       Docs: https://flightaware.com/adsb/piaware/
   Main PID: 1029 (piaware)
      Tasks: 2 (limit: 1046)
     Memory: 5.7M
        CPU: 933ms
     CGroup: /system.slice/piaware.service
             └─1029 /usr/bin/piaware -p /run/piaware/piaware.pid -plainlog -statusfile /run/piaware/status.json

Jan 03 18:05:29 ADSB-Receiver systemd[1]: Started FlightAware ADS-B uploader.
Jan 03 18:05:29 ADSB-Receiver piaware[1029]: creating pidfile /run/piaware/piaware.pid
Jan 03 18:05:29 ADSB-Receiver piaware[1029]: ****************************************************
Jan 03 18:05:29 ADSB-Receiver piaware[1029]: piaware version 8.2 is running, process ID 1029
Jan 03 18:05:29 ADSB-Receiver piaware[1029]: your system info is: Linux ADSB-Receiver 6.0.0-6-arm64 #1 SMP Debian 6.0.12-1 (2022-12-09) aarch64 GNU/Linux
Jan 03 18:05:29 ADSB-Receiver piaware[1029]: GPS: Connected to gpsd 3.22 at localhost/2947
Jan 03 18:05:30 ADSB-Receiver piaware[1029]: gpsd reported location: 50.0000, 0.00000, 24m (WGS84)
Jan 03 18:05:31 ADSB-Receiver piaware[1029]: Connecting to FlightAware adept server at piaware.flightaware.com/1200
Jan 03 18:05:31 ADSB-Receiver piaware[1029]: Connection with adept server at piaware.flightaware.com/1200 established

minutes later

● piaware.service - FlightAware ADS-B uploader
     Loaded: loaded (/lib/systemd/system/piaware.service; enabled; preset: enabled)
     Active: activating (auto-restart) (Result: signal) since Tue 2023-01-03 18:10:32 GMT; 21s ago
       Docs: https://flightaware.com/adsb/piaware/
    Process: 1089 ExecStart=/usr/bin/piaware -p /run/piaware/piaware.pid -plainlog -statusfile /run/piaware/status.json (code=killed, signal=ABRT)
   Main PID: 1089 (code=killed, signal=ABRT)
        CPU: 1.060s

Anybody else got this wierd issue?
This also affects piaware 7.2 as well, because it was the reason why I upgrade to 8.2 as I thought it might fix the problem.

abcd567 · January 3, 2023, 6:53pm

Yes, but on AMD64

I installed Debian12 (Bookworm)Testing on my VM using Alpha-1 netinstaller (debian-bookworm-DI-alpha1-amd64-netinst.iso), and installed dump1090-fa 8.2 and piaware 8.2.
Results were identical to yours.

Yesterday I removed the Debian 12 Bookworm Testing OS. Downloaded new copy of Alpha-1 netinstaller iso, then installed it again using new copy of Alpha-1 netinstaller. Next I installed dump1090-fa 8.2 and piaware 8.2. Got the exact same failure again.

Finally uninstalled Debian12 Bookworm Testing from VM

jokessler · January 3, 2023, 7:26pm

I have checked with Journalctl and this is on constant repeat

Jan 03 19:18:42 ADSB-Receiver systemd[1]: Started FlightAware ADS-B uploader.
Jan 03 19:18:42 ADSB-Receiver systemd[1]: piaware.service: Consumed 1.082s CPU time.
Jan 03 19:18:42 ADSB-Receiver systemd[1]: Stopped FlightAware ADS-B uploader.
Jan 03 19:18:42 ADSB-Receiver systemd[1]: piaware.service: Scheduled restart job, restart counter is at 171.
Jan 03 19:18:11 ADSB-Receiver systemd[1]: piaware.service: Consumed 1.082s CPU time.
Jan 03 19:18:11 ADSB-Receiver systemd[1]: piaware.service: Failed with result 'signal'.
Jan 03 19:18:11 ADSB-Receiver systemd[1]: piaware.service: Main process exited, code=killed, status=6/ABRT
Jan 03 19:18:11 ADSB-Receiver piaware[1866]: SpellFix: programming error
Jan 03 19:18:11 ADSB-Receiver piaware[1866]: TLS handshake with adept server at piaware.flightaware.com/1200 completed
Jan 03 19:18:11 ADSB-Receiver piaware[1866]: Connection with adept server at piaware.flightaware.com/1200 established
Jan 03 19:18:10 ADSB-Receiver piaware[1866]: Connecting to FlightAware adept server at piaware.flightaware.com/1200
Jan 03 19:18:10 ADSB-Receiver piaware[1866]: gpsd reported location: 50.00000, 0.00000, 10m (WGS84)
Jan 03 19:18:09 ADSB-Receiver piaware[1866]: GPS: Connected to gpsd 3.22 at localhost/2947
Jan 03 19:18:09 ADSB-Receiver piaware[1866]: your system info is: Linux ADSB-Receiver 6.0.0-6-arm64 #1 SMP Debian 6.0.12-1 (2022-12-09) aarch64 GNU/Linux
Jan 03 19:18:09 ADSB-Receiver piaware[1866]: piaware version 8.2 is running, process ID 1866
Jan 03 19:18:09 ADSB-Receiver piaware[1866]: ****************************************************
Jan 03 19:18:09 ADSB-Receiver piaware[1866]: creating pidfile /run/piaware/piaware.pid

Does piaware has a log file which I could look into?

abcd567 · January 3, 2023, 8:12pm

jokessler:

Jan 03 19:18:11 ADSB-Receiver systemd[1]: piaware.service: Failed with result 'signal'.
Jan 03 19:18:11 ADSB-Receiver systemd[1]: piaware.service: Main process exited, code=killed, status=6/ABRT
Jan 03 19:18:11 ADSB-Receiver piaware[1866]: SpellFix: programming error

Unfortunately I have deleted the Bookworm from my VM on Windows, so I cannot post journalctl output, but I remember to get similar constant repeats when ran journalctl -u piaware , and following Error messages

piaware.service: Main process exited, code=killed, status=6/ABRT
SpellFix: programming error

I dont know what is cause of this failure.
Flightaware staff @obj and @eric1tran may shed some light.

eric1tran · January 3, 2023, 8:26pm

Relevant information here: piaware crashes with itcl3 3.4.4 · Issue #94 · flightaware/piaware · GitHub

We’re looking into fixing this.

jokessler · January 3, 2023, 10:15pm

Thank you very much indeed.

I will keep an eye on the github repo for piaware, so that I can build new packages for my 2 FlightAware feeder sites.

abcd567 · January 3, 2023, 10:28pm

Currently I am using Debian11.6 (Bullseye) AMD64, with Piaware v8.2, without any issue, to feed one of my stations.

When Bookworm SID’s issue will be solved by Flightaware, I will switch to it

By the way, to build the piaware and dump1090-fa packages from Flightaware’s source code, I use the automated install script, which I have written and posted on Github. It makes life easy for repeatitive job.

https://github.com/abcd567a/piaware-ubuntu-debian-amd64

obj · January 4, 2023, 3:17am

Well, we can work around this specific issue, but the breakage is upstream so we’re limited in how much can be fixed.

It’d be appreciated if someone running bookworm and seeing the issue can submit a Debian bug report against the itcl debian package (the breakage is in itcl 3.4.4). Here is a bug report against the original itcl itself: Itcl - the [incr Tcl] extension: View Ticket

abcd567 · January 4, 2023, 7:00am

Reported Bug to Debian

jokessler · January 4, 2023, 6:13pm

Submitted to Debian with details below.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1027932

Thank you for filing a new Bug report with Debian.

You can follow progress on this Bug here: 1027932: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1027932.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 Debian Tcl/Tk Packagers <pkg-tcltk-devel@lists.alioth.debian.org>

If you wish to submit further information on this problem, please
send it to 1027932@bugs.debian.org.

Please do not send mail to owner@bugs.debian.org unless you wish
to report a problem with the Bug-tracking system.

-- 
1027932: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1027932
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems

jokessler · January 5, 2023, 1:56pm

As a interim workaround for all those who are on Debian Bookworm,
add “bullseye main” to /etc/apt/sources.list and then install itcl3 3.4.3 with “sudo apt install itcl3=3.4.3-3.1” now mark itcl3 as hold with “sudo apt-mark hold itcl3” to prevent apt from upgrading itcl3.
Piaware is now worinkg again, but without MLAT as the workaround breaks the fa-mlat-client.

obj · January 5, 2023, 2:40pm

Huh, that is very weird…

jokessler · January 5, 2023, 2:55pm

output from systemctl status piaware.service:

● piaware.service - FlightAware ADS-B uploader
     Loaded: loaded (/lib/systemd/system/piaware.service; enabled; preset: enabled)
     Active: active (running) since Thu 2023-01-05 13:45:45 GMT; 1h 8min ago
       Docs: https://flightaware.com/adsb/piaware/
   Main PID: 433 (piaware)
      Tasks: 3 (limit: 1046)
     Memory: 21.2M
        CPU: 5min 53.523s
     CGroup: /system.slice/piaware.service
             ├─433 /usr/bin/piaware -p /run/piaware/piaware.pid -plainlog -statusfile /run/piaware/status.json
             └─471 /usr/lib/piaware/helpers/faup1090 --net-bo-ipaddr localhost --net-bo-port 30005 --stdout --lat 50.791 --lon 0.325

Jan 05 14:54:10 ADSB-Receiver piaware[433]: mlat-client(929): Traceback (most recent call last):
Jan 05 14:54:10 ADSB-Receiver piaware[433]: mlat-client(929):   File "/home/.../piaware_builder/package-bullseye/debian/venv/lib/python3.10/site-packages/cx_Freeze-6.8.3-py3.10-linux-aarch64.egg/cx_Freeze/initscripts/__startup__.py", line 113, in run
Jan 05 14:54:10 ADSB-Receiver piaware[433]: mlat-client(929):     module_init.run(name + "__main__")
Jan 05 14:54:10 ADSB-Receiver piaware[433]: mlat-client(929):   File "/home/.../piaware_builder/package-bullseye/debian/venv/lib/python3.10/site-packages/cx_Freeze-6.8.3-py3.10-linux-aarch64.egg/cx_Freeze/initscripts/Console.py", line 15, in run
Jan 05 14:54:10 ADSB-Receiver piaware[433]: mlat-client(929):     exec(code, module_main.__dict__)
Jan 05 14:54:10 ADSB-Receiver piaware[433]: mlat-client(929):   File "/home/.../piaware_builder/package-bullseye/debian/venv/bin/fa-mlat-client", line 4, in <module>
Jan 05 14:54:10 ADSB-Receiver piaware[433]: mlat-client(929):     __import__('pkg_resources').run_script('MlatClient==0.2.12', 'fa-mlat-client')
Jan 05 14:54:10 ADSB-Receiver piaware[433]: mlat-client(929): ModuleNotFoundError: No module named 'pkg_resources'
Jan 05 14:54:10 ADSB-Receiver piaware[433]: got EOF from multilateration client
Jan 05 14:54:10 ADSB-Receiver piaware[433]: fa-mlat-client exited with EXIT 255

obj · January 5, 2023, 3:01pm

I think this is a cxfreeze build problem rather than something related to the itcl3 package

abcd567 · January 5, 2023, 3:23pm

~ $ ssh abcd@10.0.0.69
abcd@10.0.0.69's password:
Linux debian-12 6.0.0-6-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.0.12-1 (2022-12-09) x86_64
... ... ... ...
abcd@debian-12:~$ sudo systemctl status piaware
[sudo] password for abcd:
● piaware.service - FlightAware ADS-B uploader
     Loaded: loaded (/lib/systemd/system/piaware.service; enabled; preset: enabled)
     Active: active (running) since Thu 2023-01-05 09:58:27 EST; 27min ago
       Docs: https://flightaware.com/adsb/piaware/
   Main PID: 617 (piaware)
      Tasks: 3 (limit: 3520)
     Memory: 25.6M
        CPU: 5.407s
     CGroup: /system.slice/piaware.service
             ├─617 /usr/bin/piaware -p /run/piaware/piaware.pid -plainlog -statusfile /run/piaware/status.json
             └─662 /usr/lib/piaware/helpers/faup1090 --net-bo-ipaddr localhost --net-bo-port 30005 --stdout --lat 43.5xx --lon -79.6xx

Jan 05 10:25:02 debian-12 sudo[3346]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Jan 05 10:25:02 debian-12 sudo[3346]: pam_unix(sudo:session): session closed for user root
Jan 05 10:25:02 debian-12 piaware[617]: Starting multilateration client: /usr/lib/piaware/helpers/fa-mlat-client --input-connect localhost:30005 --input-type dump1090 --results beast,connect,localhost:30104 --results beast,listen,30105 --results ext_basestation,listen,30106 --udp-transport 2620:13d:c004:1>
Jan 05 10:25:02 debian-12 piaware[617]: mlat-client(3348): Traceback (most recent call last):
Jan 05 10:25:02 debian-12 piaware[617]: mlat-client(3348):   File "/home/abcd/piaware_builder/package-bullseye/debian/venv/lib/python3.10/site-packages/cx_Freeze-6.8.3-py3.10-linux-x86_64.egg/cx_Freeze/initscripts/__startup__.py", line 113, in run
Jan 05 10:25:02 debian-12 piaware[617]: mlat-client(3348):   File "/home/abcd/piaware_builder/package-bullseye/debian/venv/lib/python3.10/site-packages/cx_Freeze-6.8.3-py3.10-linux-x86_64.egg/cx_Freeze/initscripts/Console.py", line 15, in run
Jan 05 10:25:02 debian-12 piaware[617]: mlat-client(3348):   File "/home/abcd/piaware_builder/package-bullseye/debian/venv/bin/fa-mlat-client", line 4, in <module>
Jan 05 10:25:02 debian-12 piaware[617]: mlat-client(3348): ModuleNotFoundError: No module named 'pkg_resources'
Jan 05 10:25:02 debian-12 piaware[617]: got EOF from multilateration client
Jan 05 10:25:03 debian-12 piaware[617]: fa-mlat-client exited with EXIT 255

jokessler · January 6, 2023, 10:17am

Good news, Debian has fixed the SpellFix error in itcl 3.4.4-2


Source: itcl3
Source-Version: 3.4.4-2
Done: Sergei Golovan 

We believe that the bug you reported is fixed in the latest version of
itcl3, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1027932@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Sergei Golovan  (supplier of updated itcl3 package)

Changes:
 itcl3 (3.4.4-2) unstable; urgency=medium
 .
   * Revert the change which ports Itcl 3 to Tcl 8.7 because it breaks
     abbreviations in namespaces like [info exist var]
     (closes: #1027932, #1027857).

abcd567 · January 6, 2023, 10:48am

@obj
@eric1tran
@jokessler

Good News: Bug fixed in new version itcl3 3.4.4-2

Page 1 of 3:

Page 2 of 3:

Page 3 of 3:

jokessler · January 21, 2023, 8:12pm

cx_Freeze is at Version 6.13 since Nov.22 → https://github.com/marcelotduarte/cx_Freeze

Can I manually update cx_Freeze to 6.13 before building piaware?

I have seen in sensible-build.sh (line 154-156) that the script downloads cx_Freeze 6.8.3, but which file path to target?
Maybe this could fix the MLAT issue?

abcd567 · January 22, 2023, 1:05am

Bookworm uses Python 3.10, which is supported by cx_Freeze ver 6.13.1

I tried cx_Freeze ver 16.13.1, but build failed.

I think matching versions of zipp and importlib-metadata should also be used.

$ wget https://github.com/marcelotduarte/cx_Freeze/archive/refs/tags/6.13.1.tar.gz  

$ sha256sum 6.13.1.tar.gz
c302e15bc9c488704f0b79b3896101ae6fc81b405ce489c29b15937111bc18f3  6.13.1.tar.gz

Code below is from file `piaware_builder/sensible-build.sh`

    bullseye)
        # Bullseye has Python 3.9; cx_Freeze 6.8.3 supports this
        fetch_archive importlib_metadata-4.3.1 \
                      https://files.pythonhosted.org/packages/a4/8b/1d63614ef7ced52a7da2d40753968c40a4bbc14fd9c0ba85d612b44ffd9a/importlib_metadata-4.3.1.tar.gz \
                      2d932ea08814f745863fd20172fe7de4794ad74567db78f2377343e24520a5b6
        fetch_archive zipp-0.5.0 \
                      https://files.pythonhosted.org/packages/44/65/799bbac4c284c93ce9cbe67956a3625a4e1941d580832656bea202554117/zipp-0.5.0.tar.gz \
                      d7ac25f895fb65bff937b381353c14eb1fa23d35f40abd72a5342cd57eb57fd1
        fetch_archive cx_Freeze-6.8.3 \
                      https://github.com/anthony-tuininga/cx_Freeze/archive/6.8.3.tar.gz \
                      d39c59fdfc82106dfe1e5dce09f2537a3cc82dc8295024f40f639d94193979c3
        ;;

jokessler · January 22, 2023, 11:58am

yeah, I failed too.

But when you look in the folder package-bullseye there is a subfolder freeze-mlat-client and I don’t know there this comes from.
As I tried to manually extend sensible-build.sh with an entry for bookworm it wouldn’t download importlib, zipp and cf_Freeze at all.
So I downloaded the latest versions manually and the build failed as zipp-0.5.0 is hardcoded somethere and also a patch seems to be applied to importlib_metadata-4.3.1 and zipp-0.5.0, renaming the folders to match expected old versions does not work either.
Also downloading the old versions of zipp and importlib_metadata and just updating cx_Freeze does not compile either, even with renaming cx_Freeze to match 6.8.3, it does not compile.

I opend an issue on github against piaware-builder for this https://github.com/flightaware/piaware_builder/issues/23

Topic		Replies	Views
PiAware stopped reporting ADS-B Flight Tracking	6	233	April 23, 2024
FlightAware Incompatible With Bookworm? FlightAware	14	1172	June 12, 2023
About 2 weeks ago, my feeder stopped working. It still says connected to Flightaware ADS-B Flight Tracking	10	682	January 31, 2023
PiAware stopped feeding, local site OK ADS-B Flight Tracking	14	798	February 4, 2022
Dump1090 fails to start ADS-B Flight Tracking	11	3418	September 3, 2018