HOWTO: Airspy mini and Airspy R2: Piaware / dump1090-fa configuration

[carstenks]

@prog

This involves a lot of work (mostly testing) and there are many high priority things on my plate right now. This will have to wait.

If I understand it correct… You’re busy, and don’t give a shit, now that you sold xxx-units?

[prog]

@carstenks

If I understand it correct… You’re busy, and don’t give a shit, now that you sold xxx-units?

I think you still don’t get the nature of my involvement in ModeS/ADSB. ADSB is is probably 0.001% of the use cases. This doesn’t mean I don’t give a shit, this just means I won’t spend valuable time dedicated to the other 99.999% of the users to debug your stupid kernel problems.
Remember, using free software is a privilege, not a right. If this is too complicated for you, contact your distributor and ask for a refund.

[gtj0]

Well if airspy_adsb were open-source…
j/k

In the mean time, if anyone’s interested, here’s a systemd service file and settings that will check for samples on startup and restart airspy_adsb until it starts spitting out samples. Feel free to use outright or incorporate into your own stuff.

airspy_adsb.service

[Unit]
Description=Airspy ADSB Decoder

[Service]
Type=simple
Restart=always
RestartSec=3
TimeoutStartSec=30
EnvironmentFile=/etc/default/airspy_adsb
# Note that the parameters in ExecStart do NOT have {} because we want them passed to airspy_adsb
# as mutliple arguments
ExecStart=/usr/bin/airspy_adsb $DEVICE $PUSH $LISTEN $GAIN $SAMPLE_RATE $FIX $PREAMBLE $TIMEOUT $WHITELIST $OPTS
# ExecStartPost DOES use the {} because we want the entire watchdog command line passed to bash as 1 argument.
ExecStartPost=/usr/bin/bash -e -c ${WATCHDOG_COMMAND}

[Install]
WantedBy=multi-user.target

/etc/default/airspy_adsb

PUSH=-c localhost:30004:beast
GAIN=-g 21
SAMPLE_RATE=-m 24
WHITELIST=-w 1
TIMEOUT=-t 60
FIX=-f 1
PREAMBLE=-e 8
OPTS=-p -v

# Startup Watchdog
# Pick a free port
LISTEN=-l 30099:AVR-STRICT
# The watchdog command will ...
#   try to connect to the port 10 times at 3 second intervals.
#   stay connected until 256 bytes have been read or 5 seconds of no activity has passed.
#   any data read is passed to the test.
#   the test runs 'wc -l' and checks that at least 5 lines have been read.
#   return an error if less than 5 lines have been read.
# Do NOT enclose the command in any kind of quotes
WATCHDOG_COMMAND=socat -u -T 5 TCP:localhost:30099,readbytes=256,retry=10,interval=3 - | test `wc -l` -ge 5
# Adjust the timeouts and counts to suit your environment.

# Disable the startup watchdog by setting the command to "true"
# WATCHDOG_COMMAND=true

[prog]

To focus the debate in a more pragmatic way, you can check the Airspy host tools and simulate the streaming. The airspy_adsb is no different. For example, you can use airspy_rx -r /dev/null -t 5 -a 24000000 then kill it and see what happens. If you need to debug the code, you can check GitHub - airspy/airspyone_host: AirSpy's usemode driver and associated tools

[gtj0]

Thanks. Unfortunately, I can’t make it fail using airspy_rx.

$ airspy_rx -p1 -f 1090 -a 24000000 -t 5 -r - -g 21 2>/dev/null | pipemeter > /dev/null
  34.31M/s  170.72M    8.00k   0:00:05

Also, while creating the watchdog I noticed that occasionally, I do see a few decoded messages on the output ports.

$ airspy_adsb -v -f 1 -g 20 -m 24 -c localhost:30004:avr -e8 -w1 -t60

This was collected over about 30 seconds:

$ nc -l 30004
*991766B69F8CE1E21FD7C386E6C8;
*8B1CDF69345724C78669C77A91A8;
*FBE85C63BD41CC5C98B2E12EB2E0;
*D59767763FBD5141EFDB30DA3BAF;

[prog]

The tests are not equivalent. One uses the packing and the other doesn’t.

[wiedehopf]

@gtj0

-c localhost:30004:avr

You have something listening for AVR on 30004 localhost?
Typically 30004 will want beast format.

[gtj0]

@prog

The tests are not equivalent. One uses the packing and the other doesn’t.

An oversight on the results I pasted. I tried several combinations of parameters. I’ll repeat the test though.

@wiedehopf

You have something listening for AVR on 30004 localhost?
Typically 30004 will want beast format.

Sure, normally it’s dump1090-fa in beast format. I just used avr so I could see the ascii text via netcat. I also tried beast and piped the output through xxd so I could see the actual data. Made no difference.

[gtj0]

airspy_test

#!/bin/bash

run_airspy_rx() {
	sample_rate=$1
	gain=$2
	packing=$3
	samples=$(( 5 * sample_rate ))
	cmd="airspy_rx -f 1090 -a ${sample_rate} -n ${samples} -p ${packing} -t 5 -r - -g ${gain}"
	echo " $cmd"
	start=`date +"%s%N"`
	bytes=`${cmd} 2>/dev/null | wc -c`
	end=`date +"%s%N"`
	elapsed=$(( (end - start) / 1000 / 1000 ))
	kbytes=$(( bytes / 1024 ))
	printf "   Bytes: %dM  Rate: %dMB/s " $(( kbytes / 1024 )) $(( kbytes / (elapsed / 1000) / 1000))
	[ $bytes -ge 10000000 ] && echo pass || echo fail
}

run_airspy_adsb() {
	sample_rate=$1
	gain=$2
	packing=$3
	[ $packing -eq 1 ] && dash_p="-p" || dash_p=""
	cmd="airspy_adsb -v -f 1 -g ${gain} -m $(( sample_rate / 1000000  )) -l 30099:avr ${dash_p} -e8 -w1 -t60"
	echo " $cmd"
	${cmd} 2>/dev/null &
	disown $!
	sleep 5
	( socat -u -T 2 TCP:localhost:30099,readbytes=1024,retry=5,interval=2 - | ( lines=`wc -l` ; echo -n "   Messages: $lines " ; test $lines -ge 10 )) && echo pass || echo fail
	kill $!
	wait $! 2>/dev/null
}

declare -a sr=(12000000 20000000 24000000)
declare -a gains=(18 19 20 21)

echo "Starting airspy_rx"
for (( i=0; i < 10; i++ )) ; do
	sample_rate=${sr[$(( i % 3 ))]}
	gain=${gains[$(( i % 3 ))]}
	packing=$(( i % 2 ))

	echo "Test: $((i+1))"
	run_airspy_rx $sample_rate $gain $packing
	echo
done
echo
echo "Starting airspy_adsb"
for (( i=0; i < 10; i++ )) ; do
	sample_rate=${sr[$(( i % 3 ))]}
	gain=${gains[$(( i % 3 ))]}
	packing=$(( i % 2 ))

	echo "Test: $((i+1))"
	run_airspy_adsb $sample_rate $gain $packing
	sleep 2
	echo
done

Results:

$ ./airspy_test
Starting airspy_rx
Test: 1
 airspy_rx -f 1090 -a 12000000 -n 60000000 -p 0 -t 5 -r - -g 18
   Bytes: 85M  Rate: 21MB/s pass

Test: 2
 airspy_rx -f 1090 -a 20000000 -n 100000000 -p 1 -t 5 -r - -g 19
   Bytes: 143M  Rate: 24MB/s pass

Test: 3
 airspy_rx -f 1090 -a 24000000 -n 120000000 -p 0 -t 5 -r - -g 20
   Bytes: 171M  Rate: 35MB/s pass

Test: 4
 airspy_rx -f 1090 -a 12000000 -n 60000000 -p 1 -t 5 -r - -g 18
   Bytes: 85M  Rate: 14MB/s pass

Test: 5
 airspy_rx -f 1090 -a 20000000 -n 100000000 -p 0 -t 5 -r - -g 19
   Bytes: 143M  Rate: 36MB/s pass

Test: 6
 airspy_rx -f 1090 -a 24000000 -n 120000000 -p 1 -t 5 -r - -g 20
   Bytes: 171M  Rate: 29MB/s pass

Test: 7
 airspy_rx -f 1090 -a 12000000 -n 60000000 -p 0 -t 5 -r - -g 18
   Bytes: 85M  Rate: 21MB/s pass

Test: 8
 airspy_rx -f 1090 -a 20000000 -n 100000000 -p 1 -t 5 -r - -g 19
   Bytes: 143M  Rate: 24MB/s pass

Test: 9
 airspy_rx -f 1090 -a 24000000 -n 120000000 -p 0 -t 5 -r - -g 20
   Bytes: 171M  Rate: 35MB/s pass

Test: 10
 airspy_rx -f 1090 -a 12000000 -n 60000000 -p 1 -t 5 -r - -g 18
   Bytes: 85M  Rate: 14MB/s pass


Starting airspy_adsb
Test: 1
 airspy_adsb -v -f 1 -g 18 -m 12 -l 30099:avr  -e8 -w1 -t60
   Messages: 41 pass

Test: 2
 airspy_adsb -v -f 1 -g 19 -m 20 -l 30099:avr -p -e8 -w1 -t60
   Messages: 0 fail

Test: 3
 airspy_adsb -v -f 1 -g 20 -m 24 -l 30099:avr  -e8 -w1 -t60
   Messages: 0 fail

Test: 4
 airspy_adsb -v -f 1 -g 18 -m 12 -l 30099:avr -p -e8 -w1 -t60
   Messages: 39 pass

Test: 5
 airspy_adsb -v -f 1 -g 19 -m 20 -l 30099:avr  -e8 -w1 -t60
   Messages: 40 pass

Test: 6
 airspy_adsb -v -f 1 -g 20 -m 24 -l 30099:avr -p -e8 -w1 -t60
   Messages: 41 pass

Test: 7
 airspy_adsb -v -f 1 -g 18 -m 12 -l 30099:avr  -e8 -w1 -t60
   Messages: 40 pass

Test: 8
 airspy_adsb -v -f 1 -g 19 -m 20 -l 30099:avr -p -e8 -w1 -t60
   Messages: 1 fail

Test: 9
 airspy_adsb -v -f 1 -g 20 -m 24 -l 30099:avr  -e8 -w1 -t60
   Messages: 40 pass

Test: 10
 airspy_adsb -v -f 1 -g 18 -m 12 -l 30099:avr -p -e8 -w1 -t60
   Messages: 0 fail

airspy_rx passed all 10 tests
airspy_adsb passed only 6 out of 10.

[wiedehopf]

@gtj0

airspy_rx passed all 10 tests
airspy_adsb passed only 6 out of 10.

That somehow suggests it might be using a different libairspy, i checked and the last version i have for airspy_adsb indeed uses a libairspy that’s a bit older.

Just because it’s really simple, i integrated the most current libairspy for testing:
https://github.com/wiedehopf/airspy-conf/raw/master/airspy_adsb-linux-arm.tgz

Or are you not on arm32?

[gtj0]

@wiedehopf

Or are you not on arm32?

Using x86_64 for the tests.

[wiedehopf]

@gtj0

Using x86_64 for the test

https://github.com/wiedehopf/airspy-conf/raw/master/airspy_adsb-linux-x86_64.tgz

Well here you go.

[gtj0]

Thanks for the new version but unfortunately, it didn’t help. Can you confirm the defaults for preamble, whitelist and timeout? I was also varying those parameters but saw no change in the results.

EDIT: I also tried interleaving the airspy_rx and airspy_adsb commands to see if maybe running airspy_rx first might improve the chances of airspy_adsb succeeding but that didn’t help either.

[wiedehopf]

@gtj0

Can you confirm the defaults for preamble, whitelist and timeout?

That really shouldn’t matter in regards to working or not.
Just don’t use the option then it will use the default.

airspy_adsb -h

Try again with the version i push, probably won’t change anything but i modified the compile options a bit.

Also checking the number of messages after 2 seconds seems extreme, give it some more time to produce messages, maybe 5 seconds?

[runahue]

I tested your version and i have a lot of lost samples.

I don’t have with original 1.85
Using a rasp 3b+ with this options:

-v -f 1 -b -e 8 -w 5 -p

– Logs begin at Wed 2020-10-07 18:20:51 CEST. –
oct 07 18:21:05 satnogs airspy_adsb[350]: /!\ Lost 98304 samples /!
oct 07 18:21:05 satnogs airspy_adsb[350]: /!\ Lost 98304 samples /!
oct 07 18:21:05 satnogs airspy_adsb[350]: /!\ Lost 98304 samples /!
oct 07 18:21:05 satnogs airspy_adsb[350]: /!\ Lost 98304 samples /!
oct 07 18:21:09 satnogs airspy_adsb[350]: Push client connected to localhost:30004 (beast)
oct 07 18:21:09 satnogs airspy_adsb[350]: Push client disconnected from localhost:30004 (beast)
oct 07 18:21:17 satnogs airspy_adsb[350]: /!\ Lost 98304 samples /!
oct 07 18:21:17 satnogs airspy_adsb[350]: /!\ Lost 98304 samples /!
oct 07 18:21:17 satnogs airspy_adsb[350]: /!\ Lost 98304 samples /!
oct 07 18:21:22 satnogs airspy_adsb[350]: Push client connected to localhost:30004 (beast)

[wiedehopf]

@runahue

I tested your version and i have a lot of lost samples.

It was meant for testing in regards to the described issue, not for actual use.

Slightly different compile options can influence performance.
Also that log of yours only shows a couple of lost samples on start of the service.

[gtj0]

airspy_adsb -h doesn’t display the defaults. IIRC your guide said the default for whitelist was 2 but you recommend 3. I get different results when I specify 2 vs letting it default to what should be the default of 2. This only affects the number of aircraft captured when it’s working and doesn’t change this test. Still knowing what the defaults are would help.

The latest compiled version doesn’t improve the failure rate.

The 2 second timeout only applies after the script waits 5 seconds before starting socat, and after socat establishes a connection. socat resets the timer every time it gets data. Regardless, changing it to 5 seconds didn’t improve the failure rate.

[wiedehopf]

@gtj0

default for whitelist

default is 3, i usually use 5.

@gtj0

didn’t improve the failure rate.

Yeah no clue what the issue is.

[gtj0]

@wiedehopf

default is 3, i usually use 5.

If I correctly understood the discussion when whitelist was introduced, if -w CRC valid messages are
received for an ICAO, that id is whitelisted for -t seconds. Yes?

EDIT: Or is it that -w CRC-valid messages have to be received within -t seconds?

[gtj0]

I finally got through all 1112 posts in the other thread and I think my questions above are answered.
Particularly this post…

Airspy ADS-B decoder

This is very close to what we are doing in airspy_adsb, except that the scale of “trust” is a linear function of the “unfixed bits”: DF11 with 1-bit correction gives you 1 point DF11 without correction gives you 2 points DF17 with 1-bit correction gives you 2 points DF17 with 2-bit correction gives you 1 point DF17 without correction gives you 4 points The sum of the points over the timeout period is compared to the -w setting for whitelisting. The other messages are not used for white-l…