Monitoring network traffic to FlightAware

Thought I’d post a new thread for this, since the System Monitoring one is getting long. One more metric I was interested in monitoring was the amount of traffic specifically going to FlightAware. I noticed that as my site started receiving a lot more traffic based on the antenna/amp/location changes I made, total transmit bandwidth was adding up to about 8+ GB per day - of course the vast majority of that is internal to my network, to my VRS server, etc. However, I wanted the data, so I created some iptables firewall rules, then used collectd’s iptables plugin to monitor traffic hitting those rules. I thought I’d post this, in case anyone is interested.

http://i.imgur.com/wnf1Q8D.png

The first thing you need to do is create the iptables rules. All the connections I’ve seen my Pi make are to the 70.42.0.0/16 subnet, but perhaps FA staff can comment if there are others, or you can check it on your own Pi by doing a ‘sudo netstat -n |grep 1200’ then modifying the rules below. These should catch both the regular ADS-B traffic and the fa-mlat-client traffic.



sudo iptables -N flightaware
sudo iptables -A OUTPUT -d 70.42.0.0/16 -j flightaware -m comment --comment "fa-ip-range-out"
sudo iptables -A INPUT -s 70.42.0.0/16 -j flightaware -m comment --comment "fa-ip-range-in"


Then save them using iptables-save:


sudo iptables-save > /etc/firewall.conf

Create the file /etc/network/if-pre-up.d/iptables containing the following:



#!/bin/sh
/sbin/iptables-restore < /etc/firewall.conf


Make it executable with sudo chmod +x /etc/network/if-pre-up.d/iptables. These last few steps will ensure the rules survive a reboot.

Configure collectd.conf to read these rules:



LoadPlugin "iptables"

<Plugin "iptables">
  Chain "filter" "OUTPUT" "fa-ip-range-out"
  Chain "filter" "INPUT" "fa-ip-range-in"
</Plugin>


Restart collectd, and you should have two new folders of rrd files in /usr/lib/collectd/rrd/…

Finally, here is the graphing function:



net_fa_graph() {
  rrdtool graph \
  "$1" \
  --start end-$4 \
  --width 480 \
  --height 200 \
  --step "$5" \
  --title "FlightAware Bandwidth Usage" \
  --vertical-label "bytes/sec" \
  "TEXTALIGN:center" \
  "DEF:rx=$2/iptables-filter-INPUT/ipt_bytes-fa-ip-range-in.rrd:value:AVERAGE" \
  "DEF:tx=$2/iptables-filter-OUTPUT/ipt_bytes-fa-ip-range-out.rrd:value:AVERAGE" \
  "CDEF:tx_neg=tx,-1,*" \
  "AREA:rx#32CD32:Incoming" \
  "LINE1:rx#336600" \
  "GPRINT:rx:MAX:Max\: %4.1lf %sB/sec" \
  "GPRINT:rx:AVERAGE:Avg\: %4.1lf %SB/sec" \
  "GPRINT:rx:LAST:Current\: %4.1lf %SB/sec\c" \
  "AREA:tx_neg#4169E1:Outgoing" \
  "LINE1:tx_neg#0033CC" \
  "GPRINT:tx:MAX:Max\: %4.1lf %sB/sec" \
  "GPRINT:tx:AVERAGE:Avg\: %4.1lf %SB/sec" \
  "GPRINT:tx:LAST:Current\: %4.1lf %SB/sec\c" \
  "HRULE:0#000000" \
  --watermark "Drawn: $nowlit";
}

# This goes in the common_graphs() function - adjust the paths accordingly:
net_fa_graph /srv/www/htdocs/collectd/$2-netfa-$4.png /var/lib/collectd/rrd/$1 "$3" "$4" "$5"


70.42.6.128/25
89.151.84.224/28
206.123.125.0/25
207.210.213.224/27
216.52.171.64/26
2600:0C13:1002:0004::/64
2607:fdb8:0:119::/64

And more specifically for piaware / mlat traffic, it currently all goes to piaware.flightaware.com (round robin 70.42.6.203, 70.42.6.194)

FWIW the bandwidth you’re seeing there is quite a lot higher than I expected, even if you have mlat on. Maybe you’re measuring bits, not bytes?

I’m pretty sure it is correct, since iptables metrics are all in bytes. I have looked at the raw iptables data, and it is around 12-14KB/s. My site is busier than most, but I’d welcome you sanity checking it from the other side!

Andy

FWIW here’s an hour of traffic on my test receiver (flightaware.com/adsb/stats/user/obj#stats-8917):



06/20/2015 12:34:09 mlat(3170): Server:      0.0 kB/s from server    0.0kB/s TCP to server   4.1kB/s UDP to server
06/20/2015 12:34:09 mlat(3170): Aircraft: 118 known, 89 requested by server
06/20/2015 12:34:34 25873 msgs recv'd from dump1090-muta (2162 in last 5m); 25819 msgs sent to FlightAware

Chain OUTPUT (policy ACCEPT 333423 packets, 291374460 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 flightaware  udp  --  *      *       0.0.0.0/0            70.42.6.203          /* fa-mlat-out */
       0        0 flightaware  tcp  --  *      *       0.0.0.0/0            70.42.6.203          tcp dpt:1200 /* fa-piaware-out */
   13215 15266888 flightaware  udp  --  *      *       0.0.0.0/0            70.42.6.194          /* fa-mlat-out */
    7540  2261022 flightaware  tcp  --  *      *       0.0.0.0/0            70.42.6.194          tcp dpt:1200 /* fa-piaware-out */
   20755 17527910 flightaware  all  --  *      *       0.0.0.0/0            70.42.0.0/16         /* fa-ip-range-out */


which is around 4.9kB/s total: 4.2kB/s for mlat, 0.6kB/s for piaware position data.

Are you running mlat? If so it might be interesting to see a TCP vs UDP breakdown on your system & the stats logged by the mlat client. If you’re not running mlat I can’t see how you’d hit 15kB/s…

oooo more motivation to get v6 working in my cabinet. Must harass coworkers to finish deploying new switch…

I am running mlat, and that accounts for most of the traffic, it seems:


06/20/2015 13:50:04 2104193 msgs recv'd from dump1090-mutab (2120 in last 5m); 2104182 msgs sent to FlightAware
06/20/2015 13:51:30 mlat: Receiver connection: ready
06/20/2015 13:51:30 mlat: Server connection:   ready
06/20/2015 13:51:30 mlat: Receiver: 1084.2 msg/s received     18.0kB/s from receiver
06/20/2015 13:51:30 mlat: Server:      0.0 kB/s from server    0.0kB/s TCP to server  11.2kB/s UDP to server
06/20/2015 13:51:30 mlat: Aircraft: 170 known, 133 requested by server

http://flightaware.com/adsb/stats/user/AndyHill74

I think I’ll re-do the iptables rules so TCP and UDP are charted separately to give a breakdown of piaware vs. mlat.

Interesting, thanks! I’d speculate that this is a US vs. Europe thing, a higher proportion of aircraft will be Mode S only in the US so you end up forwarding more traffic. My receiver (in Cambridge, UK) sees a similar message rate as yours but only sends 1/3 - 1/2 of the mlat traffic.

That is interesting…it will be cool to watch over the next few years, as the proportion of Mode S only should gradually decrease, in theory.

Thanks!

Andy,

On your stats, what improvements did you add on 8 June to increase your positions and aircraft counts? I am guessing at least an amp? I just added an amp and it is helping my counts. We are not far from each other in the target rich environment of KORD.

Marty

Hi Marty - that is when I moved the antenna from a 2nd story window up to the attic, so probably 10-12 feet higher. I have had the HAB amp/filter on since almost the beginning. Target rich is right!

Andy

One of my RPi has its antenna in a second floor window and the other RPi has its antenna in the attic. That 10-15 feet of extra height does help.

Marty