Raspberry Pi + Resin.io = Feed setup in under an 20 minutes

FlightAware ADS-B Flight Tracking
1

TL;DR

  • Get first Raspberry Pi configured, connected and submitting in 10 minutes

  • Second device in under 5 minutes

  • Changes in around 3 minutes

  • Remote management and control of all devices

  • Solution designed for 1 or many, many Raspberry Devices

  • Intended to solve my need to have Raspberry Pi’s in various countries without regular physical access

  • Only requirement - ssh public key, git client, web browser and the hardware

  • Ease of setup - low

Background
About a year an half ago I wrote a post http://discussions.flightaware.com/ads-b-flight-tracking-f21/raspberry-pi-docker-from-blank-sd-to-feeding-in-15-mins-t36459.html - Raspberry
Pi + Docker = Feed setup in under an hour.

I decided to go this route because:

  • Docker was quick to share and setup.

  • Docker had become available on RPI.

  • Allow very quick, repeatable installs.

  • Main goal was to allow me to install many RPI’s in various locations.

It however had a few flaws:

  • It required some unix command line knowledge.

  • Images would have versions hard baked unless recreated.

  • Environment variables had to be hand written into docker-compose.yml file.

  • RPI’s still required manual intervention (i.e. to change something, I needed physical access - which is difficult for RPI’s I intended to install in other countries).

A couple of weeks ago I stumbled across Resin.io. It seemed to resolve many of my issues.
I am very happy with the results and thought I would share my project with everyone else.

Resin.io has allowed me to:

  • Deploy first device in under 20 minutes.

  • Deploy subsequent devices in under 5 minutes.

  • Allow very quick, upgrades of software in around 2 minutes (on all devices).

  • Remote manage and control all devices (even if they are not inside my current network).

Installation information to follow…

1 Like
2

Prerequisites

  • Obviously a Raspberry Pi (2 or 3 is best).

  • A blank micro SD card (16GB preferred, class 10 preferred)

  • RTL-SDR Dongle

  • Access to a command line with git (could be Windows, Linux, Mac OS).

  • A small amount of time.

  • One or all of the following accounts: Flightradar24, Flightaware, Planefinder.

What is going to happen?

  1. Create a Resin.io account with a application

  2. Download Resin.io image for your device, install, plugin.

  3. See device appear on Resin.io and add required variables.

  4. Clone my repo which contains basic Docker template, start up scripts.

  5. Set git remote of your new clone to your Resin.io account, push.

  6. Wait - and you’re done.

Hang on… Docker returns?
Resin.io uses a Docker template to re-create the images whenever anything changes. This takes the hard work out of you getting involved. This also means that when there is a software change, it’s very easy to trigger Resin.io to create a new image. Resin.io does all the heavy lifting, it downloads the new software, compiles it and puts it into a new Docker image which is downloaded to your device. So easy!!!

How hard is this?

  • Not at all!

  • Only need to copy a couple of git commands.

  • No Linux knowledge required.

  • Everything driven from Resin.io.

3

Before Starting
Working with Resin.io requires interaction between files on your computer and Resin.io using git.

This requires:
[ul]
[li]ssh installed[/li]
[li]git installed[/li][/ul]

If you have Linux or MacOS, these are already inbuilt. If you have Windows, these do not come preinstalled and will require some installation. The following guide might help.
[ul]
[li]https://blog.assembla.com/assemblablog/tabid/12618/bid/77264/setting-up-git-on-windows-in-four-easy-steps.aspx[/li][/ul]

If you already have these, skip to “Setting up Devices and Resin.io Account”.

The main goal of the above is to:
[ul]
[li]Ensure you have an ssh public key created (allows you to connect to Resin.io)[/li]
[li]Have a git client that allows you to push files to Resin.io[/li][/ul]

Setting up Devices and Resin.io Account
[ol]
[li]Signup at Resin.io for an account at balena dashboard
[li]Create a new application in Resin.io for a Raspberry Pi (2 or 3 - this doesn’t seem to matter).[/li]
[li]Click on your Application, then click “Download Resinos” to download the Resin.io OS. Follow these instructions.
[ol]
[li]The image will require knowledge of either your WiFi password or a fixed connection (this is for your use only)[/li][/li][li]A copy of your ssh public key.[/li]
[li]Resin.io’s application https://etcher.io/ is best for creating images.[/li][/ol]
[li]Wait for your device to become available on Resin.io website.[/li]
[li]Once available:
[ol]
[li]Click on the device.[/li][/li][li]Select “Environment Variables” on the left of the Resin.io window.[/li]
[li]Insert your GPS co-ordinate variables (See http://www.whatsmygps.com/):
[ul]
[li]LAT = - COMPULSORY[/li][/li][li]LONG = - COMPULSORY[/li][/ul]
[li]Insert the following Planefinder variable if known:
[ul]
[li]PF_SHARECODE = [/li][/ul][/li]
[li]Insert the following Flightradar24 variable if known:
[ul]
[li]FR24_KEY = [/li][/ul][/li]
[li]Insert the following Flightaware variables if known:
[ul]
[li]PIAWARE_MAC = - this has been deprecated and should only be used IF you want to carry over an old device[/li][/li][li]PIAWARE_ID = - preferred over above[/li][/ul]
[li]The following variables are optional:
[ul]
[li]GAIN = [/li][/li][li]PPM = [/li][/ul][/ol][/ol]

Regarding MAC Address (Note: Now Deprecated)
Flightaware used to generate an account based on the MAC Address of your network device. This is a unique identifier baked into either the Wifi USB device or Ethernet adapter when manufactured. This doesn’t play nicely when you change your network connection.
To get around this, piaware allows you to pass a MAC address at startup. I have included this in the installation automatically. This allows you to keep the MAC address you already have (These appear on PiAware - Claim and Link a Brand New PiAware Ground Station - FlightAware)

But highly recommend not using this and allowing Flightaware to generate a feeder id instead. This becomes PIAWARE_ID.

4

Cloning My Code and Pushing to your Device(s)

I’ve created everything you need code wise to get this working. You don’t need to do anything except setup your accounts, and push the code.

  1. Open up a terminal (Linux/MacOS) or command prompt (Windows)

  2. Clone my repo from https://bitbucket.org/inodes/resin-docker-rtlsdr
    (type this verbatim) using git clone .


$ git clone https://inodes@bitbucket.org/inodes/resin-docker-rtlsdr.git
Cloning into 'resin-docker-rtlsdr'...
remote: Counting objects: 178, done.
remote: Compressing objects: 100% (176/176), done.
remote: Total 178 (delta 106), reused 0 (delta 0)
Receiving objects: 100% (178/178), 21.84 KiB | 0 bytes/s, done.
Resolving deltas: 100% (106/106), done.
  1. Change into the repo directory you’ve just cloned

$ cd resin-docker-rtlsdr/

  1. Copy the git remote command from your application in resin (this is in top right in Resin.io page)

  2. Paste the git remote command onto command line (your command will differ).


$ git remote add resin yourlogin@git.resin.io:yourlogin/yourproject.git
  1. Push to remote using the command

$ git push resin master
..... LOTS OF OUTPUT

The above output is the build from scratch of everything required to run Flightaware and other utilities.
The great thing about this is that if you have many devices, they are all updated.

5

A few things to note
Once your device is up and running, there are a few things to note.
Resin.io allows you to see the status of each device and control it.

Devices Page

  • Control your devices

  • See what your local IP address would be

  • See the most recent commit

  • Reboot and connect to your devices

640×531 173 KB

Environment Variables

  • This is where the secret sauce is. Set variables that can be used in scripts running on the device (e.g. the start.sh I have provided)

  • If you change the location of a device, just change the variable for LONG and LAT. The device automatically rebuilds using the new variables.

640×630 102 KB

Device Summary

  • See the current state of the device. Easy to see the output from the current Docker image.

  • See the ip - from here if you’re on the local network, you can connect to the device as per usual (e.g. in my case, opening http://10.0.1.84:8080/dump1090-fa
    on my local network gives me the normal Skyview map client UI).


Device Terminal

  • Connect to any device from any location, even if you’re not on the local network.

  • Debug the application. You could for example change the scripts I provided completely, push again to resin and have a completely different result. It allows flexibility on scale.

640×577 53.1 KB

6

Please don’t configure passwords - it’s a security risk and is not necessary.

You probably want to make the piaware feeder ID configurable (this is the “feeder-id” key in piaware-config et al) rather than the MAC address as the MAC address is not used for identifying new installs.

I actually looked at resin.io a while back as a possibilty for managing FlightFeeders but it didn’t seem to scale out to thousands of devices where the host network is very unpredictable/limited and the host may not have the knowledge to set anything unusual up. Seems like it would work well for devices where you have more control over the host environment though.

7

This is a very good point point and one reason I was drawn towards Resin.io.

I always stress:

  1. Never commit credentials to code (e.g. never commit your creds to git) - (unless the credentials are committed in an encrypted format like gpg or something like encrypted yaml)
  2. Always secure accounts where possible using 2FA
  3. Never store or transmit credentials via insecure methods

This was done previous by writing creds to a personal docker-compose.yml - but that doesn’t scale well.

The credentials in this case are secured with a personal Resin.io account.
With this in mind, it’s crucial to secure this with 2FA https://dashboard.resin.io/preferences/2fa

Resin.io does not initiate a connection with the device. The device itself fetches the credentials from your own specific account securely (would love to get Resin.io’s doc on how this is done).

I suppose one way to confirm this for certain would be to somehow inject a gpg private key into the device, then encrypt secrets using gpg --sign --encrypt to yourself.
But the audience I intended for my post was one that didn’t have the technical knowledge or patience, and has possibly struggled with setting up a Raspberry Pi, let alone delving into encryption best practices.

If I could get it to work without much work (i.e. use scp to copy a gpg private key into a particular place), then I would do so. But even setting up gpg, creating your own key etc is another level on top of the fact that one had to generate an ssh private key.

I’ll change the code to switch to feeder ID.
Keeping the old method was really due to fact most of my old feeds had MAC Addresses, and I was carrying them over.

8

Hmmm… so password is irrelevant in 3.5 now?
If this is so, then it’s SO AWESOME! 8)

9

Username/password have never been necessary once the site has been associated with a user. I’ve been gradually moving everything else (docs, better support for associating sites, etc) away from them for some time and almost all the pieces are there for removing support for username/password entirely.

10

I’ve dragged relics from being a long time user. The automatic claim feature is extremely useful, particularly for those who I am aiming this solution at.

However it does rely on the user accessing flightaware.com/adsb/piaware/claim from the network that the device is on. This isn’t always possible, particularly in my case where devices will be in remote countries with no web browser access. (Side note: Is there an REST API for claiming?)

I suppose a fair work around when claim is:

  1. Connect a new device on the same network
  2. Claim using flightaware.com/adsb/piaware/claim
    3, Insert the receiver id into the variables intended for the remote device
  3. Restart the remote device using resin using receiver id variable (Instead of the generated MAC Address)
  4. …
  5. Profit $$$

So a change I can implement is:

  1. Drop username and password altogether
  2. Replace MAC with Receiver ID
  3. Change the piaware config command that is run when the image is built

It’s important to note that without the variable, a new device will appear to Flightaware each time the device is restarted. This does produce a long list of devices on flightaware.com/adsb/piaware/claim up until the device is finally claimed.

11

That is one of the missing pieces.

You can construct a URL like this: flightaware.com/adsb/piaware/cl … 3456789012
(inserting an appropriate feeder ID)
and it will claim that unclaimed feeder for the currently-logged-in account; the piaware sdcard image constructs a link like this on the top-level status page for unclaimed feeders.

But you do need to be logged in via the usual login page and so it’s hard to use from an automated script.

12

Thanks. That’s good to know.
I think for the moment the best thing for me to do is just claim on my local network, the assign the claimed id to one of the devices I own. I prefer the idea of a unique identifier to a MAC address, and note that when I was originally running Resin.io without assigning anything, I was getting repeated new ID’s. There is no way around this when using Resin apart from warning the User on the console to claim the device before restarting again and to remind them to unclaim any devices that haven’t been claimed.

Is there a way to convert old MAC oriented devices over to the new ID’s without losing history?

13

I’ve just updated my start script to deprecate the forced MAC address and user credentials with a recommendation instead to use the claim feature and then feeder-id.

To activate this simply:

  1. Change into the repo of mine you originally cloned and run git pull (pull the latest)


$ cd resin-docker-rtlsdr/
$ git pull
  1. Push those changes to resin


git push resin master
  1. Wait for a new code to deploy on your devices (a new commit will appear)
  2. Ensure one of the devices is on your local network
  3. When ready, remove PIAWARE_MAC, PIAWARE_USER, PIAWARE_PASSWORD for a single device known to be on your network
  4. Wait for a restart of the device
  5. Open flightaware.com/adsb/piaware/claim and look for the new device (note a previous MAC address device will no longer be sending)
  6. Claim the device and copy this in into a new Resin.io variable called PIAWARE_ID - this will initiate a restart of that device
  7. Done! You should see the new device sending with the claimed ID instead of the MAC

Worth noting that the new device will replace the old one. Wait for a period of time to unclaim the device.
I suspect that history is lost along with a move to ID instead of MAC.

If you have a device that is already using a feeder id, just copy and paste that into PIAWARE_ID from the very beginning and all is sweet!
Also, if a device is present elsewhere outside your network, just turn insert the id you have previously claimed into the PIAWARE_ID of that device, being certain to only use one ID per device (these are unique identifiers that are strictly associated with your account).

14

What I’d like to do is have a way to say “hey, create me a new site please” which will prompt for username/password and return a new UUID associated with that account. Then you could plumb that into the resin config. (But it’s not there yet)

Is there a way to convert old MAC oriented devices over to the new ID’s without losing history?

It’s not something you can do from your side. The plan is to globally convert all existing sites from MAC to UUID at some point in the near future (with a mapping so existing site associations are retained)

15

Having converted my devices over last night, I have to say it’s very easy. I changed my script to prompt if either a MAC address, username or password was noted in the variables.
Remove the MAC, and a new UUID is generated. Copy and paste that into Resin, the device automatically restarted and uses the UUID instead. My script then just prompts to remove the three previously mentioned variables.

I then switched over my oldest configured RPI in roughly 2 minutes. Just did as above, and it was done. Easy - and the reason I have used Resin in the first place.

Not an issue I suppose. History really only lasts 30 days, and it’s easier just switching over to the latest version of piaware and the UUID instead. Not having the credentials required is a massive bonus.

16

I’ve just had a go at this and used the PIAWARE_ID as the identifier, but I’m getting errors from the Pi.

I went to the claim page but it said there were no new devices. All it had was the same Pi running the old build.
I entered this ID 40435 as PIAWARE_ID but I’m still getting errors.



17.06.17 13:02:45 (+0100)    CGroup: /docker/228cb2eeb7fd1b263a7a19ede004175cb52
8cfa0d5c42a74c540bccf5a791a2f/system.slice/pfclient.service
17.06.17 13:02:45 (+0100)            └─293 /usr/bin/pfclient -d -i /var/run/pfcl
ient.pid -z /etc/pfclient-config.json -y /var/log/pfclient $
17.06.17 13:02:45 (+0100)
17.06.17 13:02:45 (+0100) Jun 17 11:53:43 d15c087 pfclient[289]: Starting pfclie
nt: pfclient2017-06-17 11:53:43.764531 [V] An invalid user location is set in pf
client-config.json and will be ignored

Any ideas?

17

I think I can help here. One important component is LAT and LONG variables. You can set these as per Application (if you have many RPI’s in one location), or per device.
That’s up to you, but both work. For me, I have many locations, so I set per device.

GPS Variables

If you have all your devices in one location, save yourself some time by setting these as application specific variables. There is no hard and fast rule though.

  1. Head to this site https://mycurrentlocation.net/

  2. Copy your Latitude and Longitude onto a notepad

  3. Click on your application

  4. Click on “Environment Variables” directly under “Devices”. It should read “Application Environment Variables” (i.e. the ones that are shared among all devices)

https://preview.ibb.co/ihESa5/app_variables.png

  1. Enter your first Resin.io variables as follows in form “Name” (the variable name) and “Value” (the value):

  2. LAT = , click “Add”

https://preview.ibb.co/jiL8F5/variable_gps.png

  1. LONG = , click “Add”

https://preview.ibb.co/m7hzNk/variable_gps_2.png

If you have all your devices in one location, you can save some time and make these the same across all devices. In the above screenshots, to save time I have made these “Application Environment Variables”.

This should fix your error, which relates to Planefinder.

However, further to this you can carry over an old Piaware config by going to your original feeder and looking for an address marked MAC.
This will be a hex code with colon’s.

Put this in as variable PIAWARE_MAC.

Word of warning though, you should consider migrating to feeder id, which means just leave out all PIAWARE variables, go to flightaware.com/adsb/piaware/claim search for the very, very long feeder id and submit this into Resin using variable PIAWARE_ID

Anytime you submit a variable, the device will restart and use the new variables.

18

I have made a much more comprehensive version of this post on the Sydney Airport website:
http://www.yssyforum.net/board/showthread.php?t=11523

This contains information I just glanced over here.

19

I think this is your site ID, rather than PIAWARE ID.
And easier way to see your information is to go to flightaware.com/adsb/piaware/claim

Look under previously claimed devices: “Linked PiAware Receivers”

Without actually pasting any of that information here (it is private to you, and you shouldn’t share it), it should be in the format:



Site 12345 - abcdefghi-1234-5678-975g-345jvunwbsfjg: PiAware (Debian Package Add-on) 3.5 added Friday, 16 June 2017 (xxx.yyy.www.zzz / 10.0.1.11)
Site 67890 - Old One - 9a:8b:7c:99:00:11: PiAware (Debian Package Add-on) 3.5 added Sunday, 7 February 2016 (xxx.yyy.www.zzz / 10.0.1.12)

What I have paste above is a mock up.

The first site, 12345 has a big long, long, long id with dashes. That’s a new style feeder id. Store this as PIAWARE_ID in Resin.
The second site 67890 has an old MAC address with xx:xx:xx:xx:xx - I would store this as PIAWARE_MAC to carry it over, but this has now been deprecated.

20

Where did you find that info? I see nothing like that on the statistics page for the previously linked device.

Gareth