Does any one have an example of the Linux commands that can be installed on a Amazon Web Services EC2 instance to query the FlightAware live data feeds?
Thanks!
Are you asking about FlightXML?
Or Firehose?
Or the output of an individual ADS-B receiver?
Thanks for your responses…
I’m trying to take data off of the FlightAware Firehose and store it (For Analysis) on a Amazon Web Services S3 storage bucket.
I don’t think there are S3-specific examples out there, but you can run a variety of programming languages on S3 and we have examples:
dbaker,
Thanks for your post.
S3 is a storage bin that works with Amazon Web Services Linux. So your suggested solution will not run on S3.
What I’m looking for is generic Linux code that can be used on a Linux operating system to connect to FlightAware.
The examples linked above will work fine on a Linux system.
After retrieving the data, storing it in S3 is something that you would need to develop yourself.
Thank you for the support,
Sorting to S3 is the part I know how to do.
My problem is finding an example of the Linux code for connecting to the FlightAware server.
I can see how your proposed (Copied Below) will connect to “firehose.flightaware.com” using my user name and password
Is your proposed code intended to be a re-write of agent.json within Linux?
And do you have any documents that explain how your proposed code is to be used within Linux?
A couple of “short vectors” on how to use the code you provide would be very helpful.
Rick
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import javax.net.ssl.*;
import org.json.simple.JSONObject;
import org.json.simple.parser.JSONParser;
import org.json.simple.parser.ParseException;
public class SSL_Client_json_simple {
// substitute your own username and password
private static final String initiation_command = "live username XXXXXXXX password XXXXXXXXXXXXXXXXXXXX" + "
";
private SSLSocket ssl_socket;
public static void main(String] args) {
String machineName = "firehose.flightaware.com";
RunClient(machineName);
System.out.println(" Thank you for using FlightAware ... bye now");
}
private static void RunClient(String machineName) {
System.out.println(" Running Client");
try {
SSLSocket ssl_socket;
ssl_socket = (SSLSocket) SSLSocketFactory.getDefault().createSocket(machineName, 1501);
// enable certifcate validation:
SSLParameters sslParams = new SSLParameters();
sslParams.setEndpointIdentificationAlgorithm("HTTPS");
ssl_socket.setSSLParameters(sslParams);
//send your initiation command
OutputStreamWriter writer = new OutputStreamWriter(ssl_socket.getOutputStream(), "UTF8");
writer.write(initiation_command);
writer.flush();
// read messages from FlightAware
BufferedReader reader = new BufferedReader(
new InputStreamReader(ssl_socket.getInputStream()));
String message = null;
int limit = 5; //limit number messages for testing
while (limit > 0 && (message = reader.readLine()) != null) {
System.out.println("msg: " + message + "
");
parse_json(message);
limit–;
}
//done, close everything
writer.close();
reader.close();
ssl_socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
public static void parse_json(String message) {
//using JSON.simple: Java toolkit for JSON
JSONParser jsonParser = new JSONParser();
JSONObject jsonObject;
try {
// parse message from json to JSONObject
jsonObject = (JSONObject) jsonParser.parse(message);
// retrieve values of interest associated with the keys
// for alternative ways to retrieve data checks [json.org/javadoc/org/json/JSONObject.html](http://www.json.org/javadoc/org/json/JSONObject.html)
String type = (String) jsonObject.get("type");
String ident = (String) jsonObject.get("ident");
String air_ground = (String) jsonObject.get("air_ground");
String alt = (String) jsonObject.get("alt");
String clock = (String) jsonObject.get("clock");
String id = (String) jsonObject.get("id");
String gs = (String) jsonObject.get("gs");
String heading = (String) jsonObject.get("heading");
String lat = (String) jsonObject.get("lat");
String lon = (String) jsonObject.get("lon");
String reg = (String) jsonObject.get("reg");
String squawk = (String) jsonObject.get("squawk");
String updateType = (String) jsonObject.get("updateType");
// if any field is missing, for eg if "squawk" is missing then squawk value will be null!
// print values from above
System.out.println("--------- Parsing Results --------");
System.out.println(String.format(" %-10s %-10s
"
+ "%-10s %-10s
%-10s %-10s
%-10s %-10s
"
+ "%-10s %-10s
%-10s %-10s
%-10s %-10s
"
+ "%-10s %-10s
%-10s %-10s
%-10s %-10s
"
+ "%-10s %-10s
%-10s %-10s
%-10s %-10s “,
“type”, type,
“ident”, ident,
“airground”, air_ground,
“alt”, alt,
“clock”, clock,
“id”, id,
“gs”, gs,
“heading”, heading,
“lat”, lat,
“lon”, lon,
“reg”, reg,
“squawk”, squawk,
“updateType”, updateType
));
System.out.println(”---------------------------------
");
} catch (ParseException e) {
e.printStackTrace();
}
}
}
The Java and Python examples should work unchanged under Linux, there’s no rewrite needed, I am having trouble working out what you’re asking.
I’d suggest you get one of the examples working on your target system, whichever language you are more familiar with, then look at extending it to process the data however you need to.
It sounds like you just need to replace the parse_json function in our firehose example to do whatever S3 storage operations you would like to do, rather than parsing and printing out each message.
However, you might want to consider inserting the data into an indexed database (SQL or NoSQL) rather than simply a flat file in S3, if your intent is to ever do fast access against that data in the future.