Real Time Flight Status Prediction System

02 Jun 2024          github repository

Quick demonstration of the project in action:

Noised Audio Visualization

Authors

• Dominik Klisiewicz @DominikKlisiewicz

• Julia Czapla

• Paweł Drąszcz

• Julia Czapla

• Eryk Skrętowski

• Weronika Walczak

Project Overview

This project was prepared for the subject Real Time Analytics at SGH. It was a great opportunity for us to familiarize ourselves with kafka and the concept of stream data. Aside from the educational advantages for the whole group it was also a great opportunitu for myself to experience and learn more about software project management in practice as I was the leader of the group. Essentially, the whole project can be divided into 4 main components. These are:

  • Data simulation and streaming
  • Data collection and analysis
  • web UI
  • API

Data Simulation and Streaming

As of Jun 2024 all real flight status API were paid, so our team decided on analyzing artificially created data. This tasks proved not to be as trivial as it might have seemed. The initial idea was to randomly sample points for x and y coordinates, then put them on a plain and continously move them toward the coordinates of Warsaw by moving them by a vector. Unfortunately, such an approach completely disregarded the roundness of earth and generated completely wrong results. In order to fix this haversine distance formula had to be implemented in code. All of the data produced data was then streamed to the topic using KafkaProducer. The streamed events belonged to one of a few predefined categories:

  • new_location
  • flight_landed
  • flight_crashed

Data collection and analysis

Script consumer_kafka.py was responsible for booting a Kafka consumer that connected to the kafka broker and subscribed to the topic with flight event data. A pandas DataFrame was then used to store and maintain the current status of each flight (the coordinates and if it was still flying or already landed). Then based solely on the GPS coordinates the system computed the ETA.

web UI

For presentation purposes and display of how the system could actually be used in a real setting a simple flask webapp was developped. Users may look into historical data of flights or watch the live status of the incoming flights.

API

The app was also equipped with an API that could be used by third party applications.