Travel and tourism are dynamic industries that thrive on data-driven insights. In this project, we will build a Travel and Tourism Insights Dashboard that integrates weather data and flight availability using Elasticsearch, WeatherAPI, and Skyscanner API.

This dashboard will be containerized with Docker and visualized in Kibana to create interactive insights.

This project demonstrates how to:

1. Fetch weather and flight data from APIs.
2. Store the data in Elasticsearch.
3. Containerize the application using Docker.
4. Create Kibana visualizations and dashboards for deeper insights.

Step 1: Fetch API Keys

WeatherAPI Key

1. Visit WeatherAPI.com and sign up.
2. Verify your email and log in.
3. Go to the API Keys section in your dashboard.
4. Generate a new API key and save it for later use.

Aviationstack API Key

1. Visit Aviationstack.com and sign up for a free account.
2. Verify your email and log in to the dashboard.
3. Generate an API key and copy it for later use.

Step 2: Setting Up the Environment

Create Project Directory

mkdir Travel_Tourism_Dashboard
cd Travel_Tourism_Dashboard

Add Requirements

Create a requirements.txt file with:

requests
elasticsearch
schedule

Dockerfile

FROM python:3.9-slim
WORKDIR /app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "data_ingest.py"]

Step 3: Python Script

data_ingest.py

import requests
from elasticsearch import Elasticsearch
import schedule
import time
import datetime
import os

# Elasticsearch configuration
ES_HOST = os.getenv("ES_HOST", "https://your-es-host-url")
ES_USERNAME = os.getenv("ES_USERNAME", "elastic")
ES_PASSWORD = os.getenv("ES_PASSWORD", "password")
WEATHER_INDEX = "weather-data"
FLIGHT_INDEX = "flights-data"

# API Configurations
WEATHER_API_KEY = os.getenv("WEATHER_API_KEY", "your-weatherapi-key")
AVIATIONSTACK_API_KEY = os.getenv("AVIATIONSTACK_API_KEY", "your-aviationstack-key")
CITIES = ["London", "Bangalore", "New York"]

# Connect to Elasticsearch
es = Elasticsearch(
    ES_HOST,
    basic_auth=(ES_USERNAME, ES_PASSWORD)
)

# Ensure indices exist
if not es.indices.exists(index=WEATHER_INDEX):
    es.indices.create(index=WEATHER_INDEX)
if not es.indices.exists(index=FLIGHT_INDEX):
    es.indices.create(index=FLIGHT_INDEX)

# Fetch Weather Data
def fetch_weather_data():
    for city in CITIES:
        try:
            url = f"http://api.weatherapi.com/v1/current.json?key={WEATHER_API_KEY}&q={city}&aqi=no"
            response = requests.get(url)
            data = response.json()
            weather_doc = {
                "city": city,
                "temperature": data["current"]["temp_c"],
                "humidity": data["current"]["humidity"],
                "condition": data["current"]["condition"]["text"],
                "timestamp": datetime.datetime.utcnow()
            }
            es.index(index=WEATHER_INDEX, document=weather_doc)
        except Exception as e:
            print(f"Error fetching weather data for {city}: {e}")

# Fetch Flight Data
def fetch_flight_data():
    url = f"http://api.aviationstack.com/v1/flights?access_key={AVIATIONSTACK_API_KEY}"
    try:
        response = requests.get(url)
        data = response.json()
        flight_doc = {
            "flights": data.get("data", []),
            "timestamp": datetime.datetime.utcnow()
        }
        es.index(index=FLIGHT_INDEX, document=flight_doc)
    except Exception as e:
        print(f"Error fetching flight data: {e}")

schedule.every(10).minutes.do(fetch_weather_data)
schedule.every(30).minutes.do(fetch_flight_data)

if __name__ == "__main__":
    fetch_weather_data()
    fetch_flight_data()
    while True:
        schedule.run_pending()
        time.sleep(1)

docker-compose.yml

version: '3.8'
services:
  travel-dashboard:
    build: .
    environment:
      - ES_HOST=https://your-es-host-url
      - ES_USERNAME=elastic
      - ES_PASSWORD=password
      - WEATHER_API_KEY=your-weatherapi-key
      - AVIATIONSTACK_API_KEY=your-aviationstack-key
    restart: always

Step 5: Kibana Visualizations

1. Weather Trends Line Chart: Compare temperature trends across cities over time.
2. Flight Routes Bar Chart: Display the frequency of flight routes and price trends.
3. Weather Condition Pie Chart: Visualize weather conditions by percentage across cities.
4. Humidity Heatmap: Show humidity distribution using heatmaps.
5. Flight Availability Timelines: Map flight schedules and availability in a Gantt chart.

Airport with maxium delayed arrival flight

There are a lot of amazing visualizations that can be plotted based on the available data.

Step 6: Deployment and Scalability

Deploy the container to AWS or Azure for scalability and reliability. Monitor performance and update visualizations as needed.

The project code is available on GitHub.

Conclusion

The Travel and Tourism Insights Dashboard provides a powerful framework for analyzing weather and flight data. With Elasticsearch and Kibana, users can uncover patterns and make data-driven decisions for travel planning. Explore enhancements like integrating hotel availability or travel cost estimations to make this project more impactful!

#otel #docker #kubernetes #devops #elasticsearch #observability #search #apm #APM #python #kibana

Reach out at Linkedin for any questions