In today’s digital landscape, analyzing HTTP Archive (HAR) files is crucial for understanding web performance, identifying bottlenecks, and debugging network issues. This article provides a detailed guide on creating a HAR Analyzer using Flask. We’ll also cover how to Dockerize the application and set up an environment that makes analysis seamless and efficient.

Refer to this article if you want to deploy and run the HAR Analyzer directly without using Docker.

What is a HAR File?

A HAR file is an archive file formatted in JSON that logs web requests and responses. It contains detailed information about the HTTP requests made by a browser, including headers, body content, and response details. Analyzing HAR files can help developers optimize web performance and troubleshoot issues.

Project Overview

Our HAR Analyzer project is built using Flask, a lightweight web framework for Python. We’ll encapsulate our application within a Docker container to ensure consistency across different environments.

Key Features of the HAR Analyzer

• Upload and analyze HAR files.
• Display request details including method, URL, response status, content type, time taken, source IP, response size, latency, error messages, and request payload.
• Summarize overall metrics like total requests, success and failure counts, and average response time.
• A user-friendly web interface to facilitate the analysis process.

Prerequisites

Before we start, make sure you have the following installed on your machine:

• Python 3.7 or later
• Docker and Docker Compose
• Flask (We will install this in the Docker container)
  Step 1: Create the Project Structure
  Create a directory for the project and navigate into it:

mkdir har-analyzer
cd har-analyzer

Inside this directory, create the following files:

har_analyzer/
│
├── har_analyzer.py
├── Dockerfile
├── docker-compose.yml
├── .dockerignore
└── templates/
    ├── index.html
    └── analyze.html

Step 2: Write the Flask Application

In har_analyzer.py, we will implement the core functionality to parse HAR files and serve the web interface.

from flask import Flask, render_template, request
import json
from haralyzer import HarParser

app = Flask(__name__)

# Function to parse HAR file and extract relevant data
def parse_har(file_path):
    with open(file_path, 'r') as f:
        har_data = json.load(f)

    parser = HarParser(har_data)
    entries_data = []
    total_time = 0
    status_code_counts = {1: 0, 2: 0, 3: 0, 4: 0, 5: 0}

    for entry in parser.har_data['entries']:
        request = entry['request']
        response = entry['response']
        timings = entry['timings']
        time_taken = timings['wait']  # You can adjust this to include other timings as needed

        # Status code category
        status_category = response['status'] // 100
        if status_category in status_code_counts:
            status_code_counts[status_category] += 1

        total_time += time_taken

        # Capture additional details
        entries_data.append({
            "method": request['method'],
            "url": request['url'],  # Full URL for tooltip
            "status": response['status'],
            "content_type": response['content'].get('mimeType', 'N/A'),
            "time": time_taken,  # Time in ms
            "source_ip": request.get('headers', [{}])[0].get('value', 'N/A'),  # Source IP assumed from headers
            "error_message": response.get('statusText') if response['status'] >= 400 else None,  # More robust error retrieval
            "payload": request.get('postData', {}).get('text', 'N/A'),  # Capture request payload if available
            "response_size": response.get('content', {}).get('size', 0),  # Response size in bytes
            "timings": {
                "dns": timings.get('dns', 'N/A'),
                "connect": timings.get('connect', 'N/A'),
                "send": timings.get('send', 'N/A'),
                "wait": timings.get('wait', 'N/A'),
                "receive": timings.get('receive', 'N/A'),
            }
        })

    summary = {
        "total_requests": len(entries_data),
        "total_time": total_time,
        "average_time": total_time / len(entries_data) if entries_data else 0,
        "status_code_counts": status_code_counts,
        "success_count": status_code_counts.get(2, 0),
        "failure_count": status_code_counts.get(4, 0) + status_code_counts.get(5, 0)
    }

    return entries_data, summary

# Route for HAR file analysis
@app.route('/', methods=['GET'])
def index():
    return render_template('index.html')

@app.route('/analyze', methods=['POST'])
def analyze_file():
    if 'har_file' not in request.files:
        return "No HAR file uploaded", 400
    
    har_file = request.files['har_file']
    file_path = "/tmp/harfile.har"
    har_file.save(file_path)
    
    entries_data, summary = parse_har(file_path)
    
    # Render results in a simple HTML table in the browser
    return render_template('analyze.html', summary=summary, entries_data=entries_data)

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5020, debug=True)

analyze.html

This template displays the results of the HAR file analysis.

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>HAR Analysis Results</title>
    <style>
        body {
            background: linear-gradient(to bottom right, #6dd5ed, #2193b0);
            /* Gradient background */
            font-family: Arial, sans-serif;
            color: #333;
            padding: 50px;
        }

        h1 {
            text-align: center;
            /* Center align the main title */
        }

        .summary {
            text-align: left;
            /* Left align the summary section */
            margin-top: 20px;
        }

        table {
            width: 100%;
            border-collapse: collapse;
            margin-top: 20px;
        }

        table,
        th,
        td {
            border: 1px solid black;
        }

        th,
        td {
            padding: 8px;
            text-align: left;
        }

        th {
            background-color: #f2f2f2;
        }

        footer {
            margin-top: 30px;
            color: #ffffff;
            /* Footer text color */
        }
    </style>
</head>

<body>
    <h1>HAR Analysis Results</h1>

    <div class="summary">
        <h2>Summary</h2>
        <p>Total Requests: {{ summary.total_requests }}</p>
        <p>Total Time: {{ summary.total_time }} ms</p>
        <p>Average Time: {{ summary.average_time }} ms</p>
        <p>Success Count: {{ summary.success_count }}</p>
        <p>Failure Count: {{ summary.failure_count }}</p>
    </div>

    <h3>Response Code Details</h3>
    <h4>1XX: Informational</h4>
    <p>Total 1XX Responses: {{ summary.status_code_counts[1] }}</p>

    <h4>2XX: Success</h4>
    <p>Total 2XX Responses: {{ summary.status_code_counts[2] }}</p>

    <h4>3XX: Redirection</h4>
    <p>Total 3XX Responses: {{ summary.status_code_counts[3] }}</p>

    <h4>4XX: Client Errors</h4>
    <p>Total 4XX Responses: {{ summary.status_code_counts[4] }}</p>

    <h4>5XX: Server Errors</h4>
    <p>Total 5XX Responses: {{ summary.status_code_counts[5] }}</p>

    <h2>Request Details</h2>
    <table>
        <thead>
            <tr>
                <th>Method</th>
                <th>URL</th>
                <th>Status</th>
                <th>Content-Type</th>
                <th>Time (ms)</th>
                <th>Source IP</th>
                <th>Response Size (KB)</th>
                <th>Latency (ms)</th> <!-- New Field for Latency -->
                <th>Time Breakdown</th>
                <th>Error Message</th>
                <th>Request Payload</th>
            </tr>
        </thead>
        <tbody>
            {% for entry in entries_data %}
            <tr>
                <td>{{ entry.method }}</td>
                <td>
                    <span title="{{ entry.url }}">{{ entry.url[:60] }}{% if entry.url|length > 60 %}...{% endif
                        %}</span>
                </td>
                <td>{{ entry.status }}</td>
                <td>{{ entry.content_type }}</td>
                <td>{{ entry.time }}</td>
                <td>{{ entry.source_ip }}</td>
                <td>{{ (entry.response_size / 1024) | round(2) }} KB</td> <!-- Convert size to KB -->
                <td>
                    {{ (entry.timings.dns + entry.timings.connect + entry.timings.send + entry.timings.wait +
                    entry.timings.receive) | round(2) }} ms
                </td> <!-- Calculate and display total latency -->
                <td>
                    DNS: {{ entry.timings.dns }} ms<br>
                    Connect: {{ entry.timings.connect }} ms<br>
                    Send: {{ entry.timings.send }} ms<br>
                    Wait: {{ entry.timings.wait }} ms<br>
                    Receive: {{ entry.timings.receive }} ms
                </td>
                <td>{{ entry.error_message if entry.error_message else 'N/A' }}</td>
                <td>{{ entry.payload }}</td>
            </tr>
            {% endfor %}
        </tbody>
    </table>

    <footer>
        <p>Created By <a href="https://rahulranjan.org" style="color: rgb(30, 8, 8);">Rahul Ranjan</a></p>
    </footer>

    <a href="/">Go back</a>
</body>

</html>

index.html

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>HAR Analyzer</title>
    <style>
        body {
            background: linear-gradient(to bottom right, #6dd5ed, #2193b0);
            /* Gradient background */
            font-family: Arial, sans-serif;
            color: #333;
            text-align: center;
            padding: 50px;
        }

        h1 {
            color: #280b0b;
            /* Header color */
        }

        form {
            margin-top: 20px;
            display: inline-block;
        }

        input[type="file"] {
            padding: 10px;
            border: none;
            border-radius: 5px;
            margin-bottom: 10px;
        }

        button {
            padding: 10px 15px;
            background-color: #0056b3;
            /* Button color */
            color: white;
            border: none;
            border-radius: 5px;
            cursor: pointer;
        }

        button:hover {
            background-color: #004494;
            /* Button hover color */
        }

        footer {
            margin-top: 30px;
            color: #280b0b;
            /* Footer text color */
        }
    </style>
</head>

<body>
    <h1>Welcome to the HAR Analyzer</h1>
    <form action="/analyze" method="POST" enctype="multipart/form-data">
        <input type="file" name="har_file" required>
        <br>
        <button type="submit">Analyze HAR File</button>
    </form>

    <footer>
        <p>Created By <a href="https://rahulranjan.org" style="color: rgb(30, 8, 8);">Rahul Ranjan</a></p>
    </footer>
</body>

</html>

Step 4: Create the Dockerfile

Next, we will create a Dockerfile to set up the environment for our Flask application.

# Use the official Python image from Docker Hub
FROM python:3.9-slim

# Set the working directory
WORKDIR /app

# Copy the requirements file if you have one
# In this case, we will install dependencies directly
# COPY requirements.txt .

# Install Flask and haralyzer
RUN pip install Flask haralyzer

# Copy the application code into the container
COPY . .

# Expose the port the app runs on
EXPOSE 5020

# Define the command to run the application
CMD ["python", "har_analyzer.py"]

Step 5: Create the Docker Compose Configuration

In the root directory, create a docker-compose.yml file to manage our application and its dependencies.

version: '3.8'

services:
  har-analyzer:
    build: .
    ports:
      - "5020:5020"
    volumes:
      - .:/app
    environment:
      - FLASK_ENV=development

Step 6: Build and Run the Application

Now that we have everything set up, it’s time to build and run our Docker container.

docker-compose up --build -d

You should see logs indicating that the Flask application is running.

Access the Application

Open your web browser and go to:

http://localhost:5020

Here, you can upload your HAR files for analysis.

Step 7: Analyze a HAR File

• Click on the upload button to choose a HAR file from your local machine.

After uploading, the application will process the file and display the analysis results.

Output Details

The application will provide a detailed table with various metrics, including:

• HTTP Method
• URL
• Response Status
• Content-Type
• Time Taken
• Source IP
• Response Size
• Latency
• Breakdown of DNS, Connect, Send, and Receive times
• Error Messages, if any
• Request Payload

Conclusion

This guide has thoroughly explained the process of creating a HAR Analyzer using Flask and Docker. You have learned how to structure your project, develop the Flask application, configure the Docker environment, and effectively analyze HAR files. This solution can be a valuable tool for developers and web performance engineers, helping to enhance their workflow and troubleshoot issues more efficiently.

I will add additional features and visualizations to enhance the project. The code will be uploaded to my GitHub.

Reach out at Linkedin for any questions.

#Docker #HAR #Troubleshooting #UI #Flask