How HTTP Works

Every time you open a web page, your browser and a server exchange messages using HTTP - the Hypertext Transfer Protocol. It is the foundation of the web, and understanding it is essential for anyone working with networks, security, or IoT devices.

HTTP is deceptively simple. A client sends a request. A server sends a response. That is the entire model. But the details of how requests and responses are structured, what headers do, and why the protocol is stateless reveal a system that is both elegant and full of security implications.

The Request-Response Cycle

HTTP follows a strict client-server model. The client (usually a browser) initiates a connection to the server using TCP. Once the TCP handshake completes, the client sends an HTTP request. The server processes it and returns an HTTP response. Then, depending on the HTTP version, the connection either closes or stays open for additional requests.

This cycle repeats for every resource on a page. A single web page might trigger dozens of HTTP requests - one for the HTML document, one for each CSS file, one for each JavaScript file, one for each image. The browser orchestrates all of these in parallel where possible.

sequenceDiagram
    participant C as Client - Browser
    participant S as Server
    C->>S: TCP SYN
    S->>C: TCP SYN-ACK
    C->>S: TCP ACK
    Note over C,S: TCP connection established
    C->>S: HTTP Request - GET /index.html
    S->>C: HTTP Response - 200 OK + HTML
    C->>S: HTTP Request - GET /style.css
    S->>C: HTTP Response - 200 OK + CSS
    C->>S: HTTP Request - GET /app.js
    S->>C: HTTP Response - 200 OK + JavaScript
    Note over C,S: Keep-alive - connection reused

Figure 1 - A typical HTTP session over a persistent TCP connection with multiple request-response pairs

Anatomy of an HTTP Request

An HTTP request consists of three parts: the request line, the headers, and an optional body.

The request line contains three elements separated by spaces:

• Method - what action the client wants (GET, POST, PUT, etc.)
• URI - the resource path, like /api/devices
• Version - the HTTP version, typically HTTP/1.1

A real request line looks like this: GET /scan/results HTTP/1.1

Headers follow the request line, each on its own line in Key: Value format. Common request headers include:

• Host - the domain name of the server (required in HTTP/1.1)
• User-Agent - identifies the client software
• Accept - tells the server what content types the client can handle
• Content-Type - describes the format of the request body (for POST/PUT)
• Authorization - carries credentials for authenticated endpoints
• Cookie - sends stored cookies back to the server

The body is optional and typically present only with POST or PUT requests. It contains the data being sent - a form submission, a JSON payload, or a file upload.

HTTP Methods

HTTP defines several methods, each with a specific semantic meaning:

GET retrieves a resource. It should never modify server state. GET requests have no body - all parameters go in the URL query string.

POST submits data to the server, typically creating a new resource or triggering a process. The data goes in the request body.

PUT replaces an entire resource at the specified URI. If the resource does not exist, the server may create it.

PATCH applies a partial modification to a resource - updating one field without replacing the whole object.

DELETE removes the resource at the specified URI.

HEAD is identical to GET but returns only headers, not the body. Useful for checking if a resource exists or reading its metadata without downloading it.

OPTIONS asks the server what methods and headers it supports for a given resource. This is used heavily in CORS preflight requests.

graph LR
    subgraph "Safe Methods - no side effects"
        GET[GET - retrieve] --> HEAD[HEAD - headers only]
        GET --> OPTIONS[OPTIONS - capabilities]
    end
    subgraph "Unsafe Methods - modify state"
        POST[POST - create] --> PUT[PUT - replace]
        PUT --> PATCH[PATCH - partial update]
        PATCH --> DELETE[DELETE - remove]
    end
    subgraph "Idempotent"
        GET2[GET] -.-> PUT2[PUT]
        PUT2 -.-> DELETE2[DELETE]
    end

Figure 2 - HTTP methods classified by safety and idempotency

Anatomy of an HTTP Response

The server's response mirrors the request structure: a status line, headers, and a body.

The status line contains the HTTP version, a three-digit status code, and a reason phrase: HTTP/1.1 200 OK

Response headers provide metadata about the response:

• Content-Type - the MIME type of the body (text/html, application/json, etc.)
• Content-Length - the size of the body in bytes
• Set-Cookie - instructs the client to store a cookie
• Cache-Control - tells the client how long it can cache the response
• Location - used with 3xx redirects to specify the new URL

The body contains the actual content - HTML, JSON, an image, or whatever the client requested.

Status Codes

Status codes are grouped by their first digit:

2xx - Success. The request was received, understood, and accepted. 200 OK is the most common. 201 Created confirms a new resource was made. 204 No Content means success but nothing to return.

3xx - Redirection. The client needs to take additional action. 301 Moved Permanently means the resource has a new URL forever. 302 Found is a temporary redirect. 304 Not Modified tells the client its cached version is still valid.

4xx - Client Error. The request was malformed or unauthorized. 400 Bad Request means the server cannot parse the request. 401 Unauthorized means authentication is required. 403 Forbidden means the server understood but refuses to authorize it. 404 Not Found means the resource does not exist. 429 Too Many Requests means rate limiting kicked in.

5xx - Server Error. The server failed to fulfill a valid request. 500 Internal Server Error is the generic catch-all. 502 Bad Gateway means an upstream server sent an invalid response. 503 Service Unavailable means the server is overloaded or in maintenance.

Stateless by Design

HTTP is stateless. Each request is independent - the server does not inherently remember anything about previous requests from the same client. Request number 100 from your browser is treated exactly the same as request number 1.

This design makes HTTP simple and scalable. Any server in a cluster can handle any request without needing to know what happened before. Load balancing becomes straightforward.

But statelessness creates a problem: how do you stay logged in? How does a shopping cart persist across page loads?

The answer is cookies. When a server wants the client to remember something, it sends a Set-Cookie header in the response. The browser stores the cookie and automatically includes it in every subsequent request to that domain via the Cookie header.

Session management works on top of cookies. The server generates a unique session ID, stores it in a cookie, and maps it to server-side session data. Every request carries the session ID, letting the server look up the associated state.

sequenceDiagram
    participant B as Browser
    participant S as Server
    B->>S: POST /login - username + password
    S->>S: Validate credentials
    S->>S: Create session - ID abc123
    S->>B: 200 OK + Set-Cookie - session=abc123
    Note over B: Browser stores cookie
    B->>S: GET /dashboard + Cookie - session=abc123
    S->>S: Look up session abc123
    S->>B: 200 OK + personalized dashboard
    B->>S: GET /settings + Cookie - session=abc123
    S->>B: 200 OK + user settings

Figure 3 - How cookies restore state on top of the stateless HTTP protocol

HTTP/1.1 vs HTTP/2 vs HTTP/3

HTTP/1.1 (1997) introduced persistent connections (keep-alive) so a single TCP connection could carry multiple request-response pairs sequentially. Before this, HTTP/1.0 opened a new TCP connection for every single request - massively wasteful. HTTP/1.1 also added chunked transfer encoding, the Host header (enabling virtual hosting), and pipelining (though pipelining was rarely used in practice due to head-of-line blocking).

HTTP/2 (2015) addressed head-of-line blocking at the HTTP layer through multiplexing. Multiple requests and responses can be in flight simultaneously over a single TCP connection, interleaved as binary frames. HTTP/2 also introduced header compression (HPACK) and server push (allowing the server to proactively send resources the client has not yet requested).

HTTP/3 (2022) replaces TCP entirely with QUIC, a transport protocol built on UDP. QUIC eliminates TCP-level head-of-line blocking because each stream is independent - a lost packet on one stream does not block others. QUIC also integrates TLS 1.3 directly into the transport handshake, reducing connection setup to a single round trip (or zero round trips for resumed connections).

Why HTTP Matters for Security

Understanding HTTP is critical for network security work. Captive portals - like those the BLEShark Nano can create - rely on intercepting HTTP requests and redirecting them. When a device connects to a network and tries to reach an HTTP URL, the captive portal intercepts that request and returns a redirect (302) to its own login page. This only works with plain HTTP; HTTPS connections will show certificate errors instead of redirecting cleanly.

Many IoT devices serve their configuration interfaces over HTTP on local networks. The BLEShark Nano itself hosts a web interface over HTTP on its local IP, using the request-response cycle to accept scan configurations and return results. Every button click on that interface triggers an HTTP request, and every scan result comes back as an HTTP response.

HTTP is also the foundation for REST APIs, webhook callbacks, firmware update checks, and virtually every web-based interaction in modern networking. Knowing exactly what goes into a request and response - and what each header does - gives you the ability to analyze, debug, and secure network traffic at the application layer.