Downloading files

This document describes how file downloads work in EDRI, including resumable downloads (HTTP Range) and high-throughput streaming via SharedMemoryPipe.

Two common download strategies

EDRI supports two primary ways to serve downloads:

  1. File-backed response (seekable, size-known)

    • best for classic HTTP downloads

    • supports Range resume automatically

  2. Streamed response (on-the-fly generation)

    • best for generating large outputs dynamically (e.g. zipping many files)

    • can use SharedMemoryPipe for throughput

    • does not support HTTP Range resume (size may be unknown)

Streamed downloads (on-the-fly generation)

Some downloads are too large to materialize first, or you want to generate them dynamically.

Example use cases:

  • creating a ZIP archive on the fly from many files

  • streaming a generated report

  • exporting large datasets

SharedMemoryPipe

SharedMemoryPipe provides a high-throughput tunnel from a Worker to an API client, implemented using:

  • POSIX shared memory

  • POSIX semaphores

This avoids copying large payloads through intermediate processes.

See SharedMemoryPipe for full details.

Lifetime

SharedMemoryPipe instances are per-request:

  • created for one request/response flow

  • used to transfer the payload

  • cleaned up when the request completes

Resume limitation

Classic HTTP resume via Range requires a seekable, fixed-size representation.

For streamed outputs where total size is unknown (or not stable), HTTP Range resume is not supported.

Declaring a file download

EDRI provides a File helper type for describing downloadable files.

Below are two short examples showing both supported approaches.

Event and response definition

from edri.api.dataclass import File
from edri.dataclass.event import Event, event
from edri.dataclass.response import Response, response


@response
class DownloadArchiveResponse(Response):
    file: File


@event
class DownloadArchive(Event):
    response: DownloadArchiveResponse

Option A: Path-backed download (resume-friendly)

Use this when the file exists on disk or you can generate it fully before responding.

from pathlib import Path

self.event.response.file = File(
    file_name="archive.zip",
    mime_type="application/zip",
    path=Path("/tmp/archive.zip"),
)

✔ Supports HTTP Range resume (206 Partial Content).

Option B: Streamed download via SharedMemoryPipe (throughput-friendly)

Use this when generating the output on the fly (e.g. zipping a large folder).

from edri.api.dataclass import File
from edri.utility.shared_memory_pipe import SharedMemoryPipe

with SharedMemoryPipe() as smp:
    self.event.response.file = File(
        file_name="archive.zip",
        mime_type="application/zip",
        path=smp.reader(),
    )
    self.event_send(self.event)

✔ Ideal for very large dynamic outputs ✖ Does NOT support HTTP Range resume (size unknown upfront)

Response headers

EDRI sets the following headers for file downloads.

Common headers (all downloads)

For both Path-backed and streamed downloads:

  • Content-Type is set to the file’s MIME type.

  • Content-Disposition is set to attachment and includes a UTF-8 encoded filename using the filename*= syntax.

This ensures correct media type handling and proper filename support across browsers.

Path-backed downloads (pathlib.Path)

Additional headers:

  • Accept-Ranges: bytes – indicates that byte-range requests are supported.

  • ETag – typically based on the file fingerprint. Required for proper HTTP Range resume behavior (must match If-Range).

If the client sends a satisfiable Range header with a matching If-Range:

  • Content-Length is set to the size of the returned range.

  • Content-Range describes the returned byte range (bytes <start>-<end>/<size>).

  • HTTP status: 206 Partial Content.

If the range is well-formed but unsatisfiable (matching If-Range):

  • Content-Range is bytes */<size>.

  • HTTP status: 416 Range Not Satisfiable (empty body).

If no Range header is present, If-Range does not match, or the Range is malformed:

  • Content-Length is set to the total file size.

  • HTTP status 200 OK is returned.

Streamed downloads (SharedMemoryPipe)

For streamed responses:

  • If the total size is known up-front, Content-Length is set.

  • If the total size is unknown, EDRI falls back to Transfer-Encoding: chunked.

Because streamed outputs are not necessarily seekable and may not have a stable total size, HTTP Range resume is not supported in this mode. There is also no File.fingerprint to compute up front — the content is produced incrementally — so the If-Range validator the resume path depends on cannot exist.

Implementation note

SharedMemoryPipe is not selected automatically by EDRI. They are explicitly created by the developer — typically inside a Worker.

The intended pattern is:

  • A Worker creates the SharedMemoryPipe

  • The Worker assigns the pipe reader to event.response.file

  • The Worker sends the event

  • The Worker streams bytes into the pipe

This design keeps heavy data transfer out of the Manager thread and enables high-throughput streaming directly from the Worker to the API client.