Architecture

The zephyr repository serves as extraction engine within the Zephyr ecosystem. For how a request gets here in the first place (frontend → GitHub issue → this pipeline), see The Zephyr Ecosystem and Using the Web Interface.

Repository Structure

zephyr/
├── src/zephyr/                  # Core Python package
│   ├── process_manager.py       # Orchestrates the full extraction workflow
│   ├── file_fetch.py            # Locates NetCDF files matching a request
│   ├── fileload.py              # Opens files and detects/normalises grid type
│   ├── path_constructor.py      # Builds directory paths from query options
│   ├── date_range_selector.py   # Filters files by date range
│   ├── nearest_grid_point.py    # Nearest-grid-point extraction logic
│   ├── rectangular_clipping.py  # Rectangular clipping logic
│   ├── unrotation.py            # Rotated-grid -> regular lat/lon conversion
│   ├── transform_lcc.py         # LCC -> regular lat/lon conversion
│   ├── add_attributes.py        # Adds version attributes to output datasets
│   └── version.py               # Package version
├── run/
│   └── launch_zephyr.py         # CLI entry point
├── tests/
│   ├── zephyr/                  # Unit tests for core extraction logic
│   ├── find_files/              # Tests for file-finding and path logic
│   └── requests/                # Integration tests for each dataset type
├── tools/
│   └── setup.sh                 # Environment setup script (uv sync + Jupyter kernel)
├── pyproject.toml               # Package metadata and dependencies (runtime, test, docs)
└── uv.lock                      # Pinned, resolved dependency versions

Processing Flow

launch_zephyr.py hands off to ProcessManager (src/zephyr/process_manager.py), which drives the whole pipeline for a single request:

        flowchart TD
    A["Read JSON request"] --> B["file_fetch.FileExtractor:<br/>find matching files under /net/atmos/data"]
    B --> C["fileload.read_file:<br/>open file, detect grid type"]
    C --> D{"Grid type?"}
    D -->|"Rotated"| E["unrotation.py"]
    D -->|"LCC"| F["transform_lcc.py"]
    D -->|"Regular"| G["(no transform)"]
    E --> H["extraction_query_options.selected_option"]
    F --> H
    G --> H
    H -->|"nearest_grid_point"| I["nearest_grid_point.py"]
    H -->|"rectangular_clipping"| J["rectangular_clipping.py"]
    I --> K["add_attributes.py:<br/>zephyr / zephyr-request version, issue ID"]
    J --> K
    K --> L["Write output NetCDF"]
    

1. file_fetch.FileExtractor turns files_query_options into one or more directory paths (via path_constructor.py’s per-dataset DirectoryPathBuilder subclasses) and, within each, uses date_range_selector.py to keep only the files overlapping the requested start/end.

2. fileload.read_file opens each matching file and detects its grid type (regular / rotated / LCC) — see Supported Datasets.

3. Depending on extraction_query_options.selected_option, process_nearest_gridpoint.py or process_rectangular_clipping.py builds the spatial subset (using nearest_grid_point.py / rectangular_clipping.py, after unrotating or reprojecting the grid if needed).

4. add_attributes.py stamps the output with the zephyr version, and — if provided — the zephyr_request_version and github_issue_id for traceability back to the originating request.

5. The result is written to --output_directory with a filename prefix describing the extraction (see Running an Extraction).