MetDrivers API

For narrative coverage of the transport-binary format and the replay-gate contract, see Binary format.

AtmosTransport.MetDrivers.AbstractMassClosure Type

julia

AbstractMassClosure

Strategy for ensuring column dry mass conservation.

source

AtmosTransport.MetDrivers.AbstractMassFluxMetDriver Type

julia

AbstractMassFluxMetDriver <: AbstractMetDriver

Reads pre-computed mass fluxes (am, bm, cm, m).

source

AtmosTransport.MetDrivers.AbstractMetDriver Type

julia

AbstractMetDriver

Supertype for all meteorological data drivers.

Required methods

julia

total_windows(driver) -> Int
window_dt(driver) -> FT (seconds per met window)
steps_per_window(driver) -> Int
steps_per_window(driver, win_index) -> Int
steps_per_window_schedule(driver) -> Vector{Int}
load_transport_window(driver, win_index)
driver_grid(driver)
air_mass_basis(driver)

source

AtmosTransport.MetDrivers.AbstractRawMetDriver Type

julia

AbstractRawMetDriver <: AbstractMetDriver

Reads raw wind fields and computes mass fluxes on the fly.

source

AtmosTransport.MetDrivers.AbstractTransportWindow Type

julia

AbstractTransportWindow{Basis}

Typed forcing window for one transport interval.

A transport window carries the preprocessed mass/flux forcing that the runtime needs for one met interval. It does not own tracer state.

source

AtmosTransport.MetDrivers.ConvectionForcing Type

julia

ConvectionForcing{CM, DT, TM}

Container for one window (or one substep) of convective mass-flux forcing. Three optional payload slots:

cmfmc — cloud updraft mass flux at level interfaces. Supported layouts are structured (Nx, Ny, Nz+1), face-indexed (ncell, Nz+1), and cubed-sphere panel tuples NTuple{6, <:AbstractArray{FT, 3}} with per-panel shape (Nc, Nc, Nz+1). GCHP / CMFMCConvection consumer.
dtrain — detraining mass flux at layer centers. Supported layouts match cmfmc, with layer-center shape (Nx, Ny, Nz), (ncell, Nz), or NTuple{6} of (Nc, Nc, Nz). When nothing, CMFMCConvection falls through to Tiedtke-style single-flux transport.
tm5_fields :: Union{Nothing, NamedTuple{(:entu, :detu, :entd, :detd)}} — four-field entrainment/detrainment arrays at layer centers (Nx, Ny, Nz). TM5 / TM5Convection consumer.

Invariants (enforced by the inner constructor)

DTRAIN requires CMFMC. dtrain !== nothing ⇒ cmfmc !== nothing. DTRAIN without CMFMC is meaningless (DTRAIN detrains from the updraft mass flux; no mass flux, nothing to detrain).
Dual capability is allowed (plan 18 v5.1 §2.22 Decision 28). A binary may carry both CMFMC and TM5 payloads simultaneously; the sim selects which capability to consume based on the installed operator. v5.1 §2.4's stricter "no mixing CMFMC with TM5" language is superseded by Decision 28's allowed-combinations table. The invariant here only enforces the load-bearing constraint above.
Capability is INVARIANT for the lifetime of a DrivenSimulation (Decision 27). copy_convection_forcing! enforces strict tuple match between dst and src so a mid-run capability toggle raises an error — catches both "stale values" (dst has a field src doesn't) and "missing destination" (src has a field dst doesn't).

Default construction

ConvectionForcing() produces an all-nothing placeholder. This is the initial value of TransportModel.convection_forcing; DrivenSimulation allocates real buffers at construction (plan 18 v5.1 Decision 26) via allocate_convection_forcing_like.

See also

has_convection_forcing — capability probe.
copy_convection_forcing! — per-substep refresh copy.
allocate_convection_forcing_like — sim-construction allocation.

source

AtmosTransport.MetDrivers.CubedSphereBinaryHeader Type

julia

CubedSphereBinaryHeader

Parsed header for a cubed-sphere transport binary.

source

AtmosTransport.MetDrivers.CubedSphereBinaryReader Type

julia

CubedSphereBinaryReader{FT}

Reader for cubed-sphere transport binaries. Data is memory-mapped for zero-copy access to per-window payloads.

source

AtmosTransport.MetDrivers.DiagnoseVerticalFromHorizontal Type

julia

DiagnoseVerticalFromHorizontal <: AbstractMassClosure

Diagnose vertical fluxes from horizontal convergence + pressure tendency. This is the default for ERA5 (spectral or gridded) and the standard approach in TM5: cm is not read from data but computed from am, bm, and the B coefficients of the vertical coordinate.

source

AtmosTransport.MetDrivers.NativeVerticalFluxClosure Type

julia

NativeVerticalFluxClosure <: AbstractMassClosure

Use natively provided vertical fluxes (e.g., omega or etadot from the host model). No diagnosis needed.

source

AtmosTransport.MetDrivers.PBLSurfaceForcing Type

julia

PBLSurfaceForcing(pblh, ustar, hflux, t2m)

Container for raw surface fields used by the PBL diffusion closure.

Fields are topology-shaped 2D arrays:

structured: (Nx, Ny)
face-indexed: (ncell,) when such a path is added
cubed sphere: NTuple{6, <:AbstractMatrix} with one (Nc, Nc) panel

Units follow the canonical runtime contract:

pblh - boundary-layer height [m]
ustar - friction velocity [m s^-1]
hflux - upward sensible heat flux [W m^-2]
t2m - 2 m air temperature [K]

source

AtmosTransport.MetDrivers.PressureTendencyClosure Type

julia

PressureTendencyClosure <: AbstractMassClosure

Use the surface pressure tendency dp/dt to close the vertical budget.

source

AtmosTransport.MetDrivers.StreamingTransportBinaryWriter Type

julia

StreamingTransportBinaryWriter{FT}

Handle for incrementally writing transport-binary windows to disk without holding all windows in memory. Created by open_streaming_transport_binary, each window is written via write_streaming_window!, and the file is finalised by close_streaming_transport_binary!.

Memory footprint: one elems_per_window-length pack buffer (Vector{FT}) plus the open IOStream. All other per-window data is owned by the caller.

source

AtmosTransport.MetDrivers.TransportBinaryContract Type

julia

TransportBinaryContract(; source_flux_sampling, air_mass_sampling,
                          flux_sampling, flux_kind, delta_semantics,
                          humidity_sampling,
                          poisson_balance_target_scale,
                          poisson_balance_target_semantics)

Self-describing transport-binary timing/basis contract. All eight fields are required — no defaults — so a writer cannot produce an ambiguous binary. Readers call validate_transport_contract! on the parsed header to decide whether the file is trustworthy.

Canonical usage: construct via canonical_window_constant_contract for the memo-37 path (tracer drift = 0 on uniform IC for Upwind over 2 days).

Symbol fields are validated against the _TRANSPORT_ALLOWED_* tuples at construction time. Combinations are also checked:

delta_semantics === :forward_window_endpoint_difference requires the payload to carry dm (or dm + dhflux); the writer is responsible for honoring this.
humidity_sampling === :window_endpoints requires qv_start + qv_end in the payload; :single_field requires qv; :none requires neither.

source

AtmosTransport.MetDrivers.TransportBinaryDriver Type

julia

TransportBinaryDriver

Standalone src met driver backed by a topology-generic transport binary.

source

AtmosTransport.MetDrivers.TransportBinaryHeader Type

julia

TransportBinaryHeader

Metadata for a topology-generic preprocessed transport binary.

Key fields

grid_type: :latlon or :reduced_gaussian
horizontal_topology: :structureddirectional (LL, arrays (Nx, Ny, Nz)) or :faceindexed (RG, arrays (ncell, Nz) + (nface_h, Nz))
ncell: total horizontal cells (LL: Nx × Ny; RG: sum of nlon_per_ring)
nface_h: total horizontal faces (LL: (Nx+1)×Ny + Nx×(Ny+1); RG: ncell + Σ boundary_counts)
nlevel: number of vertical levels (k=1 = TOA, k=nlevel = surface)
nwindow: windows per day (typically 24 for hourly ERA5)
A_ifc: hybrid A coefficients [Pa] at nlevel + 1 half-levels. p_half[k] = A_ifc[k] + B_ifc[k] × ps. For ERA5 tropo34: A_ifc[1] = 0 (TOA), A_ifc[end] = 0 (surface where B=1).
B_ifc: hybrid B coefficients [dimensionless] at nlevel + 1 half-levels. B_ifc[1] = 0 (TOA, pure pressure levels), B_ifc[end] = 1.0 (surface, pure sigma level).
mass_basis: :moist or :dry — determines whether the stored air mass includes or excludes water vapour.
flux_kind: :substep_mass_amount — stored flux is the mass [kg] per substep (divide by steps_per_window to get per-window total).
flux_sampling: :window_constant — same flux applied at every substep within a window.

source

AtmosTransport.MetDrivers.TransportBinaryReader Type

julia

TransportBinaryReader{FT, DiskFT}

Reader for topology-generic preprocessed transport binaries.

Implemented combinations:

grid_type = :latlon, horizontal_topology = :structureddirectional
grid_type = :reduced_gaussian, horizontal_topology = :faceindexed

source

AtmosTransport.MetDrivers.allocate_convection_forcing_like Method

julia

allocate_convection_forcing_like(src::ConvectionForcing, backend_hint) -> ConvectionForcing

Build a destination ConvectionForcing whose array fields are similar(src_field) — same shape, same element type, same backend (inferred from backend_hint, typically model.state.air_mass). Capability (which fields are non-nothing) exactly matches src.

Used by DrivenSimulation construction (plan 18 v5.1 Decision 26) to seed model.convection_forcing from the first loaded window. After this step, copy_convection_forcing! reuses the same buffers across all subsequent substeps.

The all-nothing placeholder ConvectionForcing() produces another all-nothing placeholder when run through this helper.

source

AtmosTransport.MetDrivers.binary_capabilities Method

julia

binary_capabilities(reader) -> NamedTuple

Summarise what operators this binary can drive. Works on either TransportBinaryReader (LL/RG) or CubedSphereBinaryReader (CS) via the existing has_flux_delta, has_tm5_convection, has_cmfmc, and has_qv predicates. Fields:

advection :: Bool — always true (m, am, bm, cm are required).
replay_gate :: Bool — plan-39 dam/dbm/dcm/dm present.
tm5_convection :: Bool — entu/detu/entd/detd all present.
cmfmc_convection :: Bool — cmfmc present (CS only; LL/RG returns false).
surface_pressure :: Bool — ps present.
humidity :: Bool — qv or qv_start/qv_end present.
mass_basis :: Symbol — :dry or :moist.
grid_type :: Symbol — :latlon / :reduced_gaussian / :cubed_sphere.
payload_sections :: Vector{Symbol} — raw set for debugging.

source

AtmosTransport.MetDrivers.build_air_mass! Function

julia

build_air_mass!(cell_mass, met::MetState, grid::AtmosGrid,
                driver::AbstractMetDriver)

Compute dry air mass from surface pressure and humidity. m_dry[i,j,k] = Δp[k](ps) × area[i,j] / g × (1 - q[i,j,k])

source

AtmosTransport.MetDrivers.build_dry_fluxes! Function

julia

build_dry_fluxes!(fluxes::AbstractFaceFluxState, cell_mass,
                  met::MetState, grid::AtmosGrid,
                  driver::AbstractMetDriver,
                  closure::AbstractMassClosure)

Build dry face mass fluxes from meteorological fields.

This is the single most important interface function in the architecture. After this call, fluxes contains horizontal and vertical dry mass fluxes whose storage layout matches the mesh's flux_topology:

Structured meshes (StructuredFluxTopology) → directional arrays
Unstructured meshes (FaceIndexedFluxTopology) → face-indexed array

cell_mass is filled with dry air mass per cell [kg].

Each met driver provides a specialized method dispatching on the concrete flux state type and grid type appropriate for its target mesh.

source

AtmosTransport.MetDrivers.canonical_window_constant_contract Method

julia

canonical_window_constant_contract(; steps_per_window,
                                     humidity_sampling = :none,
                                     source_flux_sampling = :window_start_endpoint,
                                     include_flux_delta = true) -> TransportBinaryContract

Build the canonical contract for the validated memo-37 path (flux_sampling = :window_constant, per-substep mass amounts). The Poisson target scale is 1 / (2 * steps_per_window) — matching the TM5 r1112 horizontal-sweep count of 2 * steps_per_window per window.

include_flux_delta = true implies delta_semantics = :forward_window_endpoint_difference (the writer must include dm in the payload); false implies :none.

source

AtmosTransport.MetDrivers.close_streaming_transport_binary! Method

julia

close_streaming_transport_binary!(writer) -> String

Flush and close the streaming transport binary. Returns the file path. Warns if the number of windows written does not match the expected count.

source

AtmosTransport.MetDrivers.copy_convection_forcing! Method

julia

copy_convection_forcing!(dst::ConvectionForcing, src::ConvectionForcing) -> dst

Copy src's arrays into dst's preallocated buffers in place. Preserves === identity of the destination's arrays — this is what makes the per-substep refresh zero-allocation after the sim-construction allocation step (Decision 26).

Enforces strict capability match first (Decision 27): both sides must have identical _cap(...) tuples. Otherwise throws ArgumentError. This catches both directions of mismatch (dst has a field src lacks, or vice versa) — both are silent correctness hazards.

Used by DrivenSimulation._refresh_forcing! (Commit 8) to populate sim.model.convection_forcing from sim.window.convection each substep.

source

AtmosTransport.MetDrivers.cs_window_count Method

julia

cs_window_count(reader) -> Int

Number of time windows in the binary.

source

AtmosTransport.MetDrivers.current_time Method

julia

current_time(meteo) -> Float64

Simulation time [s] at the start of the next step. Threaded through operator apply! methods by plan 17 Commit 4:

julia

apply!(state, meteo, grid, op, dt; workspace)

Every operator that consumes time (ExponentialDecay rates, ImplicitVerticalDiffusion Kz refresh, future emission-rate StepwiseFields, etc.) reads current_time(meteo) once per call and passes the resulting scalar to each update_field!(f, t).

Canonical usage (plan 18 A3)

Production: meteo = sim::DrivenSimulation; returns sim.time, advanced by sim.time += sim.Δt at the end of each step!(sim). See src/Models/DrivenSimulation.jl.
Unit tests without a sim: meteo = nothing; returns 0.0.
Legacy driver stub (meteo = ::AbstractMetDriver): returns 0.0. Retained for backward compatibility but should not be relied upon — the driver is stateless (struct holds only the reader + grid) and cannot provide real time information. Any code that previously passed meteo = sim.driver silently got 0.0; plan 18 A3 changes DrivenSimulation.step! to pass meteo = sim so the sim's clock is the canonical source.

source

AtmosTransport.MetDrivers.flux_interpolation_mode Method

How should flux forcing vary within a met window?

source

AtmosTransport.MetDrivers.has_convection_forcing Method

julia

has_convection_forcing(forcing::ConvectionForcing) -> Bool

Whether forcing carries any non-nothing payload. Returns false for the all-nothing placeholder. Used by _refresh_forcing! as a gate so the copy path is skipped for models without an active convection operator.

The window-level overload has_convection_forcing(window) = window.convection !== nothing is defined in TransportBinaryDriver.jl alongside the window struct extensions (plan 18 Commit 2).

source

AtmosTransport.MetDrivers.has_tm5_convection Method

julia

has_tm5_convection(r::TransportBinaryReader) -> Bool

true if the binary carries all four TM5 convection sections (entu, detu, entd, detd) — the contract enforced by the preprocessor when tm5_convection = true. Used by the TransportBinaryDriver to decide whether to populate ConvectionForcing.tm5_fields on loaded windows.

source

AtmosTransport.MetDrivers.inspect_binary Method

julia

inspect_binary(path; io = stdout) -> NamedTuple

Open the binary at path (auto-detecting LL/RG vs CS format), print a capability-augmented report to io, and return the binary_capabilities NamedTuple for programmatic consumption (tests, CLI capability-intersection, folder-level validation).

Obsolete format_version < TRANSPORT_BINARY_FORMAT_VERSION files are rejected here just like runtime drivers; inspect the raw JSON header with external tools if a stale file must be audited.

source

AtmosTransport.MetDrivers.load_cs_window Method

julia

load_cs_window(reader, win) -> NamedTuple

Load window win from a cubed-sphere transport binary. Returns NTuples of per-panel arrays plus optional cmfmc / dtrain payloads when they are present in the binary.

source

AtmosTransport.MetDrivers.load_surface_window! Method

julia

load_surface_window!(reader::CubedSphereBinaryReader, win) -> PBLSurfaceForcing | nothing

Load the raw PBL surface payload for one CS window. This is a convenience wrapper over load_cs_window; callers that already need the advection fields should use load_cs_window(reader, win).surface to avoid a second payload read.

source

AtmosTransport.MetDrivers.load_surface_window! Method

julia

load_surface_window!(reader, win; pblh=..., ustar=..., hflux=..., t2m=...) -> PBLSurfaceForcing | nothing

Load raw PBL surface fields for window win. Returns nothing when the binary lacks surface sections. A binary carrying only a subset of pblh, ustar, pbl_hflux, and t2m is rejected because the runtime PBL closure needs the complete raw surface payload. The on-disk heat-flux section is pbl_hflux; callers still receive it as PBLSurfaceForcing.hflux.

source

AtmosTransport.MetDrivers.load_tm5_convection_window! Method

julia

load_tm5_convection_window!(reader, win; entu=..., detu=..., entd=..., detd=...) -> NamedTuple | nothing

Load the four TM5 convection layer-center fields for window win. Returns (; entu, detu, entd, detd) when the binary carries all four sections, or nothing when no TM5 data is present. Allocates only if the caller doesn't provide pre-allocated buffers.

All fields share the same shape as m: (Nx, Ny, Nz) for structured or (ncells, Nz) for face-indexed binaries. Orientation is as written by the preprocessor (AtmosTransport: k=1=TOA, k=Nz=surface); no runtime reorientation happens here — the kernel (plan 23 Commit 4) reads them directly.

Invariant: if ANY of the four sections is present in the header, ALL four must be present. This mirrors the ConvectionForcing.tm5_fields NamedTuple contract — partial payload is not a valid convection forcing.

source

AtmosTransport.MetDrivers.load_transport_window Method

julia

load_transport_window(driver, win)

Load one typed forcing window from the transport binary.

source

AtmosTransport.MetDrivers.mesh_convention Method

julia

mesh_convention(reader::CubedSphereBinaryReader) -> AbstractCubedSpherePanelConvention

Return the panel-numbering convention declared in the binary header.

Returns GnomonicPanelConvention() for ERA5-CS binaries and GEOSNativePanelConvention() for GEOS-FP/IT binaries tagged with panel_convention="geos_native". Callers should pass the result directly to CubedSphereMesh(; convention=mesh_convention(reader)) to guarantee that the halo exchange uses the correct edge-to-edge connectivity table.

source

AtmosTransport.MetDrivers.mesh_definition Method

julia

mesh_definition(reader::CubedSphereBinaryReader) -> CubedSphereDefinition

Return the full cubed-sphere geometry definition declared in the binary header. New binaries record cs_coordinate_law, cs_center_law, panel_convention, and longitude_offset_deg; older binaries are upgraded by convention (geos_native -> GMAO equal-distance, gnomonic -> legacy equiangular).

source

AtmosTransport.MetDrivers.open_streaming_transport_binary Method

julia

open_streaming_transport_binary(path, grid::AtmosGrid{<:ReducedGaussianMesh},
                                nwindow, sample_window; kwargs...)

Open a transport binary file for streaming (per-window) writes on a reduced-Gaussian grid.

sample_window is a NamedTuple with the same keys as the windows that will be written (e.g. (m=..., hflux=..., cm=..., ps=...)). Its arrays must have the correct sizes but their values are ignored — it is only used to determine payload_sections and to validate dimensions.

Returns a StreamingTransportBinaryWriter.

source

AtmosTransport.MetDrivers.recompute_cm_from_dm_target! Method

julia

recompute_cm_from_dm_target!(cm, am, bm, m, dm_target)

Structured-directional convenience wrapper for the shared explicit-dm continuity closure.

source

AtmosTransport.MetDrivers.recompute_cm_from_dm_target! Method

julia

recompute_cm_from_dm_target!(layout, div_h, cm, m, dm_target, flux_args...)

Shared explicit-dm continuity closure. The topology-specific code only builds div_h; the vertical integration and residual redistribution are common.

source

AtmosTransport.MetDrivers.recompute_faceindexed_cm_from_dm_target! Method

julia

recompute_faceindexed_cm_from_dm_target!(cm, hflux, face_left, face_right,
                                         div_scratch, m, dm_target)

Face-indexed convenience wrapper for the shared explicit-dm continuity closure.

source

AtmosTransport.MetDrivers.supports_convection Method

Does this driver provide convective mass flux / detrainment for convection?

source

AtmosTransport.MetDrivers.supports_diffusion Method

Does this driver provide diffusivity fields for boundary-layer diffusion?

source

AtmosTransport.MetDrivers.supports_moisture Method

Does this driver provide specific humidity for dry-mass correction?

source

AtmosTransport.MetDrivers.supports_native_vertical_flux Method

Does this driver provide native vertical mass fluxes (vs diagnosing from continuity)?

source

AtmosTransport.MetDrivers.uses_binary_substep_contract Method

Does this driver provide a verified per-window timestep contract?

source

AtmosTransport.MetDrivers.validate_transport_contract! Method

julia

validate_transport_contract!(header::AbstractDict; allow_legacy::Bool = false)

Assert that header declares the current transport-binary contract and that the timing metadata is self-consistent. format_version is a hard boundary: only TRANSPORT_BINARY_FORMAT_VERSION is accepted. Older files are obsolete and must be regenerated rather than loaded through compatibility defaults.

Shared between TransportBinaryDriver, TransportBinaryReader, and the scripts/diagnostics/inspect_transport_binary.jl tool so there is ONE validator every reader-facing tool calls. allow_legacy is retained for API compatibility but no longer bypasses the current runtime contract.

source

AtmosTransport.MetDrivers.verify_window_continuity Method

julia

verify_window_continuity(div_h, m_cur, cm, m_next, steps_per_window)
verify_window_continuity(layout, div_h, m_cur, cm, m_next, steps_per_window, flux_args...)

Replay one window's stored mass-flux data against the stored air-mass endpoints. The first form assumes div_h is already populated; the second builds it through the topology-specific layout.

source

AtmosTransport.MetDrivers.verify_window_continuity_cs Method

julia

verify_window_continuity_cs(m_cur, am, bm, cm, m_next, steps_per_window)

Cubed-sphere convenience wrapper for the shared replay kernel. The caller must provide panel-local arrays with synchronized boundary mirrors so each panel's structured divergence matches the closed-sphere flux field.

source

AtmosTransport.MetDrivers.verify_window_continuity_ll Method

julia

verify_window_continuity_ll(m_cur, am, bm, cm, m_next, steps_per_window)

Structured-directional convenience wrapper for the shared replay kernel.

source

AtmosTransport.MetDrivers.verify_window_continuity_rg Method

julia

verify_window_continuity_rg(m_cur, hflux, cm, m_next, face_left, face_right,
                            div_scratch, steps_per_window)

Face-indexed convenience wrapper for the shared replay kernel.

source

AtmosTransport.MetDrivers.write_streaming_window! Method

julia

write_streaming_window!(writer, window)

Pack and write a single window to the streaming transport binary. Windows must be written in order (1, 2, …, nwindow).

source

MetDrivers API ​

MetDrivers API