Skip to content

GriddedIO API ​

AtmosTransport.GriddedIO owns the conventions of native gridded input files (surface-flux inventories, land fractions, initial-condition fields). Each rule is a type the caller names, and the methods dispatch on it:

The rules were measured on the TRENDY v14, GFED5, FLUXCOM-X, CAMS, GridFED and EDGAR files; src/GriddedIO/README.md lists them with their invariants.

AtmosTransport.GriddedIO Module
julia
GriddedIO

The one owner of the conventions of native gridded input files (surface-flux inventories, land fractions, initial-condition fields): how their coordinates are found, which values are fill, how a latitude-longitude field is oriented, which cell faces its coordinates imply, how its time axis is decoded, how a compressed input is staged, and how its flux units are read. It also owns the schema of the surface-flux files the model writes and the runtime reads (cs_native and lat-lon), with their one writer and reader.

Every rule here was measured on the files the model reads (the TRENDY v14 S3 models, GFED5, FLUXCOM-X, CAMS/LMDZ, GridFED, EDGAR); the measurements are in docs/memos/2026-10-10_python_flux_port_p0_facts.md. The rules are typed: the caller names an axis (LongitudeAxis), a fill policy (MaskedAsZero), a time decoder (CFDecoder), and the methods dispatch on it.

Contents

Molar masses and the year length come from Parameters (SPECIES_MOLAR_MASS, CARBON_MOLAR_MASS, DAYS_PER_JULIAN_YEAR).

src/AtmosTransport.jl loads this module after Regridding and before Preprocessing; it imports Parameters, Grids, Output (the shared cubed-sphere geometry and provenance blocks) and AtomicFiles. See src/GriddedIO/README.md for the file map and invariants.

source
AtmosTransport.GriddedIO.CS_NATIVE_FLUX_DIMS Constant
julia
CS_NATIVE_FLUX_DIMS

Dimension names of a cs_native flux series in Julia order, ("Xdim", "Ydim", "nf", "time"); in NetCDF (CF) order (time, nf, Ydim, Xdim). Xdim and Ydim index the cells of a panel and nf the six panels, in the panel convention of the mesh (Output.define_cs_horizontal_geometry!).

source
AtmosTransport.GriddedIO.DEFAULT_TIME_DECODERS Constant

The decoders decode_time uses unless given others: (CFDecoder(), MonthsSince()). Axes without usable units need a decoder named per product.

source
AtmosTransport.GriddedIO.STAMP_SPACING_TOLERANCE Constant
julia
STAMP_SPACING_TOLERANCE

Spacings of interval-start stamps that differ by at most this much are the same step in stamp_intervals (float stamps such as days since eighths decode to the millisecond).

source
AtmosTransport.GriddedIO.AbstractAreaBasis Type
julia
AbstractAreaBasis

What a flux is per: PerSquareMetre (a density) or PerCell (a per-cell rate; the caller divides by the cell area).

source
AtmosTransport.GriddedIO.AbstractAxis Type
julia
AbstractAxis

A coordinate axis of a gridded input file: LongitudeAxis, LatitudeAxis, TimeAxis or VerticalAxis. find_coordinate dispatches on it.

source
AtmosTransport.GriddedIO.AbstractFaceRule Type
julia
AbstractFaceRule

How source_latlon_mesh derives cell faces from a file's cell centres: SnappedFaces (the default) or UniformSpanFaces.

source
AtmosTransport.GriddedIO.AbstractFillPolicy Type
julia
AbstractFillPolicy

Which attributes of a variable mark fill values, and what read_raw puts in their place:

PolicyFill marked byFill becomesUse
MaskedAsZero_FillValue and missing_value0.0runtime flux loaders; FLUXCOM-X land fraction
MaskedAsNaN_FillValue and missing_valueNaNflux sources whose statistics count only valid cells
FillValueOnly_FillValue onlyNaNvariables whose missing_value is wrong (IBIS land fraction)

A _FillValue of NaN matches NaN values. scale_factor and add_offset are applied as NCDatasets applies them (no flux variable of the measured inputs has them). valid_min, valid_max and valid_range are not applied.

source
AtmosTransport.GriddedIO.AbstractMonthStamp Type
julia
AbstractMonthStamp

Where a monthly time stamp sits in its month, for DecimalYears and AuxiliaryTimeVariable: MonthStart, MidMonth or MonthEnd. Which one a product uses is a fact about the product, named in its configuration; the catalog's last_month check (check_last_month) catches a wrong choice.

source
AtmosTransport.GriddedIO.AbstractSubstance Type
julia
AbstractSubstance

What a flux amount counts: TargetSpecies, Carbon or a named Species.

source
AtmosTransport.GriddedIO.AbstractTimeBasis Type
julia
AbstractTimeBasis

The time unit of a flux: PerSecond, PerDay, PerMonth or PerYear. period_seconds gives its length.

source
AtmosTransport.GriddedIO.AbstractTimeDecoder Type
julia
AbstractTimeDecoder

A rule that turns a time axis into DateTime stamps. decode_time classifies the axis by its raw units string and uses the first decoder of its list that accepts that class:

Raw unitsDecoder
days/hours/minutes/seconds since <origin>CFDecoder
months since <origin>MonthsSince
none, years since …, or unparseableDecimalYears or AuxiliaryTimeVariable, named per product
source
AtmosTransport.GriddedIO.AuxiliaryTimeVariable Type
julia
AuxiliaryTimeVariable(variable, origin::Date, unit::Dates.Period, stamp::AbstractMonthStamp)

Decoder of a monthly time axis that is only an index (no units), whose stamps are in another variable of the file. CARDAMOM: time = 1 … 264; the Time variable (units "d", long name "Monthly time step given in days since 01/01/2003") holds 31, 59, 90, …, 8036, the end of each month, so AuxiliaryTimeVariable("Time", Date(2003, 1, 1), Day(1), MonthEnd()) gives 2003-01 … 2024-12. Stamp k is DateTime(origin) + v_k · unit, rounded to the millisecond, where v_k is value k of variable, which must have as many values as the axis; it decodes to the first instant of the month it belongs to under stamp.

source
AtmosTransport.GriddedIO.CFDecoder Type
julia
CFDecoder()

Decoder of <unit> since <origin> axes whose unit has a fixed length (day, hour, minute, second, millisecond; also the abbreviations d, h, hr, min, s, sec), through CFTime, honouring the calendar attribute:

  • standard, gregorian, proleptic_gregorian, julian or no calendar: NCDatasets' own decoding, rounded to the millisecond (the runtime's stamps, bit for bit; a julian axis keeps its instants, so its dates shift by the Julian–Gregorian difference);

  • noleap/365_day, all_leap/366_day, 360_day: the calendar date and time of day of each stamp in its own calendar (values rounded to the millisecond first), carried over to a DateTime. Every noleap/365_day date exists in the proleptic Gregorian calendar; an all_leap/366_day or 360_day date that does not (29 February of a common year, 30 February) is an ArgumentError naming it. decode_month_keys takes the months of such axes from the calendar dates themselves.

A trailing " UTC" on the origin is dropped. Fill values in the axis are an error.

source
AtmosTransport.GriddedIO.CSNativeFluxWriter Type
julia
CSNativeFluxWriter

A cs_native flux file being written, returned by open_cs_native_writer. Slices go in with write_slices!; publish! completes the file and renames it onto its path, and abandon! discards it. Until then only a staging file next to the target exists (AtomicFiles), so readers of the path see the previous file (or none), never a partial one.

source
AtmosTransport.GriddedIO.Carbon Type
julia
Carbon()

Amounts of carbon (kgC, g C, mol C). Converted to a carbon-bearing species with one carbon atom per molecule by M(species) / CARBON_MOLAR_MASS, which conserves carbon atoms (CO₂: SPECIES_MOLAR_MASS.co2 / CARBON_MOLAR_MASS ≈ 3.66419).

source
AtmosTransport.GriddedIO.DecimalYears Type
julia
DecimalYears(stamp::AbstractMonthStamp)

Decoder of monthly axes stored as decimal years without usable units (ELM-FATES, month starts: DecimalYears(MonthStart()); VISIT-UT, mid-month, units years since AD 0-Jan-1st (double): DecimalYears(MidMonth())). With m = 12 v months since year 0, the month index is k̂ = round(m − o) (o = 0 for month starts, ½ for mid-month stamps, 1 for month ends) and the residual |m − o − k̂| must not exceed 10⁻⁶ month, else an ArgumentError. Rounding to the nearest month is required: ELM-FATES has 1296 values 9.1e-13 month below the integer, where a plain floor gives 2592 bad steps. Each stamp decodes to the first instant of its month.

source
AtmosTransport.GriddedIO.FillValueOnly Type
julia
FillValueOnly()

Fill policy: only _FillValue marks fill (it becomes NaN); missing_value is ignored. For variables whose missing_value attribute is wrong: the IBIS land-fraction file declares missing_value = 0, which is a valid fraction.

source
AtmosTransport.GriddedIO.FluxUnits Type
julia
FluxUnits{S, A, T}

Parsed flux units (parse_flux_units).

Fields

source
AtmosTransport.GriddedIO.LatLonOrientation Type
julia
LatLonOrientation(flip_lat, lon_shift, wrap_lon)

How orient reorders a latitude-longitude field (lon, lat, ...):

  • flip_lat: reverse the latitude axis (the file stores north to south);

  • lon_shift: move the first lon_shift longitude columns to the end (a circular shift by -lon_shift);

  • wrap_lon: map the reordered longitudes into [0°, 360°) (orient_coordinates).

Built by latlon_orientation.

source
AtmosTransport.GriddedIO.LatLonPointElevation Type
julia
LatLonPointElevation

A latitude-longitude digital elevation model read by read_point_elevation: ascending longitudes lon (any convention, spanning less than 360°) and latitudes lat, the ground elevation values[i, j] (m above sea level, Float32, NaN where the file has fill), whether the longitudes wrap around the globe (periodic), and the file path. Output.point_elevation(dem, lon, lat) interpolates it bilinearly between the four surrounding centres.

source
AtmosTransport.GriddedIO.LatitudeAxis Type

Latitude axis; coordinate variable names lat, latitude, y (in that order).

source
AtmosTransport.GriddedIO.LongitudeAxis Type

Longitude axis; coordinate variable names lon, longitude, x (in that order).

source
AtmosTransport.GriddedIO.MaskedAsNaN Type
julia
MaskedAsNaN()

Fill policy: values equal to _FillValue or to any missing_value become NaN, so a caller can count valid values (isfinite) before replacing them. VISIT, VISIT-UT and iMAPLE mark fill only through missing_value = -99999.

source
AtmosTransport.GriddedIO.MaskedAsZero Type
julia
MaskedAsZero()

Fill policy: values equal to _FillValue or to any missing_value become 0.0. This is the runtime convention (nomissing(ds[var][...], 0)).

source
AtmosTransport.GriddedIO.MidMonth Type

Stamps in the middle of the month: decimal year y + (k + ½)/12; the stamp lies in its month.

source
AtmosTransport.GriddedIO.MonthEnd Type

Stamps at the end of the month: the first instant of the next month (CARDAMOM's Time = 31, 59, 90, … days since 2003-01-01 for January, February, March, …); decimal year y + (k + 1)/12.

source
AtmosTransport.GriddedIO.MonthKey Type
julia
MonthKey(year, month)
MonthKey(t::Dates.TimeType)
MonthKey("YYYY-MM")

A calendar month of the proleptic Gregorian calendar. Keys order chronologically; k + n is the month n months later.

source
AtmosTransport.GriddedIO.MonthStart Type

Stamps at the start of the month: decimal year y + k/12; the stamp lies in its month.

source
AtmosTransport.GriddedIO.MonthsSince Type
julia
MonthsSince()

Decoder of months since <origin> axes (CABLE-POP, ISAM, LPJml, VISIT, iMAPLE). The month of stamp v is the origin month plus ⌊v + 10⁻⁶⌋. The origin must be a valid date Y-M[-D] with an optional time of day (an ArgumentError otherwise, see decode_time); only its year and month are read (CABLE-POP's origin is 1700-1-16 12:00:00). Integer stamps (month starts) and stamps k + ½ (mid-month) both give month k. Each stamp decodes to the first instant of its month; the calendar attribute does not matter.

source
AtmosTransport.GriddedIO.PerCell Type

Flux per grid cell (no area in the units, or cell-1): GFED g C per month, EDGAR Tonnes.

source
AtmosTransport.GriddedIO.PerDay Type

Per day of 86400 s (d-1, /day).

source
AtmosTransport.GriddedIO.PerMonth Type

Per calendar month (month-1, /month, per month): the length of the month containing the interval.

source
AtmosTransport.GriddedIO.PerSecond Type

Per second (s-1, /s).

source
AtmosTransport.GriddedIO.PerSquareMetre Type

Flux per square metre (m-2, /m2).

source
AtmosTransport.GriddedIO.PerYear Type

Per Julian year of DAYS_PER_JULIAN_YEAR days (yr-1, /year; EDGAR Tonnes).

source
AtmosTransport.GriddedIO.SnappedFaces Type
julia
SnappedFaces()

Face rule of source_latlon_mesh, its default: regular global axes snap to the grid their resolution implies, with faces built from integers; pole-centred, truncated, Gaussian and regional latitudes get midpoint faces with the pole and truncation rules of the source_latlon_mesh docstring.

source
AtmosTransport.GriddedIO.Species Type
julia
Species(name::Symbol)

Amounts of the species name (kgCO2 → Species(:co2)). Names are lower case; the molar mass of a species comes from SPECIES_MOLAR_MASS.

source
AtmosTransport.GriddedIO.TargetSpecies Type
julia
TargetSpecies()

The units name no substance (kg m-2 s-1): the amount is of the output species, the runtime's reading of such files. Molar units cannot be of an unnamed substance.

source
AtmosTransport.GriddedIO.TimeAxis Type

Time axis; coordinate variable names time, t (in that order).

source
AtmosTransport.GriddedIO.TimeInterval Type
julia
TimeInterval(start::DateTime, stop::DateTime)

The half-open time interval [start, stop); start < stop is required.

source
AtmosTransport.GriddedIO.UniformSpanFaces Type
julia
UniformSpanFaces()

Face rule of source_latlon_mesh that gives the uniform mesh of the former runtime surface-flux and initial-condition loaders (Models.InitialConditionIO), bit for bit. The mesh is uniform: for n centres x, the spacing is the mean spacing (x[n] − x[1]) / (n − 1) and the faces lie half a spacing outside the first and last centres. A latitude extent within 1e-3 of a cell of ±90° becomes ±90°; a longitude span within 1e-3 of a cell of 360° becomes exactly 360°, measured from the west face.

Taking the spacing from the whole span, not the first difference, keeps Float32-stored centres from biasing every cell (GridFED stores its 0.1° centres in Float32; their first difference errs by about 1e-4 relative), and the extent snapping keeps the regridder from missing a sliver of the sphere (GridFED: 1.5e-5° of longitude, 2.1e-6 of the global flux). The faces still follow the centres' round-off: GridFED's west face is 7.45e-10°, where SnappedFaces gives 0. The rule assumes uniform, cell-centred centres; pole-centred rows (the 143 LMDZ latitudes of the CATRINE SF6 initial state, CLM, ELM, iMAPLE), truncated and Gaussian latitudes get uniform faces that do not bound their cells.

source
AtmosTransport.GriddedIO.VerticalAxis Type

Vertical axis of initial-condition files; coordinate variable names lev, level, plev, z, hybrid, nhym (in that order).

source
AtmosTransport.GriddedIO.abandon! Method
julia
abandon!(w::CSNativeFluxWriter) -> nothing

Close the writer and remove its staging file; the path keeps its previous file, if any. Abandoning a closed writer does nothing.

source
AtmosTransport.GriddedIO.canonical_units_string Method
julia
canonical_units_string(species::Symbol) -> String

The units string the flux writers use for a density of species: "kg CO2 m-2 s-1" for :co2 ("kg SF6 m-2 s-1", "kg Rn222 m-2 s-1"). parse_flux_units reads it back as kg of Species(species) per m² per second. The species must be one of SPECIES_MOLAR_MASS.

source
AtmosTransport.GriddedIO.check_cell_area Method
julia
check_cell_area(ds, mesh::CubedSphereMesh; file = path of ds) -> nothing

The cell-area gate of a cs_native file. The runtime multiplies the stored densities by mesh.cell_areas; a file prepared on another cube geometry (an equiangular instead of the GMAO cube, another radius) has other cell areas, and its emitted mass would be silently wrong. When the dataset ds has a cell_area variable, shape (Nc, Nc) or (Nc, Nc, 6), each panel a must satisfy isapprox(a, mesh.cell_areas; rtol = 1e-4): the norm-based relative difference ‖a − A‖ ≤ 10⁻⁴ max(‖a‖, ‖A‖) (Frobenius norm over the panel). Fill values in cell_area fail the gate; a file without cell_area passes.

This is the gate the runtime has applied since the cs_native kind exists; legacy files written with Float32-era geometry (per-cell area differences up to 1.75e-4 on C90 polar panels) pass it.

source
AtmosTransport.GriddedIO.check_last_month Method
julia
check_last_month(keys, last_month::MonthKey; context = "time axis") -> keys

Throw an ArgumentError unless the last of the month keys is last_month (a catalog's last_month, e.g. MonthKey(2024, 12) for TRENDY v14); return keys otherwise.

source
AtmosTransport.GriddedIO.check_latlon_cell_area Method
julia
check_latlon_cell_area(area, mesh_area, units; file, rtol = 1e-2, flux = nothing) -> area

The cell-area gate of a lat-lon flux file whose cell_area or area variable sets the inventory's total (area [m²], oriented like the flux). Its units must be square metres ("m2", "m^2", "m**2"; an empty string, for an absent attribute, is accepted), and it must agree with mesh_area, the cell areas of the mesh that the file's coordinates define on the runtime sphere, as isapprox(area, mesh_area; rtol): the norm-based relative difference ‖a − A‖ ≤ rtol max(‖a‖, ‖A‖) (Frobenius norm). The default 1 % passes any Earth-radius convention (6.371 versus 6.378137 × 10⁶ m is 0.22 % in area) and the files' own cell definitions, while areas in km² or a land fraction, which would scale the emitted total, fail. A size mismatch is a DimensionMismatch.

With flux (the flux slice, oriented like area), every cell whose flux is nonzero must have a finite, positive area: the norm test alone passes a few masked (zero) rows, such as polar rows, and their emission would be dropped without a word (per-cell units give a cell without area no flux; per-m² units give it no rate).

source
AtmosTransport.GriddedIO.cs_native_flux_info Method
julia
cs_native_flux_info(path_or_ds, variable; mesh::CubedSphereMesh, slices = :)
    -> (; slices::UnitRange{Int}, times::Vector{DateTime}, units::FluxUnits)

The checks of read_cs_native_flux that need no flux values (layout, time axis, units, cell-area gate) on the cs_native series variable of a file path or an open NetCDF dataset, and the slices slices selects (:, a range of slice indices, or a function of the decoded times returning one): their indices, times and the parsed units. A caller allocates its arrays from length(slices) and streams the values in with read_cs_native_flux!.

source
AtmosTransport.GriddedIO.decode_month_keys Function
julia
decode_month_keys(ds, name, decoders::Tuple = DEFAULT_TIME_DECODERS) -> Vector{MonthKey}

The month of each stamp of the time axis name of ds, in the axis's own calendar: the flux builders' monthly keys. Classification, decoder choice and origin checks are those of decode_time. A CFDecoder axis in noleap/365_day, all_leap/366_day or 360_day gives the year and month of each calendar date without converting it to DateTime, so 30 February of a 360_day axis is February; other axes give month_keys(decode_time(ds, name, decoders)).

source
AtmosTransport.GriddedIO.decode_month_seconds Method
julia
decode_month_seconds(ds, name) -> Vector{Float64}

Length in seconds of the calendar month represented by each slice of time axis name. This is the conversion basis for flux variables whose units are per calendar month.

When the coordinate names a CF bounds variable, each slice is assigned from the midpoint of its bounds. For fixed-length CF units the bounds' actual duration is returned; for months since bounds, one calendar month is required. Without bounds, the coordinate stamp assigns the month. Month lengths use the coordinate's calendar (standard by default), including 360_day, 365_day/noleap, and 366_day/all_leap. Thus neither a slice's ordinal position nor the proleptic-Gregorian month length is guessed.

source
AtmosTransport.GriddedIO.decode_slice_intervals Method
julia
decode_slice_intervals(ds, name) -> Vector{TimeInterval}

The interval of time that each slice of the time axis name of ds stands for: known_slice_intervals when the axis defines them (CF bounds, a months since axis), else stamp_intervals of its stamps (decode_time). A series spread over several files must derive the intervals of its bare stamps from the stamps of all its files together (stamp_intervals(sort(all stamps))): one file of a daily series holds one stamp, whose period no single file tells.

source
AtmosTransport.GriddedIO.decode_time Function
julia
decode_time(ds, name, decoders::Tuple = DEFAULT_TIME_DECODERS) -> Vector{DateTime}

Time stamps of the time axis name of the NetCDF dataset ds. The axis is classified by the units attribute of the raw variable before any CFTime call (a months since axis never reaches CFTime), and decoded by the first decoder in decoders that accepts its class (see AbstractTimeDecoder); an ArgumentError names the decoder that is needed when none does. The length of the axis is never used to guess its meaning, and time bounds are not read.

The origin of <unit> since <origin> units must be a valid instant: an origin Y-M-D[ h:m[:s]] with a month outside 1:12, a day outside 1:31 (or beyond the month's length in the axis's calendar, for CFDecoder) or an invalid time of day is an ArgumentError, never normalised (months since 2000-13-01 is not January 2001). A months since origin must have that form.

The month-resolution decoders (MonthsSince, DecimalYears, AuxiliaryTimeVariable) return the first instant of each month; CFDecoder returns the stamps themselves. DateTime is proleptic Gregorian: a stamp of a non-Gregorian calendar whose date that calendar lacks (30 February of 360_day, 29 February of a common year in all_leap or 360_day) is an ArgumentError naming the date; decode_month_keys gives the months of such an axis.

source
AtmosTransport.GriddedIO.decode_time_bounds Method
julia
decode_time_bounds(ds, name) -> Union{Nothing, Matrix{DateTime}}

Decode the optional CF bounds named by time coordinate name. The returned matrix has shape (2, ntime), with interval starts in row 1 and ends in row 2. nothing means that the coordinate has no bounds attribute. Bounds inherit the coordinate's units and calendar and are validated as finite, ordered intervals before decoding. They are decoded as decode_time decodes the coordinate: a bound of a julian axis, or a standard bound before 1582-10-15, is the same instant in the proleptic Gregorian calendar (Julian 2000-01-01 is 2000-01-14), never its Julian calendar fields.

source
AtmosTransport.GriddedIO.find_coordinate Method
julia
find_coordinate(ds, axis::AbstractAxis; length = nothing, required = true) -> String

Name of the coordinate variable of axis in the NetCDF dataset ds: the first of the axis's candidate names (coordinate_names(axis)) that is a variable of ds. Coordinates are identified by variable name only; units, long_name, standard_name and axis attributes are never consulted, because real files mislabel them (FLUXCOM-X: lon with units = degrees_north) or omit them (GFED5 NRT).

The found variable must be one-dimensional and, when length is given, have length values (the size of the matching dimension of the data variable); otherwise an ArgumentError (not 1-D) or DimensionMismatch (wrong length) is thrown. Bounds variables (lat_bnds, lon_bnds, time_bnds) are never returned. With required = false a missing coordinate returns nothing instead of throwing.

source
AtmosTransport.GriddedIO.interval Method
julia
interval(k::MonthKey) -> TimeInterval

The month k as [first instant of k, first instant of the next month).

source
AtmosTransport.GriddedIO.kg_per_unit Method
julia
kg_per_unit(u::FluxUnits, species::Symbol) -> Float64

Kilograms of species per stated amount unit of u:

  • mass units of the TargetSpecies or of Species(species): amount_per_unit;

  • molar units of Species(species): amount_per_unit × M(species);

  • Carbon: amount_per_unit × M(species) / M(C) (mass) or amount_per_unit × M(species) (molar), for species with one carbon atom (CO₂). For kgC and :co2 this is exactly SPECIES_MOLAR_MASS.co2 / CARBON_MOLAR_MASS.

M is SPECIES_MOLAR_MASS and CARBON_MOLAR_MASS. Units of another species than species throw.

source
AtmosTransport.GriddedIO.known_slice_intervals Method
julia
known_slice_intervals(ds, name) -> Union{Nothing, Vector{TimeInterval}}

The intervals that the time axis name of ds defines by itself: its CF bounds (decode_time_bounds), or, for a months since axis (MonthsSince), the calendar month of each stamp. nothing for CF stamps without bounds, whose intervals depend on the whole series (stamp_intervals).

source
AtmosTransport.GriddedIO.latlon_orientation Method
julia
latlon_orientation(lon, lat) -> LatLonOrientation

Orientation that puts a file's latitude-longitude field into the runtime's convention: latitude ascending (south to north), and, when any longitude is negative, the longitude axis rolled to start at its first value ≥ 0 and the coordinates taken modulo 360°. A longitude axis without negative values, or with no value ≥ 0, is left as it is.

Examples: centres -179.5 … 179.5 become 0.5 … 359.5 (shift 180); the wrapped CLM-FATES axis 0 … 177.5, -180 … -2.5 keeps its order (shift 0) and becomes 0 … 357.5; latitudes 89.75 … -89.75 (VISIT) are flipped.

source
AtmosTransport.GriddedIO.mass_rate_factor Method
julia
mass_rate_factor(u::FluxUnits, species::Symbol, iv::TimeInterval) -> Float64
mass_rate_factor(u::FluxUnits, species::Symbol) -> Float64

Factor that turns a value in units u into kg of species per second (per m² or per cell, as u.area says): kg_per_unit(u, species) /period_seconds(u, iv). The form without iv is for units whose time unit does not depend on the calendar (not per month). For per-second units it is kg_per_unit exactly.

source
AtmosTransport.GriddedIO.month_keys Method
julia
month_keys(times) -> Vector{MonthKey}

The calendar month of each time stamp.

source
AtmosTransport.GriddedIO.open_cs_native_writer Method
julia
open_cs_native_writer(path, mesh::CubedSphereMesh, intervals::Vector{TimeInterval},
                      variables; species::Symbol, chunk_time::Integer, deflate = 1,
                      attributes = Dict(), config_text = nothing) -> CSNativeFluxWriter
open_cs_native_writer(f, path, mesh, intervals, variables; kwargs...) -> String

Open a cs_native surface-flux file for mesh with one slice per interval. variables lists the flux series, each a name or name => attributes (for example "GPP_CO2_FLUX" => Dict("long_name" => "gross primary production")). The file is written to a staging file next to path and appears at path only when publish! succeeds; a relative path is taken relative to the working directory at opening. The do-block form calls f(writer), then publish!, and returns the absolute path; if f throws, the file is abandoned.

The file (NetCDF-4):

  • Geometry from Output.define_cs_horizontal_geometry!(ds, mesh): dimensions Xdim, Ydim, Xcorner, Ycorner, nf; cell centres lons/lats, corners, cell_area = mesh.cell_areas (Float64, m²) and the cubed_sphere grid mapping with semi_major_axis = mesh.radius.

  • Time: the intervals must be contiguous (stop of one is start of the next). time holds each interval's start (Float64, "hours since <first day> 00:00:00", calendar = "proleptic_gregorian"), time_bnds (nv, time) its start and stop. Slices are start-stamped, so the runtime's stepwise scheme holds each slice for exactly its interval.

  • Series: one Float32 variable per entry of variables, dimensions CS_NATIVE_FLUX_DIMS, units = canonical_units_string(species) ("kg CO2 m-2 s-1" for :co2), sign_convention = "positive into the atmosphere", grid_mapping = "cubed_sphere", coordinates = "lons lats", and no _FillValue. Chunks are (Nc, Nc, 1, chunk_time) (Julia order; chunk_time is clamped to the number of slices), deflate level deflate with shuffle: chunk_time = 1 for daily files and 24 for hourly files (the P0 read benchmarks, design Section 2.4).

  • Global attributes: Conventions = "CF-1.8", species, the provenance block of Output.define_provenance_attributes! (its creation_date is the publication time), config_text and config_sha1 (SHA-1 of the text) when config_text is given, then attributes. A caller attribute may not replace one the writer sets.

species must be a key of SPECIES_MOLAR_MASS.

source
AtmosTransport.GriddedIO.orient Method
julia
orient(A::AbstractArray, o::LatLonOrientation) -> Array

A new array with the two leading dimensions (lon, lat) of A reordered by o (longitude rolled, latitude flipped); trailing dimensions (time, level, ...) are untouched. Values are copied, never computed, so the result is bitwise the input in the new order.

source
AtmosTransport.GriddedIO.orient_coordinates Method
julia
orient_coordinates(lon, lat, o::LatLonOrientation) -> (lon, lat)

The coordinates that match orient(A, o), as Vector{Float64}: latitude reversed when o.flip_lat; longitude rolled by o.lon_shift and, when o.wrap_lon, taken modulo 360° (mod(λ, 360)).

source
AtmosTransport.GriddedIO.parse_flux_units Method
julia
parse_flux_units(s; substance = nothing, molar_default = nothing) -> FluxUnits

Parse the flux units string s. Case, whitespace, ./* separators and LaTeX exponent markup (m$^{-2}$) are ignored. The grammar is

julia
<amount>[<substance>]<factors>[, <qualifier>]
  • amount: kg, g, tonnes, mol, mmol, µmol/umol;

  • substance (optional, right after the amount): C (carbon) or a species name such as CO2, (CO2);

  • factors, in any order: an area (m-2, /m2, cell-1) and a time (s-1, /s, d-1, /day, month-1, /month, per month, yr-1, /year); without an area the flux is per cell. A time is required, except for Tonnes, the EDGAR convention for tonnes per cell and year;

  • qualifier: only per unit land area occupied by the PFT (SDGVM ra).

substance names the substance when the string does not (TRENDY files write kg m-2 s-1 for kg of carbon: substance = :carbon; ECCO-Darwin writes mmol m-2 s-1 of CO₂: substance = :co2). :c/:carbon give Carbon, other names a Species. A substance named both in the string and by the keyword must agree. Without either, mass units are of the TargetSpecies; molar units without a substance take molar_default (the runtime loaders pass the tracer's species, so mmol m-2 s-1 is moles of that species) and throw when it is nothing.

Examples: "kg m-2 s-1", "kg m-2 s-1", "kg.m-2.s-1", "kg m$^{-2}$ s$^{-1}$", "kgC m-2 s-1", "gC m-2 d-1", "g C per month" (per cell), "kgCO2/month/m2", "Tonnes", "mmol m-2 s-1" (with substance), and the runtime's "kg/m2/s", "kg(CO2) m^-2 s^-1", "kg/m²/s". Anything else throws an ArgumentError.

source
AtmosTransport.GriddedIO.period_seconds Method
julia
period_seconds(u::FluxUnits, iv::TimeInterval) -> Float64
period_seconds(u::FluxUnits) -> Float64

Length in seconds of the time unit of u: 1 (per second), 86400 (per day), DAYS_PER_JULIAN_YEAR × 86400 (per year), or the length of the calendar month that contains the interval iv (per month; iv must lie in one month, and the form without iv throws).

source
AtmosTransport.GriddedIO.publish! Method
julia
publish!(w::CSNativeFluxWriter) -> String

Complete the file and publish it at its path by one atomic rename (AtomicFiles.commit_atomic_staging); return the absolute path. Every slice of every series must have been written; otherwise, and on any other error, the staging file is removed (the writer is abandoned) and the error is thrown. The provenance creation_date is set to the publication time.

source
AtmosTransport.GriddedIO.read_cs_native_flux! Function
julia
read_cs_native_flux!(dest_panels, path_or_ds, variable, slices = :; mesh::CubedSphereMesh)
    -> (; slices::UnitRange{Int}, times::Vector{DateTime}, units::FluxUnits)

Read slices slices of the cs_native series variable (of a file path or an open NetCDF dataset) panel by panel into the caller's arrays: dest_panels[p], size (Nc, Nc, length(slices)), receives panel p, each stored value converted once to its element type (FT(x), as FT.(ds[variable][:, :, p, slices]) does). Beyond the checks it allocates one panel-slab read buffer (two when the variable declares fill values or packing), so a caller that fills its final arrays holds the series once, never the whole cube twice. This is the runtime's loader (surface_flux.kind = "cs_native").

Every check of read_cs_native_flux applies (cs_native_flux_info runs first); a value that is not finite once converted (a Float64 value beyond the Float32 range read into Float32) is an error too. slices is any selection cs_native_flux_info takes. Returns the slices read, their times and the parsed units; the caller converts with mass_rate_factor(units, species) and the mesh cell areas.

source
AtmosTransport.GriddedIO.read_cs_native_flux Method
julia
read_cs_native_flux(path, variable; mesh::CubedSphereMesh, slices = :)
    -> (; density, times::Vector{DateTime}, units::FluxUnits)

Read the cs_native flux series variable of the file path for the runtime mesh mesh into one array, the whole-file form of read_cs_native_flux! (the runtime streams panel by panel into its own arrays instead). The surface-flux builder uses it for its readback.

Checks, each an error when it fails:

  • the variable has dimensions CS_NATIVE_FLUX_DIMS with Xdim = Ydim = mesh.Nc, six panels and at least one slice;

  • its time axis (found by find_coordinate) decodes with decode_time and the default decoders, and is strictly increasing;

  • its units parse (parse_flux_units) as kg of a species or of carbon per m² per second (kg CO2 m-2 s-1, kg m-2 s-1, kgC m-2 s-1); gram, molar, per-cell, other time units and missing units are refused;

  • a stored cell_area passes check_cell_area;

  • no value of the slices read is fill (_FillValue or missing_value) or non-finite.

slices selects the slices to read: : (all), a range of slice indices, or a function of the decoded times (Vector{DateTime}) returning such a range, so a caller can select by time without opening the file twice.

Returns density (Julia order (Xdim, Ydim, nf, slice), in the stored floating-point type: Float32 for files of open_cs_native_writer), the times of the slices read, and the parsed units. The caller turns densities into kg of its species per second per cell with mass_rate_factor(units, species) and the mesh cell areas.

source
AtmosTransport.GriddedIO.read_model_surface_height Method
julia
read_model_surface_height(path, mesh) -> Output.ModelSurfaceHeight

Read the per-grid model surface height dataset written by write_model_surface_height for mesh (LatLonMesh, ReducedGaussianMesh or CubedSphereMesh). The variable surface_height must be in metres; its stored cell centres must be the mesh's (unit-sphere chord within 1e-7, about 6e-6°, of the Float64 centres on the cubed sphere; within 8 eps of a Float32 lat-lon or reduced-Gaussian mesh), a stored cubed-sphere cell_area must pass check_cell_area, and every value must be finite. Returns the field in the mesh's native layout (a matrix, a vector of cells, or six panels) as Output.ModelSurfaceHeight, which the observation sampler queries per cell.

source
AtmosTransport.GriddedIO.read_point_elevation Method
julia
read_point_elevation(path; variable = "z") -> LatLonPointElevation

Read a lat-lon DEM for point lookups. The file holds 1-D longitude and latitude centre coordinates (found by name, find_coordinate) and a 2-D variable variable on those two coordinate dimensions, with the ground elevation in metres above sea level; units must say metres. The axes are identified by the variable's dimension names: (lon, lat) in Julia order (CF/NetCDF order (lat, lon), the usual layout) or the transpose, which is read transposed back; any other dimensions are an error. Fill values (_FillValue, missing_value) become NaN. Latitudes may be stored in either order (they are flipped to ascending with orient); longitudes must be strictly ascending in any convention (−180…180, 0…360, or a regional window). A gridline-registered global DEM, whose first and last longitudes are the same meridian 360° apart, has its last column dropped. The DEM must give the land or water surface: over the ocean the ground is the sea surface (0 m), so a bathymetric product (ETOPO) must be clipped at 0 over the sea first; a value below −450 m (deeper than the Dead Sea shore) is rejected as bathymetry or an undeclared fill value, and a value above 9000 m (higher than Mount Everest) as an undeclared fill value.

The whole variable is read into memory as Float32: about 0.9 GB for a global 1-arc-minute grid (21600 × 10800).

source
AtmosTransport.GriddedIO.read_raw Method
julia
read_raw([T = Float64,] ds, name, policy::AbstractFillPolicy, idx...) -> Array{T}

Values of variable name of the NetCDF dataset ds at indices idx (all values when idx is empty), as element type T (Float64 by default), with fill values replaced by the rule of policy (MaskedAsZero, MaskedAsNaN, FillValueOnly). Indices follow the Julia (reversed CF) dimension order, as ds[name][idx...]. All-integer indices give a 0-dimensional array.

Each stored value is converted once, straight to T, so a Float32 run reads into Float32 without a Float64 copy, and MaskedAsZero reproduces T.(nomissing(ds[name][idx...], zero(T))) bit for bit.

source
AtmosTransport.GriddedIO.seconds_since Method
julia
seconds_since(t, reference::DateTime) -> Float64
seconds_since(times, reference::DateTime) -> Vector{Float64}

Seconds from reference to each time stamp, Dates.value(t - reference) / 1000 (millisecond resolution, the resolution of DateTime); negative before reference. This is the runtime's slice-time convention.

source
AtmosTransport.GriddedIO.source_latlon_mesh Function
julia
source_latlon_mesh(lon, lat, rule = SnappedFaces(); radius) -> LatLonMesh{Float64}
source_latlon_mesh(ds, rule = SnappedFaces(); radius) -> LatLonMesh{Float64}

The latitude-longitude mesh of a gridded input file whose cell centres are lon and lat [degrees], with faces derived by the face rule rule (SnappedFaces, described below, or UniformSpanFaces). The second form reads the centres of the dataset's coordinates (find_coordinate). radius [m] has no default: the source mesh must sit on the sphere of the mesh it is regridded to.

Under SnappedFaces, the mesh is built on the explicit-face constructor LatLonMesh(λᶠ, φᶠ; radius); latitude rows of a descending axis are stored ascending (Grids.latitude_row_order gives the data permutation), and longitude columns keep their input order (a wrapped axis is unwrapped, not rolled).

Faces under SnappedFaces

Regular global axes: the snapping rule. For n stored centres x_k, k = 0 … n-1, with res = 180/n (latitude) or 360/n (longitude) and the origin x₀ = -90 (ascending latitude), +90 (descending latitude), -180 (longitude with a negative centre) or 0 (longitude without one), direction s = ±1:

  1. k̂ = round((x_k − x₀)/(s·res) − ½); require k̂ = k and |(x_k − x₀)/(s·res) − ½ − k̂| ≤ 1e-6 cells plus the storage rounding of the centres, 2 eps(T) max|x| / res cells for storage precision T (Float64 values that are all Float32 numbers, for longitudes also after subtracting 360°, count as Float32: GridFED's Float32 0.1° centres sit up to 3e-5 cells off). Otherwise an ArgumentError: a half-cell offset or a wrong resolution.

  2. The faces are x₀ + s·j·res, j = 0 … n, built from integers (the correctly rounded ratio (x₀ n + s j 180)/n), so the outer faces are exactly ±90 and -180/180 (or 0/360).

A longitude axis may start anywhere on that grid (after a roll or a wrap); a global axis whose centres sit on multiples of res (a centre on 0° or ±180°) gets faces at the centres ± res/2, so its first face is not ±180 or 0 (CLASSIC -180.5°, CLM -0.625°, CLM-FATES and ELM-FATES -1.25°, ELM -180.625°, JSBACH -180.9375°). Any other registration throws.

Other latitude axes (pole-centred, truncated, Gaussian, regional): faces at the midpoints of adjacent centres. An outer face is the pole when the outer centre is closer to the pole than the adjacent spacing (pole-centred rows of CLM, ELM and iMAPLE, whose polar cells are half-width caps; the Gaussian JSBACH rows, whose midpoint faces with polar outer faces reproduce the Python audit); otherwise it is the outer centre minus half the adjacent spacing (the FATES grids, truncated south of their row at -59.684°: southern face -60.6316°). Such meshes have non-uniform latitude faces (Δφ = NaN), so dx, dy and other Δφ-based metrics throw.

Regional longitude axes (spanning less than 360°): uniform faces from the first centre minus to the last centre plus half the mean spacing.

Faces are never taken from lat_bnds/lon_bnds.

source
AtmosTransport.GriddedIO.stage_input Method
julia
stage_input(path; staging_dir) -> String

A path at which the input file path can be opened as NetCDF:

  • an uncompressed file (any name not ending in .gz, .tar.gz or .tgz) is returned as it is;

  • *.gz is decompressed with gzip -dc;

  • *.tar.gz / *.tgz must hold exactly one file, extracted with tar -xzOf.

The decompressed file is written to staging_dir (created if needed; there is no default, in particular not /tmp) as <key>-<name>, where key is the SHA-1 of the source's absolute path, size and modification time, so a staged file is reused until its source changes. It is published with AtomicFiles.atomic_publish only after it decompressed completely and starts with a NetCDF signature, so a concurrent or interrupted stager never leaves a partial file. gzip and tar must be on the PATH. Staged files are not removed; the staging directory is the caller's.

source
AtmosTransport.GriddedIO.stamp_intervals Method
julia
stamp_intervals(stamps; source = "the time stamps") -> Vector{TimeInterval}

The interval each of the strictly increasing interval-start stamps (the bare stamps of a whole series, merged over its files) stands for. A series has a regular step, and each slice holds for that step:

  • Month starts (every stamp the first instant of a month): a slice holds until the next stamp when that is one month later, or one year later from a 1 January to the next 1 January (an annual series, also after a monthly part); the last stamp holds for one calendar month, or one year when the last spacing is such a year. A slice before any other spacing (Jan 1 then Mar 1) holds for its own month only, so the months in between are a gap.

  • Other stamps: the step is the smallest spacing; a slice holds until the next stamp when the spacing is that step (within STAMP_SPACING_TOLERANCE), else for one step (a gap follows), and the last stamp holds for one step.

A gap is not an error here: the coverage check of the surface-flux loaders refuses a run only when the run crosses it. These are ArgumentErrors, since the period of a slice cannot be inferred:

  • a single stamp: one stamp gives no spacing, and a 1 January 00Z stamp may stand for a day, a month or a year;

  • stamps about one month (28 to 31 days) or one year (365 to 366 days) apart that are not all month starts, e.g. mid-month stamps (Jan 15, Feb 15, …): whether each slice holds from its stamp or is centred on it is unknown;

  • stamps that do not increase strictly (a stamp in two files).

The remedy is CF bounds on the time coordinate, or interval-start stamps. source names the stamps in these messages.

source
AtmosTransport.GriddedIO.write_latlon_flux Method
julia
write_latlon_flux(path, mesh::LatLonMesh, variables; species::Symbol,
                  attributes = Dict(), deflate = 1, config_text = nothing) -> String

Write a static latitude-longitude surface-flux file for surface_flux.kind = "file" atomically (AtomicFiles.atomic_publish) and return path.

  • Static only: each variable is one field (lon, lat), with no time dimension. The runtime's kind = "file" loader reads one static field per tracer and is not a time-varying kind, so a series would never be read as one; time-varying fluxes on the cubed sphere go to cs_native files (open_cs_native_writer).

  • variables lists the fields as name => data or name => (data, attributes); data is Float32 kg of species per m² per second, positive into the atmosphere, size (Nx, Ny), Julia order (lon, lat). Every value must be finite; the file stores no _FillValue.

  • The mesh must be one the runtime rebuilds exactly. The runtime takes the cell faces from the written centres (oriented, then SnappedFaces), never from lon_bnds/lat_bnds. A mesh whose faces that rule does not reproduce bit for bit is refused with an ArgumentError: its file would be regridded on other cells.

  • Coordinates: lon, lat (Float64 cell centres mesh.λᶜ, mesh.φᶜ) with lon_bnds, lat_bnds (the mesh faces) and cell_area (lon, lat) from cell_area(mesh, i, j) [m²].

  • Each field: units = canonical_units_string(species), sign_convention = "positive into the atmosphere", cell_measures = "area: cell_area"; chunks (Nx, Ny), deflate level deflate with shuffle.

  • Global attributes as for the cs_native writer: Conventions, species, provenance, config_text/config_sha1 when given, then attributes.

source
AtmosTransport.GriddedIO.write_model_surface_height Method
julia
write_model_surface_height(path, mesh, field; attributes = Dict()) -> String

Write the per-grid model surface height dataset atomically (AtomicFiles.atomic_publish) and return path. field holds the surface height in m above sea level, finite everywhere, in the mesh's layout: (Nx, Ny) on a LatLonMesh, (ncells,) on a ReducedGaussianMesh, (Nc, Nc, 6) on a CubedSphereMesh (panels in the mesh's convention).

The file holds the Float64 variable surface_height (units = "m", standard_name = "surface_altitude") and the mesh geometry: lat-lon lon, lat, bounds and cell_area; cubed-sphere Xdim/Ydim/nf, lons/lats, corners, cell_area and the cubed_sphere mapping (Output.define_cs_horizontal_geometry!); reduced Gaussian cell with longitude, latitude and cell_area. Global attributes: Conventions, title, the provenance block, then attributes (for example the source file and the regridding). read_model_surface_height reads it back.

source
AtmosTransport.GriddedIO.write_slices! Method
julia
write_slices!(w::CSNativeFluxWriter, name, k::AbstractUnitRange,
              density::AbstractArray{Float32, 4}) -> w

Write the densities density, size (Nc, Nc, 6, length(k)), into slices k of the series name (kg of the species per m² per second, Julia order (Xdim, Ydim, nf, time)). Every value must be finite (an ArgumentError names the first that is not); the file stores no fill values. Slices may be written in any order and rewritten; publish! requires every slice of every series.

source