Output#

This chapter describes what a run produces: progress output in the terminal, the returned particle state, and the Parquet trajectory files.

Terminal output#

The run monitor prints one line per window and a summary at the end. You choose the backend with monitor= or the runmonitor_backend setting ("print", "log" or "rich"); the counterpart in Fortran is log_level. See Monitoring for the backends and for verbose debug output.

001/010 2007-06-01 441 active, #sub:6, wall:2.2s [io:0ms jax:2.2s py:74ms]
...
010/010 2007-06-01 441 active, #sub:7, wall:459ms [io:60ms jax:361ms py:38ms]
--- total wall:6.4s [io:500ms 8%  jax:5.4s 85%  py:476ms 7%]

The returned state#

Every integrator returns the final ParticleState: positions (x, y, z), cells (ia, ja, ka), times (tt, ts, t0), active and reason, one entry per particle in seeding order. These positions are subset-relative. To save the final state on its own:

tracmass.write_trajectories_parquet("final_positions.parquet", result)

Trajectory files#

Fortran TRACMASS writes text files: _ini (seeding positions), _run (all positions) and _out (exit positions). TRACMASS-JAX writes one Parquet file per run with all three kinds of rows, told apart by the event_type column.

Enabling recording#

# 1. Give a file name
tracmass.time_dependent(state, grid, lazy, recorder="traj.parquet")

# 2. Pass your own recorder, e.g. to add metadata or reuse it
with tracmass.ParticleRecorder("traj.parquet", metadata={"experiment": "A"}) as rec:
    tracmass.time_dependent(state, grid, lazy, recorder=rec)

# 3. Let the settings name it (output_name or auto_record)
tracmass.run(env="omi")

With output_name or auto_record = true in the active environment, the file is

{base_output_dir}[/{output_name or env}]/{stem}_{forward|reverse}_{dtm_start}.parquet

where stem is output_name, or trm_{env} with auto_record. The sub-folder is used when use_sub_folder = true. A recorder the integrator built for you is closed when the integrator returns, even after an error.

Columns#

Column

Type

Description

event_type

string

"snapshot", "rampup" or "deactivated"

window

int32

Window number (0 at the start of the run)

dtm

timestamp

Model time of the row

id

int32

Particle id, its index in the particle state

x, y, z

float64

Position in grid-cell units, absolute (full-grid) x and y

ia, ja, ka

int32

1-based cell, absolute ia and ja

tt

float64

Time since the start of the run, s

t0

float64

Release time, s (tells the cohorts of a repeat-seeded run apart)

reason

int16

Why the particle stopped (deactivated rows); see Stopping trajectories

The row types:

  • snapshot: the active particles at every window boundary, from the seeding positions at window 0 (Fortran _ini) to the final positions (together, Fortran _run).

  • deactivated: one row per particle at the window in which it stopped, with its last position and reason (Fortran _out).

  • rampup: extra positions inside the first windows (see below).

Positions are written in the full-grid frame: the recorder adds grid.i_offset and grid.j_offset, so files from different subsets can be compared directly.

Run metadata#

Every file written through an integrator carries a JSON description of the run under the Parquet schema key tracmass.run:

Key

Description

gcm, gcm_product, gcm_subset

Adapter, dataset (e.g. "glorys12v1") and subset

env

Settings environment

integrator

Integrator function

direction

"forward" or "reverse"

dtm_start, dtm_end, timestep_seconds

Time span and GCM timestep

i_offset, j_offset

Subset start (only on subset grids)

rampup_windows, rampup_substeps, save_dt_seconds

Ramp-up saves

seed_freq, seed_windows

Repeated releases

traj_max_time, timax_seconds

Trajectory lifetime

tracmass_version, created

Provenance

Keys you put in ParticleRecorder(metadata=...) yourself are kept. To read the metadata without loading any rows:

import json
import pyarrow.parquet as pq

meta = json.loads(pq.read_schema("traj.parquet").metadata[b"tracmass.run"])

Ramp-up saves#

To resolve fast early dispersal, time_dependent and time_dependent_patched can save positions rampup_substeps times per window during the first rampup_windows windows:

tracmass.time_dependent(state, grid, lazy, rampup_windows=2, rampup_substeps=5)
# daily GLORYS: positions every 4.8 h for the first two days, then daily
df = tracmass.load(rampup=True)
df.attrs["tracmass.run"]["save_dt_seconds"]
# [17280.0] * 10 + [86400.0] * (n_windows - 2)

The extra rows have event_type="rampup" (with window set to the window they fall in and dtm to their exact time). Window-boundary rows stay "snapshot", so code that reads snapshots sees the same rows as before.

Particles are stopped exactly at each save time and restarted, which also refreshes the time-interpolation weight there. During ramp-up the run therefore behaves like one with a finer iter_sub, so runs with and without ramp-up are not bit-identical. Ramp-up saves are not available with time_dependent_streaming or scheme="analytical".

Reading trajectories — tracmass.load#

tracmass.load reads a trajectory file into a pandas.DataFrame. It finds the file the same way the recorder names it:

df = tracmass.load()                                    # newest file of any run
df = tracmass.load(env="omi_falkor")                    # newest omi_falkor file
df = tracmass.load(env="omi_falkor", dtm="2007-06-01")  # that run's file
df = tracmass.load(dtm="2007-06-01", reverse=True)      # active env, backward run
df = tracmass.load("traj.parquet", columns=["id", "dtm", "x", "y"])
df = tracmass.load(filters={"window": 0})               # rows of window 0 only
df = tracmass.load(filters=[("window", "<", 10), ("reason", "==", 2)])
df = tracmass.load(env="omi_falkor", lonlat=True)       # + lon, lat columns
df = tracmass.load(clean=False)                         # every row and column
df = tracmass.load(rampup=True)                         # + ramp-up rows, event_type

df.attrs["path"]           # the file that was read
df.attrs["tracmass.run"]   # the run metadata

  • By default (clean=True) only the snapshot rows are returned, with the columns id, x, y, dtm. An explicit columns= replaces that list, and clean=False returns everything.

  • filters are applied while reading, so row groups that cannot match are skipped. That makes reading one window of a large file fast.

  • lonlat=True adds lon and lat with the adapter’s xy_to_lonlat. The adapter is gcm=, else the one in the file’s metadata, else [env.gcm] name. The file’s gcm_product picks the right grid, even if the adapter is set to another product now.

  • With env and/or dtm, the files of that environment are searched in base_output_dir, its {env} and {output_name} sub-folders. Without arguments, the newest .parquet in base_output_dir or one of its sub-folders is read.

See Analysing trajectories for examples.

Not yet available#

The Fortran stream-function files (_psixy, _psiyz, _psixr, _psiyr, _psirr), the divergence file and the _rerun file have no counterpart yet. Stream functions can be computed from the trajectory files in the meantime.