# Output

This chapter describes what a run produces: progress output in the
terminal, the returned particle state, and the Parquet trajectory files.

## Terminal output

The run monitor prints one line per window and a summary at the end. You
choose the backend with `monitor=` or the `runmonitor_backend` setting
(`"print"`, `"log"` or `"rich"`); the counterpart in Fortran is
`log_level`. See [Monitoring](tools/monitoring.md) for the backends and
for `verbose` debug output.

```text
001/010 2007-06-01 441 active, #sub:6, wall:2.2s [io:0ms jax:2.2s py:74ms]
...
010/010 2007-06-01 441 active, #sub:7, wall:459ms [io:60ms jax:361ms py:38ms]
--- total wall:6.4s [io:500ms 8%  jax:5.4s 85%  py:476ms 7%]
```

## The returned state

Every integrator returns the final `ParticleState`: positions (`x`, `y`,
`z`), cells (`ia`, `ja`, `ka`), times (`tt`, `ts`, `t0`), `active` and
`reason`, one entry per particle in seeding order. These positions are
subset-relative. To save the final state on its own:

```python
tracmass.write_trajectories_parquet("final_positions.parquet", result)
```

## Trajectory files

Fortran TRACMASS writes text files: `_ini` (seeding positions), `_run`
(all positions) and `_out` (exit positions). TRACMASS-JAX writes **one
Parquet file** per run with all three kinds of rows, told apart by the
`event_type` column.

### Enabling recording

```python
# 1. Give a file name
tracmass.time_dependent(state, grid, lazy, recorder="traj.parquet")

# 2. Pass your own recorder, e.g. to add metadata or reuse it
with tracmass.ParticleRecorder("traj.parquet", metadata={"experiment": "A"}) as rec:
    tracmass.time_dependent(state, grid, lazy, recorder=rec)

# 3. Let the settings name it (output_name or auto_record)
tracmass.run(env="omi")
```

With `output_name` or `auto_record = true` in the active environment, the
file is

```text
{base_output_dir}[/{output_name or env}]/{stem}_{forward|reverse}_{dtm_start}.parquet
```

where `stem` is `output_name`, or `trm_{env}` with `auto_record`. The
sub-folder is used when `use_sub_folder = true`. A recorder the integrator
built for you is closed when the integrator returns, even after an error.

### Columns

| Column | Type | Description |
|---|---|---|
| `event_type` | string | `"snapshot"`, `"rampup"` or `"deactivated"` |
| `window` | int32 | Window number (0 at the start of the run) |
| `dtm` | timestamp | Model time of the row |
| `id` | int32 | Particle id, its index in the particle state |
| `x`, `y`, `z` | float64 | Position in grid-cell units, absolute (full-grid) `x` and `y` |
| `ia`, `ja`, `ka` | int32 | 1-based cell, absolute `ia` and `ja` |
| `tt` | float64 | Time since the start of the run, s |
| `t0` | float64 | Release time, s (tells the cohorts of a repeat-seeded run apart) |
| `reason` | int16 | Why the particle stopped (`deactivated` rows); see [Stopping trajectories](tutorials/stopping.md) |

The row types:

- **`snapshot`**: the active particles at every window boundary, from the
  seeding positions at window 0 (Fortran `_ini`) to the final positions
  (together, Fortran `_run`).
- **`deactivated`**: one row per particle at the window in which it stopped,
  with its last position and `reason` (Fortran `_out`).
- **`rampup`**: extra positions inside the first windows (see below).

Positions are written in the **full-grid** frame: the recorder adds
`grid.i_offset` and `grid.j_offset`, so files from different subsets can be
compared directly.

### Run metadata

Every file written through an integrator carries a JSON description of the
run under the Parquet schema key `tracmass.run`:

| Key | Description |
|---|---|
| `gcm`, `gcm_product`, `gcm_subset` | Adapter, dataset (e.g. `"glorys12v1"`) and subset |
| `env` | Settings environment |
| `integrator` | Integrator function |
| `direction` | `"forward"` or `"reverse"` |
| `dtm_start`, `dtm_end`, `timestep_seconds` | Time span and GCM timestep |
| `i_offset`, `j_offset` | Subset start (only on subset grids) |
| `rampup_windows`, `rampup_substeps`, `save_dt_seconds` | Ramp-up saves |
| `seed_freq`, `seed_windows` | Repeated releases |
| `traj_max_time`, `timax_seconds` | Trajectory lifetime |
| `tracmass_version`, `created` | Provenance |

Keys you put in `ParticleRecorder(metadata=...)` yourself are kept. To read
the metadata without loading any rows:

```python
import json
import pyarrow.parquet as pq

meta = json.loads(pq.read_schema("traj.parquet").metadata[b"tracmass.run"])
```

### Ramp-up saves

To resolve fast early dispersal, `time_dependent` and
`time_dependent_patched` can save positions `rampup_substeps` times per
window during the first `rampup_windows` windows:

```python
tracmass.time_dependent(state, grid, lazy, rampup_windows=2, rampup_substeps=5)
# daily GLORYS: positions every 4.8 h for the first two days, then daily
df = tracmass.load(rampup=True)
df.attrs["tracmass.run"]["save_dt_seconds"]
# [17280.0] * 10 + [86400.0] * (n_windows - 2)
```

The extra rows have `event_type="rampup"` (with `window` set to the window
they fall in and `dtm` to their exact time). Window-boundary rows stay
`"snapshot"`, so code that reads snapshots sees the same rows as before.

Particles are stopped exactly at each save time and restarted, which also
refreshes the time-interpolation weight there. During ramp-up the run
therefore behaves like one with a finer `iter_sub`, so runs with and
without ramp-up are not bit-identical. Ramp-up saves are not available
with `time_dependent_streaming` or `scheme="analytical"`.

## Reading trajectories — `tracmass.load`

`tracmass.load` reads a trajectory file into a `pandas.DataFrame`. It finds
the file the same way the recorder names it:

```python
df = tracmass.load()                                    # newest file of any run
df = tracmass.load(env="omi_falkor")                    # newest omi_falkor file
df = tracmass.load(env="omi_falkor", dtm="2007-06-01")  # that run's file
df = tracmass.load(dtm="2007-06-01", reverse=True)      # active env, backward run
df = tracmass.load("traj.parquet", columns=["id", "dtm", "x", "y"])
df = tracmass.load(filters={"window": 0})               # rows of window 0 only
df = tracmass.load(filters=[("window", "<", 10), ("reason", "==", 2)])
df = tracmass.load(env="omi_falkor", lonlat=True)       # + lon, lat columns
df = tracmass.load(clean=False)                         # every row and column
df = tracmass.load(rampup=True)                         # + ramp-up rows, event_type

df.attrs["path"]           # the file that was read
df.attrs["tracmass.run"]   # the run metadata
```

- By default (`clean=True`) only the `snapshot` rows are returned, with the
  columns `id`, `x`, `y`, `dtm`. An explicit `columns=` replaces that list,
  and `clean=False` returns everything.
- `filters` are applied while reading, so row groups that cannot match are
  skipped. That makes reading one window of a large file fast.
- `lonlat=True` adds `lon` and `lat` with the adapter's `xy_to_lonlat`. The
  adapter is `gcm=`, else the one in the file's metadata, else
  `[env.gcm] name`. The file's `gcm_product` picks the right grid, even if
  the adapter is set to another product now.
- With `env` and/or `dtm`, the files of that environment are searched in
  `base_output_dir`, its `{env}` and `{output_name}` sub-folders. Without
  arguments, the newest `.parquet` in `base_output_dir` or one of its
  sub-folders is read.

See [Analysing trajectories](tutorials/analysis.md) for examples.

## Not yet available

The Fortran stream-function files (`_psixy`, `_psiyz`, `_psixr`, `_psiyr`,
`_psirr`), the divergence file and the `_rerun` file have no counterpart
yet. Stream functions can be computed from the trajectory files in the
meantime.