sidescanpy.processing.correct#

One correction API: apply an ordered list of steps, each skippable and replaceable, recording every applied step in processing_history.

correct() – one frequency channel dataset.
correct_survey() – every channel of a survey (supplies the survey’s aux records to the "interpolate_aux" step).
correct_file() – read a .ss.h5 file, correct it, write it atomically (in place by default), optionally in ping chunks to bound memory.
correction_steps() – a set of requested built-in steps in canonical order, with "interpolate_aux" added when a step needs it.

Steps#

A step is a built-in name, a Step (custom name + function) or any callable fn(ds, **params) -> ds. Built-in steps, in canonical order:

"crop_farrange": crop_far_range(). Params: cutoff_sample, n_bins, min_valid_fraction, rise_threshold_db, margin_bins.
"interpolate_aux": interpolate_aux_onto_dataset(); needs the survey’s aux records (aux= / correct_survey()); correct() skips it, with a WARNING, when no aux is given. Params: fields, method, extrapolation, drop_invalid, held_values, max_hold_s, per_ping_fallback, groups, merge_window_s.
"bottom_track": altitude_from_bottom_track(): set altitude_m from the first bottom return where it is missing (NaN or <= 0), before the geometric step needs it. Not in DEFAULT_STEPS. Params: replace, max_diff_m, min_altitude_m, min_return_m, min_return_fraction, smooth_window, threshold_fraction, min_run, smooth_pings.
"layback": apply_layback(): move ship positions astern to the towfish by the recorded layback (plus tow-point offset), before geometric/georef. Only moves positions whose position_source is "ship"; otherwise logs a WARNING and changes nothing. Not in DEFAULT_STEPS. Params: layback_m, tow_point_aft_m, tow_point_starboard_m, position_source, use_cable_out, ellipsoid.
"tvg": apply_tvg(). Params: spreading_loss_db, absorption_db_per_m, min_range_m, sound_velocity_ms, plus temperature_c, salinity_psu, depth_m, ph (Francois-Garrison inputs, used only when absorption_db_per_m is None; model of Francois & Garrison, J. Acoust. Soc. Am. 72(3) 896-907 and 72(6) 1879-1890, 1982).
"geometric": compute_geometric_ranges(). Params: expose_unmasked_z.
"range_correction": apply_empirical_range_correction() (needs tvg + geometric). Params: table (required: a RangeCorrectionTable or the path of a saved one; no built-in default).
"georef": compute_georeferencing() (needs geometric). Params: ellipsoid.

Steps run in the order given; dependencies are the caller’s to respect. DEFAULT_STEPS is interpolate_aux -> tvg -> geometric -> georef.

Conventions#

Skip a step by leaving it out of steps. params may only name steps that are in steps (a typo raises ValueError); a parameter the step does not accept raises TypeError. Both are checked before anything runs.
Idempotent: a step already applied (see sidescanpy.core.history.has_step()) is skipped with a DEBUG log, so re-running is a no-op. Custom steps are matched by name, so give each one a distinct name (lambdas are rejected for that reason).
force=True re-applies already-applied steps. A radiometric step (tvg/range_correction) is first inverted – together with every radiometric step applied after it, all of which must then be in steps – so its new parameters replace, not stack on, the old ones.
History: every applied step appends one record {"step", "params", "function", "version", "timestamp_utc"} with the parameters actually used (defaults resolved). Built-in steps write their own record; custom steps are recorded here with the bound arguments.
No uncorrected copy is kept: sidescanpy.processing.inversion.invert() undoes recorded steps.

2026 rshomberg@uri.edu

Contents

`DEFAULT_STEPS`	Built-in immutable sequence.
`Step`	A caller-supplied correction step with an explicit history name.
`correction_steps`	Built-in step names in canonical order, adding `"interpolate_aux"` when needed.
`correct`	Apply correction `steps` to one frequency channel dataset.
`correct_survey`	`correct()` every frequency channel of `survey`.
`estimate_correct_disk_bytes`	Free disk space `correct_file()` needs to correct `path` with `steps`.
`correct_file`	Correct a `.ss.h5` file and write it atomically.

sidescanpy.processing.correct.DEFAULT_STEPS#

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable’s items.

If the argument is a tuple, the return value is the same object.

class sidescanpy.processing.correct.Step(name, fn)#

Bases: object

A caller-supplied correction step with an explicit history name.

fn(ds, **params) -> xr.Dataset. The name is what processing_history records and what idempotency matches on; it must not be a built-in name.

sidescanpy.processing.correct.correction_steps(requested)#

Built-in step names in canonical order, adding "interpolate_aux" when needed.

"interpolate_aux" is added whenever any of layback / tvg / geometric / range_correction / georef is requested, since each reads per-ping navigation/attitude fields. Duplicates collapse.

Raises:

ValueError: A name that is not a built-in step.

sidescanpy.processing.correct.correct(ds, steps=('interpolate_aux', 'tvg', 'geometric', 'georef'), params=None, *, aux=None, force=False)#

Apply correction steps to one frequency channel dataset.

Parameters:

dsxr.Dataset: Frequency channel dataset (not modified).
stepssequence of str / Step / callable, default DEFAULT_STEPS: Applied in order; see the module docstring for built-ins.
paramsmapping, optional: {step_name: {param: value}} overrides for steps in steps.
auxSideScanSurvey, optional: Supplies navigation/attitude records to "interpolate_aux". When omitted, that step is skipped unless params gives it parameters, which is an error. The skip logs a WARNING (navigation and attitude stay the per-ping values as recorded), or DEBUG when the step was already applied to ds.
forcebool, default False: Re-apply steps that were already applied (see module docstring).

Returns:

xr.Dataset: Corrected dataset with one history record per applied step.

Raises:

ValueError: Unknown step name, params for a step not in steps, params["interpolate_aux"] given without aux, or an impossible force request.
TypeError: A parameter the step does not accept.

sidescanpy.processing.correct.correct_survey(survey, steps=('interpolate_aux', 'tvg', 'geometric', 'georef'), params=None, *, frequency_params=None, force=False, progress=None)#

correct() every frequency channel of survey.

The survey’s own aux records feed "interpolate_aux". params apply to every channel; frequency_params ({freq_key: {step: {param: value}}}) overrides them for one frequency, e.g. a per-frequency absorption_db_per_m or range-correction table. Returns a new (shallow-copied) survey; survey is not modified. progress, if given, is called with "<freq_key>: <step>" before each step.

sidescanpy.processing.correct.estimate_correct_disk_bytes(path, steps=('interpolate_aux', 'tvg', 'geometric', 'georef'), *, chunk_pings=None)#

Free disk space correct_file() needs to correct path with steps.

Output size is estimated as size * (1 + sum(growth[step])) (growth per step in _DISK_GROWTH, measured on real data: tvg + geometric + georef takes a converted file to ~9.2x), times a 1.2 safety factor. The original file stays until the atomic replace, so the full output size is needed. With chunk_pings the uncompressed per-chunk temp files are added (n_cells * bytes per cell). Deliberately conservative: it assumes every requested step will run.

sidescanpy.processing.correct.correct_file(path, steps=('interpolate_aux', 'tvg', 'geometric', 'georef'), params=None, *, frequency_params=None, out_path=None, chunk_pings=None, force=False, progress=None)#

Correct a .ss.h5 file and write it atomically.

Parameters:

pathpath: Input .ss.h5 file.
steps, params, force: As for correct(); applied to every frequency channel, with the file’s aux records feeding "interpolate_aux".
frequency_paramsmapping, optional: {freq_key: {step: {param: value}}} overrides of params for one frequency (see correct_survey()).
out_pathpath, optional: Output file. Default: path (in place, via a temp file + rename, so a failure never corrupts the original).
chunk_pingsint, optional: Process each channel in chunks of this many pings, bounding peak memory to roughly one chunk’s worth of derived arrays. Results are identical to the unchunked path: channel-wide parameters (TVG sound velocity, far-range cutoff) are resolved once per channel first. Custom steps must be ping-independent to be used this way.
progresscallable, optional: Called with a short status string per step / chunk / write.

Returns:

Path: The written file (path unchanged and not rewritten when every requested step was already applied and out_path is not given).

Notes

A legacy backscatter_raw variable is dropped when the file is rewritten. Free space is checked first (estimate_correct_disk_bytes()); OSError if it is insufficient.