sidescanpy.export.ml.dataset#

Annotated training datasets of target-centered (and background) images.

build_target_dataset() turns converted .ss.h5 surveys plus a target catalog into one COCO + YOLO dataset per export style, rendered at one render level and configured by a DatasetSpec:

Targets at or above min_confidence are correlated to their ping (correlate_target()) and grouped by file and frequency.
For each target a bounded ping neighborhood is loaded once and every style is cut from it n_augment times (independent jitter, optional random flip). Every catalog target inside a window becomes an annotation.
Optionally, n_background screened no-target windows per style (sample_background_windows()).
Images are written as single-channel 8- or 16-bit PNGs, with targets.coco.json, a YOLO label set, augmentations.json (when draws vary) and a top-level manifest.json.

Plotting (contact sheets, context views) is optional and supplied by the caller as callables, so this module never imports matplotlib.

Output layout per style directory (layout="flat", the default):

<style>/target_<id>[_a<k>].png, bg_<n>[_a<k>].png
<style>/targets.coco.json
<style>/targets.yolo/{data.yaml,labels/<stem>.txt}
<style>/augmentations.json       (n_augment > 1, random_flip or backgrounds)
<style>/splits/{train,val}.txt   (val_fraction > 0)

layout="ultralytics":

<style>/images/{train,val}/<file>.png
<style>/labels/{train,val}/<stem>.txt
<style>/{train,val}.coco.json
<style>/data.yaml                (Ultralytics dataset file)

With several resolutions each gets its own res_<m>m/ directory.

2026 rshomberg@uri.edu

Contents

`LAYOUTS`	Dataset directory layouts (see the module docstring).
`DatasetSpec`	Everything that shapes a target dataset (defaults match `sidescan-export chip`).
`SkippedTarget`	A target (or one style of it) left out of the dataset, and why.
`StyleResult`	What one style directory received.
`DatasetResult`	Summary of a `build_target_dataset()` run.
`build_target_dataset`	Render a COCO + YOLO dataset of target (and background) images.

sidescanpy.export.ml.dataset.LAYOUTS = ('flat', 'ultralytics')#: Dataset directory layouts (see the module docstring).

class sidescanpy.export.ml.dataset.DatasetSpec(styles=('chip', 'single-swath', 'dual-swath'), render_level='radiometric', size_px=400, swath_size_px=512, resolution_m=None, value_var='radiometric', resample_method='griddata', across_axis='ground', swath_fill=False, box_size=40.0, box_size_m=None, with_context=False, jitter_px=0, n_augment=1, random_flip=False, gamma=1.0, pct_low=1.0, pct_high=99.0, normalization_scope='image', bit_depth=8, min_confidence='verified', categories=None, max_match_dist_m=25.0, seed=None, n_background=0, background_margin_m=0.0, background_max_nan_frac=0.3, background_ping_ranges=None, val_fraction=0.0, split_block_pings=500, layout='flat', frequencies=None)#

Bases: object

Everything that shapes a target dataset (defaults match sidescan-export chip).

Attributes:

stylestuple of str: Subset of CHIP_STYLES.
render_levelstr: One of RENDER_LEVELS.
size_px, swath_size_pxint: Output size of chip / the swath styles.
resolution_mfloat, list of float, or None: Geometric/georef meters per pixel; None chooses per window; a list builds one dataset per resolution under res_<m>m/.
value_varstr: Geometric/georef (and pixel context) values: uncorrected, db or radiometric.
resample_method, across_axis, swath_fill: See extract_geo_style_window().
box_sizefloat: Placeholder annotation box (pixels) for targets with no measured bbox.
box_size_mfloat, optional: Geometric/georef placeholder box in meters instead.
with_contextbool: Write context/context_<id>.png views (needs a context callable).
jitter_px, n_augment, random_flip: Augmentation: center jitter, draws per target, random flips.
gamma, pct_low, pct_high, normalization_scope, bit_depth: Display transform (see to_uint8()).
min_confidencestr: Targets below this are neither rendered nor annotated (backgrounds still avoid them).
categorieslist of str, optional: Fixed class order (see resolve_categories()).
max_match_dist_mfloat: Target-to-ping correlation distance.
seedint, optional
n_background, background_margin_m, background_max_nan_frac, background_ping_ranges: Background windows (see sample_background_windows()).
val_fractionfloat: Share of targets (by id hash) and background ping blocks put in the validation split; 0 disables splitting.
split_block_pingsint: Background ping-block size for the split hash, so overlapping background windows share a split.
layoutstr: One of LAYOUTS.
frequencieslist of str, optional: Frequency keys (e.g. ["hz_900000"]) that target and background windows may come from. Default: every frequency in the files. Use it to keep a dead or unwanted channel out of the dataset; a target is otherwise matched to the frequency whose sample spacing is closest to its catalog’s, which can be ambiguous when frequencies share a sample spacing.

classmethod from_dict(data)#: Build from a mapping (e.g. parsed YAML); unknown keys are an error.

classmethod from_yaml(path)#: Load from a YAML (or JSON) file holding one mapping.

to_dict()#: JSON/YAML-ready mapping that from_dict() accepts.

size_for(style)#: Output size of style.

property is_pixel#: Whether the render level crops native sample/ping pixels.

property split_enabled#: Whether a validation split is assigned.

class sidescanpy.export.ml.dataset.SkippedTarget(target_id, reason, style=None)#

Bases: object

A target (or one style of it) left out of the dataset, and why.

class sidescanpy.export.ml.dataset.StyleResult(style, directory, images=<factory>, n_target_images=0, n_background_images=0, n_annotations=0, split_counts=<factory>)#

Bases: object

What one style directory received.

class sidescanpy.export.ml.dataset.DatasetResult(out_dir, styles=<factory>, skipped=<factory>, n_targets=0, n_matched=0, manifest_path=None)#

Bases: object

Summary of a build_target_dataset() run.

Attributes:

out_dirPath
stylesdict: {relative style directory: StyleResult} ("chip", or "res_0.05m/chip" with several resolutions). Styles that produced no image are absent.
skippedlist of SkippedTarget
n_targetsint: Targets at or above min_confidence.
n_matchedint: Of those, correlated to a ping.
manifest_pathPath or None

property n_images#: Images written across every style.

sidescanpy.export.ml.dataset.build_target_dataset(sources, targets, spec, out_dir, *, contact_sheet=None, context=None)#

Render a COCO + YOLO dataset of target (and background) images.

Parameters:

sourcespath or sequence of paths: Directory of corrected .ss.h5 files, or the files themselves.
targetssequence of Target: The whole catalog, any confidence: only targets at or above spec.min_confidence are rendered and annotated, but background windows avoid all of them.
specDatasetSpec
out_dirpath
contact_sheetcallable, optional: Called once per style as contact_sheet(thumbnails, captions, *, boxes, box_size, suptitle, path) to draw a contact sheet (e.g. with sidescanpy.viz.contact_sheet.contact_sheet()).
contextcallable, optional: Called per target as context(image, target, *, coordinate_system, resolution_m, box_size, path) when spec.with_context.

Returns:

DatasetResult: Nothing is rendered when no target correlates (n_matched == 0).

Raises:

ValueError: A target’s category is missing from spec.categories.