sidescanpy.processing.aux_fixes#

Reduce sample-and-hold auxiliary records to the sensor’s real fixes.

Many recorders log navigation and attitude as sample-and-hold: every ping or packet carries the latest value the sensor delivered, so the same value repeats until the sensor updates (a ~1 Hz GPS under ~10 Hz pings repeats each position about ten times). Interpolating linearly between those held entries keeps the stair-steps: consecutive pings stack on one position and then jump. held_value_fixes() keeps only the entries where a value actually changed, timed where the change most likely happened, so that interpolating the result recovers a smooth track. Some recorders also spread one sensor update over consecutive entries (the bulk change, then a small follow-up one entry later); merge_window_s collapses such changes into one update event, and aux_update_events() reports the change and event statistics needed to choose it.

2026 rshomberg@uri.edu

Contents

`POSITION_GROUP`	Latitude and longitude as one group, for `groups=[POSITION_GROUP]`.
`held_value_fixes`	Keep only the real fixes of a sample-and-hold auxiliary record.
`AuxUpdateStats`	How one variable group of a sample-and-hold record updates.
`aux_update_events`	Change and update-event statistics of a sample-and-hold record.

sidescanpy.processing.aux_fixes.POSITION_GROUP = ('latitude', 'longitude')#: Latitude and longitude as one group, for groups=[POSITION_GROUP]. Not the default: some recorders log the two coordinates of one GPS fix in separate entries, and joining them turns each half-update into a fix with a stale other coordinate.

sidescanpy.processing.aux_fixes.held_value_fixes(record, *, groups=None, max_hold_s=2.0, merge_window_s=None)#

Keep only the real fixes of a sample-and-hold auxiliary record.

Parameters:

recordxr.Dataset: Auxiliary record with dimension obs and a time coordinate along it (datetime64). Entries need not be sorted.
groupssequence of sequence of str, optional: Variables that update together. A group’s entry is a fix when any of its variables changed. By default every numeric obs variable is its own group, latitude and longitude included: some recorders log the two coordinates of one fix in separate entries a fraction of a second apart, and a joint group would turn each half-update into a fix carrying a stale other coordinate. Pass groups=[POSITION_GROUP] to join them. Variables not named in groups are single-variable groups.
max_hold_sfloat, default 2.0: A value unchanged for longer than this (from the first to the last entry that carries it) is treated as a real hold, such as a stop or a sensor dropout: an extra anchor entry keeps the value at the time of the last entry before the change, so interpolation stays flat across the hold instead of ramping through it. np.inf disables anchors. With merge_window_s, the hold is measured from the last change of the update event.
merge_window_sfloat or mapping of str to float, optional: Within each group, consecutive changes at most this many seconds apart (chained: each measured from the previous change) collapse into one update event, for recorders that log one sensor update over consecutive entries (the bulk change, then a small follow-up one entry later). The event carries the group’s value after its last change and is timed by the fix-time rule applied to its first change (see Notes). Choose a window longer than the follow-up delay and shorter than the real update interval; aux_update_events() shows both. A float applies to every group; a mapping {variable: seconds} applies only to the groups holding those variables (a group takes the smallest window of its variables) and leaves the others unmerged, so a window sized for a ~3 Hz GPS does not chain the changes of a ~10 Hz roll sensor into one event. None (default) keeps every change as its own fix. Groups are merged independently.

Returns:

xr.Dataset: A record on a new obs axis: the sorted union of every group’s fix and anchor times. A variable is NaN at times that are not one of its own fixes (drop NaN rows per variable before interpolating; the interpolate_aux step does). Variables are float64. The input attrs are kept, plus held_value_fixes_entries_in, held_value_fixes_entries_out, held_value_fixes_dropped (entries in minus out) and held_value_fixes_max_hold_s, plus held_value_fixes_merge_window_s when merge_window_s is set (a float, or a JSON string of the mapping). Non-numeric variables and variables without the obs dimension are left out.

Raises:

KeyError: If record has no time coordinate, or groups names a variable the record does not have.
ValueError: If max_hold_s is negative or NaN, or a merge_window_s value is negative or not finite.

Notes

Per group, after sorting by time (stable, so logging order is kept within equal times) and dropping entries where any of the group’s variables is NaN or the time is NaT:

Duplicate timestamps collapse to the last entry logged at that time, the most recent value the logger had. Earlier entries at the same time are superseded, not averaged (averaging a stale and a new value would invent a position, and does not apply to compass angles). When the entries at one time disagree, the update was seen at that time, so a fix there keeps that time.
A fix is an entry whose value differs from the previous entry (exact inequality; this is right for compass angles too, which the interpolator unwraps). The first entry is always a fix. NaN entries are never fixes.
Fix time. When the previous entry repeated the value before it (the value was being held), the new value arrived somewhere between the two entries, so the fix is placed at their midpoint. When the previous entry was itself a fix, the stream is updating at the logging rate and the entry’s own time is kept. A record in which every entry differs from the previous one is therefore returned with its values and times unchanged. The first fix keeps its own time, as does a fix at a duplicated timestamp whose entries disagree.
Holds longer than max_hold_s add an anchor (see above), including the hold that ends the record.
Merged updates (merge_window_s). The first entry of the record is never merged with what follows: it is the start of the record, not an observed change. An event takes the time its first change would have had as a fix (the midpoint after a hold) and the value after its last change. The first changed entry is when the update started reaching the log; the follow-up entries complete the same update rather than report new motion, so the sensor’s fix epoch is best estimated from the first. Timing at the last change instead would add a delay that varies with how many entries an update happened to spread over, which shows up as speed jitter; timing at the first gives a near-constant latency, which leaves speed unbiased. A stream that changes at every entry with entries closer than merge_window_s chains into long events; a WARNING is logged when an event combines more than five changes.

class sidescanpy.processing.aux_fixes.AuxUpdateStats(group, entries, changes, events, merge_window_s, change_gap_entries, change_gap_s, event_gap_entries, event_gap_s, merged_fraction)#

Bases: object

How one variable group of a sample-and-hold record updates.

Attributes:

grouptuple of str: The group’s variables.
entriesint: Entries with a valid time and every group variable finite.
changesint: Entries whose value differs from the previous one (after duplicate timestamps collapse to their last entry). The record’s first entry is not counted.
eventsint: Update events after merging changes within merge_window_s (equal to changes without merging). The first entry is not counted.
merge_window_sfloat or None: The merge window used for this group.
change_gap_entries, change_gap_sdict of str to float: Gap between consecutive changes, in entries (of the de-duplicated finite sequence) and seconds, at percentiles p5, p25, p50, p75, p95. NaN with fewer than two changes.
event_gap_entries, event_gap_sdict of str to float: Same, between the first changes of consecutive events.
merged_fractionfloat: Fraction of changes that come at most merge_window_s after the previous change (the first change has no previous one and counts as not merged). NaN without a merge window or without changes.

as_dict()#: JSON-friendly copy (group as a list).

sidescanpy.processing.aux_fixes.aux_update_events(record, *, groups=None, merge_window_s=None)#

Change and update-event statistics of a sample-and-hold record.

A diagnostic for choosing groups and merge_window_s of held_value_fixes(), which uses the same change detection and merging. A sensor that delivers each update over two consecutive log entries shows bimodal change gaps (one peak at 1 entry, one at the real update interval) that become unimodal events once merged.

Parameters:

recordxr.Dataset: Auxiliary record with dimension obs and a time coordinate.
groupssequence of sequence of str, optional: As in held_value_fixes(); every other numeric obs variable is its own group.
merge_window_sfloat or mapping of str to float, optional: As in held_value_fixes(). None reports changes as events.

Returns:

dict of str to AuxUpdateStats: One entry per group, keyed by its variable names joined with "+" ("latitude+longitude" for POSITION_GROUP).

Raises:

KeyError: If record has no time coordinate, or groups names a variable the record does not have.
ValueError: If a merge_window_s value is negative or not finite.