sidescanpy.io.xtf.converter#

Convert XTF sonar pings into a SideScanSurvey.

Takes the per-file output of sidescanpy.io.xtf.read_xtf() (one XtfPingData per file) and builds a validated survey shaped like the JSF and SDF ones:

One frequency channel per port/starboard pair sharing a ChanInfo frequency, keyed hz_<Hz>. ChanInfo.Frequency is in kHz (the per-channel header’s Frequency is a 16-bit kHz value too); a value of 10 000 or more cannot be a sidescan frequency in kHz and is taken as Hz (logged at DEBUG).
backscatter_intensity is (ping, channel, sample) float32 with channel 0 = port, 1 = starboard. Samples beyond a ping’s NumSamples and a side absent from a ping are NaN; valid_samples (ping, channel) holds the recorded count (0 where absent), as in the SDF converter. Sample 0 is nearest the sonar on both sides: XTF stores port samples far -> near (so [port | stbd] reads as one across-track line), and the converter reverses each port ping’s valid samples. Verified on real EdgeTech 4200 files: the first bottom return of the reversed port data tracks SensorPrimaryAltitude the way the starboard data does as stored.
Samples are multiplied by 2**-Weight from each channel header. EdgeTech Discover writes the raw block-floating-point samples and puts the JSF weighting factor N in Weight, which can change from ping to ping; scaling makes the samples equal to the JSF ones for the same ping (checked on a public Discover 7.22 recording with both files). Writers that leave Weight at 0 are unaffected. The range seen is recorded in xtf_weight_min / xtf_weight_max. Some older Klein-written files hold garbage (e.g. -8408); blocks with |Weight| above 30 are left unscaled, with a WARNING, and counted in xtf_weight_ignored.
Each frequency has its own ping axis. A ping holding neither channel of a frequency (some files’ early packets carry only the low-frequency pair, see sidescanpy.io.xtf.reader) is left out of that frequency, logged at DEBUG and counted in the xtf_pings_absent attribute.
Per-ping fields come from the ping header: SensorYcoordinate / SensorXcoordinate → latitude / longitude (degrees with NavUnits 3; with NavUnits 0 they are northing / easting in metres and are converted, see below), SensorHeading, SensorPitch (positive nose up), SensorRoll (positive roll to starboard, i.e. port up, the JSF convention), Heave, SensorDepth → heading_deg, pitch_deg, roll_deg, heave_m, depth_m. SensorPrimaryAltitude → altitude_m and fathometer_reading_m, with 0 (no bottom lock) as NaN, the way the JSF converter turns an invalid altitude into NaN.
Times from Year … Second + HSeconds (10 ms resolution, the recorder’s time base, normally UTC).

Sound velocity and sample rate#

Some recorders write half the sound velocity in SoundVelocity (the one-way factor c / 2). A value below 1000 m/s is not a plausible sound speed in water, so it is doubled (logged at DEBUG); a plausible value is kept. Where the channel header gives both SlantRange and TimeDuration the choice is cross-checked against SlantRange / TimeDuration (which is c / 2) and a disagreement over 5 % logged at WARNING. Zero or negative values become NaN, so downstream code falls back to 1500 m/s as for SDF.

sample_rate_hz is NumSamples / TimeDuration (falling back to NumSamples * c / (2 * SlantRange) when TimeDuration is 0), so the slant range of the last sample, n * c / (2 * fs), equals TimeDuration * c / 2. The survey model has one sample rate per frequency. Files whose range setting changes mid-file (the sample count staying fixed) are meant to be split first with sidescanpy.io.xtf.split_by_sample_rate() and each run converted on its own. When unsplit pings still mix rates, the most common rate is used and the pings at other rates are reported at WARNING and counted in xtf_pings_other_sample_rate; their ranges are then wrong by the ratio of the rates. For split runs the tolerance is twice the split’s rtol, so the WARNING does not fire for them.

Projected navigation#

With NavUnits 0 the file header says SensorXcoordinate / SensorYcoordinate (and ShipXcoordinate / ShipYcoordinate, not used) are easting / northing in metres, but not in which projection. The caller names it with nav_crs (an EPSG code, a string pyproj accepts, or a pyproj.CRS); the sensor positions are converted to WGS84 longitude / latitude with pyproj, a position of exactly (0, 0) becomes NaN (no fix), and the CRS is recorded in xtf_nav_crs. Without nav_crs such a file raises ValueError: the zone is not guessed. nav_crs is ignored for files with NavUnits 3.

Layback and position source#

Layback → layback_m and CableOut (+ CableOutHundredths) → cable_out_m, per ping, as written (some writers leave Layback NaN or 0). position_source says whether latitude / longitude already are the towfish: towfish when SensorX/Ycoordinate sit more than SHIP_SENSOR_TOL_M from ShipX/Ycoordinate on at least SHIP_SENSOR_FRACTION of the pings (the writer offset them), otherwise unknown. The reader never infers ship: neither the format nor the writers document it, so the call is left to a user who knows the system as deployed (pass position_source="ship" to the layback step). Ship equal to Sensor does not mean the ship: EdgeTech Discover writes the laid-back towfish position in both fields. Files that look like ship positions (written by a SHIP_POSITION_PROGRAMS program, no Ship coordinates, Layback > 0 on at least SHIP_SENSOR_FRACTION of the pings: Triton Isis records the layback without applying it) say so in position_source_reason but stay unknown. The layback correction step moves only ship positions.

One exception is detected and corrected. Some writers (seen in GeoDAS files) offset Sensor from Ship by their tow-point offset and layback rotated by the recorded heading. With no heading recorded (SensorHeading 0 throughout) that offset points the same way, south at heading 0, on every line whatever the course, so Sensor is not the towfish. When SensorHeading is 0 on every ping, Sensor sits a fixed east/north offset from Ship, and the line runs well away from north, the Ship positions become latitude / longitude with position_source ship, and the offset rotated back by heading 0 (less any recorded Layback) is recorded as tow_point_aft_m / tow_point_starboard_m for the layback step. That step rotates by heading_deg, so set a heading first (the course over ground, course_over_ground()).

Not converted: pulse_length_us is not in the XTF ping or channel headers and is 0.0 (as for SDF); gain codes are not interpreted.

Provenance attributes on every frequency dataset (xtf_ prefix): recording program and version, sonar name and type, channel numbers and names, whether sound velocity was doubled, the Weight range, and counts of reader anomalies summed over the converted files (packets with fewer channel blocks than declared, clipped channel blocks, channel blocks without ChanInfo, unknown and non-sonar packets, resyncs, truncated files). A single split run (XtfPingData.segment set) adds xtf_segment_index and xtf_segment_count (1-based, kept runs of the file), xtf_segment_ping_start / xtf_segment_ping_stop (its half-open range in the file’s sonar pings), xtf_file_pings, and xtf_runs_dropped_short / xtf_pings_dropped_short_runs (runs of the file shorter than the split’s min_pings, and their pings).

2026 rshomberg@uri.edu

Contents

`HALVED_SOUND_VELOCITY_BELOW_MS`	`SoundVelocity` values below this (m/s) are taken as halved and doubled.
`SAMPLE_RATE_RTOL`	Relative tolerance for treating two pings' sample rates as equal.
`SHIP_SENSOR_TOL_M`	Sensor further than this (m) from Ship counts as a separate (towfish) position.
`SHIP_SENSOR_FRACTION`	Fraction of pings with a Ship position that must be apart for "towfish".
`SHIP_POSITION_PROGRAMS`	Recording programs (lower-case name prefixes) whose Sensor coordinates, with no Ship coordinates and a recorded layback, look like the ship's position.
`ZERO_HEADING_OFFSET_SD_M`	Largest standard deviation (m) of a Sensor - Ship offset taken as fixed.
`ZERO_HEADING_MIN_TRACK_M`	Shortest Ship track (m) over which a fixed offset is judged.
`ZERO_HEADING_MIN_COURSE_DEG`	A fixed offset is only suspect on courses further than this from north.
`convert`	Convert XTF sonar pings into a validated SideScanSurvey.

sidescanpy.io.xtf.converter.HALVED_SOUND_VELOCITY_BELOW_MS = 1000.0#: SoundVelocity values below this (m/s) are taken as halved and doubled.

sidescanpy.io.xtf.converter.SAMPLE_RATE_RTOL = 0.001#: Relative tolerance for treating two pings’ sample rates as equal.

sidescanpy.io.xtf.converter.SHIP_SENSOR_TOL_M = 0.5#: Sensor further than this (m) from Ship counts as a separate (towfish) position.

sidescanpy.io.xtf.converter.SHIP_SENSOR_FRACTION = 0.95#: Fraction of pings with a Ship position that must be apart for “towfish”.

sidescanpy.io.xtf.converter.SHIP_POSITION_PROGRAMS = ('isis',)#: Recording programs (lower-case name prefixes) whose Sensor coordinates, with no Ship coordinates and a recorded layback, look like the ship’s position. Reported in position_source_reason only, never assumed.

sidescanpy.io.xtf.converter.ZERO_HEADING_OFFSET_SD_M = 0.1#: Largest standard deviation (m) of a Sensor - Ship offset taken as fixed.

sidescanpy.io.xtf.converter.ZERO_HEADING_MIN_TRACK_M = 20.0#: Shortest Ship track (m) over which a fixed offset is judged.

sidescanpy.io.xtf.converter.ZERO_HEADING_MIN_COURSE_DEG = 20.0#: A fixed offset is only suspect on courses further than this from north.

sidescanpy.io.xtf.converter.convert(pings, *, source_files=None, survey_name=None, nav_crs=None)#

Convert XTF sonar pings into a validated SideScanSurvey.

Parameters:

pingsXtfPingData or iterable of XtfPingData: Output of sidescanpy.io.xtf.read_xtf(), one per file. Several files are merged into one survey: pings are concatenated in the given order and stably sorted by time if that order is not chronological; frequencies are matched by value and shorter pings NaN-padded to the longest.
source_fileslist[Path], optional: Source paths stored on the survey. Defaults to the paths in pings.
survey_namestr, optional: Survey name. Defaults to the first source file’s stem.
nav_crsint, str or pyproj.CRS, optional: Projected CRS of the coordinates in files with NavUnits 0 (easting / northing in metres), e.g. 32619 or "EPSG:32619" for UTM zone 19N. Required for such files; ignored for files with NavUnits 3. Positions are converted to WGS84 longitude / latitude (see the module docstring).

Returns:

SideScanSurvey: One frequency channel per port/starboard frequency, navigation and attitude records from the ping headers, an empty DVL record.

Raises:

ValueError: If there are no sonar pings, no port/starboard channels, more than one channel pair of unknown frequency, a frequency whose sample rate cannot be derived, a file with NavUnits 0 and no nav_crs, or a nav_crs that is not projected.
NotImplementedError: If a file’s NavUnits is neither 0 nor 3.