TSResourceSource ends its stream at EOS instead of crashing the pipeline
mid-graph when the resource is unrecoverable -- the reconnect budget is
exhausted, or a resource error occurs with reconnect=False -- so
downstream elements drain what is in flight instead of dropping it. The
failure is still logged and recorded as the element's eos_error, and
Pipeline.run() raises sgn.PipelineEOSError after the drain so the run
still exits abnormally. Configuration errors (missing hooks, assertion
failures) keep crashing immediately
Require sgn >= 0.12.2, for eos_error and PipelineEOSError
TSResourceSource worker deliveries are stop-aware: puts on a full
output queue poll for a stop request instead of blocking forever, so a
worker mid-delivery during teardown no longer wedges and keeps the
process alive. Stream-position bookkeeping is per delivered buffer, so
an interrupted delivery never records data it did not emit
FileRetentionMixin.clean_up_directory accepts a glob pattern, so a
sink sharing an output directory with other writers can sweep only its
own files instead of everything with a matching extension. A value
with no glob characters is still taken as a plain suffix, as before.
A new recursive keyword sweeps subdirectories
FileRetentionMixin.clean_up_directory now adopts matching files into
the retention cache (in modification-time order) instead of only
deleting those already past retention_time. Files a previous run
left behind therefore age out of the directory as the new run
proceeds rather than surviving indefinitely, and max_files counts
them toward its limit, so count-based retention also works across
restarts and no longer requires retention_time to be set
FileRetentionMixin logging is quiet at INFO in steady state: the
startup scan always logs one INFO line with adopted/deleted counts,
only the first reap of a run is logged at INFO with later cleanup at
DEBUG, and a new files_reaped counter lets sinks report reap counts
in their own summaries
TSResourceSource logs an INFO line when a stream resumes after
reconnect attempts, pairing with the existing interruption warning
so recovery is visible without DEBUG
Pluggable transition_profile on AdaptiveCorrelate: CosSquaredTransition
(default), DiscreteTransition, ReverseDiscreteTransition,
LinearTransition, and PlanckTaperTransition, importable from
sgnts.transforms.correlate
AdaptiveCorrelate queues multiple filter updates, applying each from its own
validity offset quantized to a sample boundary, with last-write-wins for
updates sharing an offset
taps_reversed on Correlate: correlate with pre-reversed taps (i.e.
convolution) while latency=0 still means causal
method on Correlate: force scipy's "direct" or "fft" convolution
backend instead of its "auto" heuristic, which can mis-select for
overlap-save shapes at small strides
use_simd_resample on Resampler (default False): dispatch numpy-backend
resampling in both directions to the sgnl_cpu_interp SIMD extension.
Outputs agree with the numpy path to ~1e-14 but are not bit-identical;
unsupported dtypes and short buffers fall back
inclusive on Threshold (default True): select |data| >= threshold or,
when False, the strict >
'square'/'toggle' signal type in FakeSeriesSource: a 50%-duty wave
alternating between 0 and const at fsin
compact_threshold on Audioadapter (default 16, None disables): merge
contiguous same-gap-state buffers into chunks of at most 4 MiB, keeping
read paths cheap on long-overlap/short-stride streams
Register TSIterSource, TSAppSink, ANDTransform, Multiply,
NaryTransform, and Real as sgn_elements entry points
TSSlice has a total order (lexicographic by (start, stop), as in
igwn_segments), with null slices sorting first
gpstime is a hard dependency; gpsnow is imported from it directly
SegmentSource derives its declared prototype from its values (namespace,
device, and dtype promoted across segments) and materializes scalar values at
that spec, instead of declaring float64 and emitting whatever set_data
produced
Resampler with use_gstlal_cpu_upsample=True raises ImportError at
configure time when sgnl_cpu_interp is missing instead of silently falling
back to the numpy path
AdaptiveCorrelate warns (RuntimeWarning) on more than one filter update
per stride under a continuous transition profile instead of raising
Resampler returns an empty array for buffers shorter than the kernel in
both directions, instead of the spurious samples (or ValueError for
multi-channel data) scipy's "valid" correlate produced by swapping inputs
TSSlices.simplify() no longer returns overlapping slices when one slice
contains another
Correlate.corr() applies every row of a >= 3-dimensional filter bank
(previously only shape[0] of the flattened rows) and no longer reshapes
self.filters in place
Correlate.corr() maps filter row j to data channel j for
multi-channel data; a 1-D stream is still correlated against every row
Correlate.corr() raises when the filters in hand have a different length
than the configured shape, rather than silently correlating with a
truncated kernel
Add a resource hook API to TSResourceSource (resource_open,
resource_read, optional resource_setup and resource_gap_lag)
providing a resilient worker loop: reconnect with capped exponential
backoff, gap-bridged discontinuities with re-delivered spans trimmed,
and outage gap synthesis on live streams. Overriding worker_process
directly is unchanged
filter_dtype on AdaptiveCorrelate: declare the filters' dtype when
it differs from the incoming data's, so the output dtype is declared
correctly before the first filter update arrives. A declaration that
conflicts with actual filters, or a filter update that would change
the output dtype mid-stream, raises ValueError
Correlate declares its output dtype from its filters, so downstream
elements see the correct dtype when the filters change it (e.g.
complex filters over real input), including during gap startup
Fix scrambled multi-channel output from the vectorized numpy
upsampling path (regression in 0.12.0): channels were interleaved
into the polyphase axis. Single-channel output was unaffected
heartbeat_interval on TSResourceSource and FakeSeriesSource
(real_time=True): the longest time a starved live source may stall the
pipeline's graph loop before yielding a zero-length heartbeat frame.
Defaults to a quarter of the stride; float("inf") restores strict
blocking. The default resolution is exposed as
sgnts.base.resolve_heartbeat_interval().
TSFrame.latency(): seconds between the current GPS time and the frame's
start offset. NullSeriesSink verbose output now reports this value
(previously measured from the frame offset plus one max-rate stride).
use_strided_downsample on Resampler: force the strided downsampling
path on or off. The default (None) selects automatically per buffer,
using strided correlation when the decimation factor is large enough that
it beats scipy's FFT path (results are bit-comparable).
up_half_length on Resampler: half length of the upsampling
anti-imaging kernel (default 8, the previous fixed value). Increase it to
deepen the stopband when upsampling data with strong narrow-band content.
pad_zeros on Correlate: zero-pad the filter history on startup so the
pipeline starts producing output immediately instead of stalling until a
full filter length of data has arrived (default False, the previous
behavior).
TSFrame.frame_from_masked(): build a frame from a 1-D (data, mask) pair
in any supported backend, with True mask values marking gap samples.
Backend-agnostic successor to frame_from_ma(), which is now a thin
numpy-only wrapper over it.
mask_runs() in sgnts.base.signal (exported from sgnts.base): yields
(start, stop, value) runs of a 1-D boolean mask, a backend-agnostic
replacement for numpy.ma.clump_masked/clump_unmasked.
Live sources now emit bounded heartbeats by default while waiting on the
wall clock (at most ~4 frames per stride, one carrying data when flowing at
real time), so pipeline elements that fall behind can catch up at roughly
stride/heartbeat_interval times real time. Previously a caught-up live
source pinned the whole pipeline at exactly 1x real time.
TSResourceSource.latest_offset is the minimum delivered end offset across
pads (was the maximum), and the heartbeat decision in new() is per-pad,
fixing a crash when workers deliver different block sizes per pad.
The numpy upsample path is vectorized: all polyphase sub-kernels are
applied in one batched sliding-window contraction instead of one
scipy.correlate call per phase, which dominated the cost for large
upsampling factors.
TSFrame.gap_mask() returns a boolean array in the frame's own backend
and device (previously always numpy), True where samples came from gap
buffers; together with filleddata() it forms the backend-agnostic
(data, mask) pair replacing maskeddata(), which remains numpy-only.
Threshold and NullSeriesSink accept any array backend (previously
numpy-only).
Require sgn >= 0.11: the pad-spec wrappers rely on sgn's positional
pad-callback convention introduced in sgn 0.11.0 (the previous >= 0.8
pin fails at runtime with a TypeError on every pad callback).
TSFrame.align() preserves the frame's EOS flag and metadata instead
of dropping them on the realigned frame.
A source whose requested end falls mid-stride delivers the final partial
stride (sliced to the requested end, carrying EOS) instead of dropping it
as an EOS heartbeat.
A resource worker that goes silent mid-stream raises ValueError after
in_queue_timeout seconds of no deliveries instead of heartbeating
forever: the dead-resource silence clock now spans deadline-bounded reads.
TSFrame.set_buffers recomputes the frame's span (offset/noffset) from
the new buffers, so a final frame sliced to a source's mid-stride end no
longer reports its pre-slice span (and no longer advances the next frame
past the requested end).
Matmul declares its promoted output dtype (result_type with the matrix),
so its all-gap output frames match the dtype its data frames actually carry
(e.g. a float64 matrix applied to a float32 stream).
Align declares each output pad's spec from its matched input pad, so a
mixed-dtype pass-through works under the declare-or-inherit output spec rule
instead of raising.
Stamp each sink pad's all-gap prepared and heartbeat frames with that pad's
own spec instead of the last-pulled pad's, so mixed-dtype inputs keep their
correct specs through gap stretches regardless of pull order
TSFrame.align() preserves the spec hint, so buffer-aligned all-gap frames
no longer decay to the numpy default
Adder sums into a fresh array of the promoted dtype instead of
accumulating in place into its first input buffer, which mutated input data
shared through fan-out links and made the output dtype order-dependent
The default output spec inherits the input spec only when all inputs share
one backend identity; mixed input dtypes raise BackendError unless the
element declares output_prototype()
TSFrame takes an optional spec_hint constructor field to stamp an
all-gap frame's backend identity at construction
TSResourceSource records each pad's output prototype from the first buffer
the worker delivers, so a pad whose stream begins with a gap (e.g. a channel
starting after the frame origin) finalizes its spec with the real backend and
dtype instead of the numpy float64 default — non-default-dtype channels no
longer trip the backend spec check when their data arrives.
A TSResourceSource worker can declare a pad's backend/dtype ahead of
gap-only deliveries by sending a zero-length data buffer: it is recorded as
the pad's prototype and not queued as stream data.
Breaking: "the array is the backend". An element's array backend
(NumPy/PyTorch) is inferred from its data via the Python Array API, not
configured. The backend= argument is removed from every element and from
AdapterConfig; choose the backend at a source, or insert a Converter where
backends cross. See docs/user/use-array-backends.md.
Elements declare a backends capability (ClassVar, default
frozenset({"numpy"}), or ANY_BACKEND); data in an undeclared backend raises
BackendError at runtime, naming the pad and the fix.
An element declares its output backend/dtype by example with
output_prototype(self, pad) (built from input_prototype(name, dtype=,
device=)). Most transforms override nothing (dtype is inherited); a source
returns a literal prototype; a Converter (which sets changes_backend = True)
is the only element that may change namespace/device.
TSFrame.filleddata() takes no arguments (it uses the frame's spec);
SeriesBuffer.filleddata(reference) fills a single buffer's gap to match a
reference array.
Adder properly tracks per-buffer gap state: each output buffer is a gap
only when all corresponding input buffers are gaps (with fill_gaps=True,
the default) or when any input is a gap (fill_gaps=False)
Multi-pad buffer alignment now derives slice boundaries from the union of
every pad's non-gap edges, so data in mixed gap/non-gap regions is no
longer silently promoted to a gap before the element sees it
The ArrayBackend / NumpyBackend / TorchBackend / NumpyArray /
TorchArray abstractions and the set_dtype / set_device globals — dtype
and device now travel with the data.
SeriesBuffer.filleddata() with no reference, and scalar data=0/1
buffer construction (both can only numpy-default): pass a real array or a
reference array of the intended backend.
Add FileRetentionMixin to sgnts.sinks.retention, providing reusable
count-based and time-based file cleanup policies for sinks that write files
in stride mode, including a directory-scan method for recovering after restart
When a source reaches its requested end, emit heartbeats instead of raising
an error, so downstream elements can drain cleanly
AudioAdapter now raises a clear error when its stride is incompatible with
the application stride, instead of failing obscurely later
Elements now read adapter settings (stride, overlap, skip_gaps,
offset_shift, pad_zeros_startup) directly from adapter_config rather
than cached copies on the element; user code that accessed these as element
attributes should switch to adapter_config.*
Format large offset integers with _ digit separators in assertion messages
for readability
Fix Ctrl-C hangs in pipelines using TSResourceSource under SignalEOS;
the source now cooperatively drains on interrupt
Fix TSResourceSource emitting spurious heartbeats when the worker produces
data in blocks smaller than the application stride; data is now accumulated
until a full stride is ready
Fix pad_zeros_startup accounting for multi-sink elements
Fix adapter alignment producing 0-duration gap buffers, which caused
cross-pad duration mismatches in multi-pad elements
Overhaul documentation: restructure into User Guide, Developer Guide,
Background, and Reference sections following Divio guidelines; add dedicated
offset background page; split built-in elements reference into Sources,
Transforms, and Sinks; add testable code snippets throughout
Standardize on seconds for time quantities, expose _ns properties for
nanosecond equivalents
Standardize start/end parameters for TSSource and TSResourceSource
Change TSResourceSource base class from ParallelizeSourceElement to
ParallelizeBase
Have TSResourceSource derive from TSSource
Simplify TSCompose to wrap Compose while keeping the public API