metobs_toolkit.station.Station.persistence_check#
- Station.persistence_check(obstype: str = 'temp', timewindow: str | Timedelta = Timedelta('0 days 01:00:00'), min_records_per_window: int = 5, whiteset: WhiteSet = WhiteSet(empty)) None[source]#
Check if values are not constant in a moving time window.
Perform a persistence check on a time series to identify periods where observations remain constant within a specified time window. If the values are constant, all records in the moving window are flagged as outliers.
- Parameters:
obstype (str, optional) – The target observation to check. By default “temp”
timewindow (str or pandas.Timedelta) – The size of the rolling time window to check for persistence. The default is pandas.Timedelta(“60min”)
min_records_per_window (int) – The minimum number of non-NaN records required within the time window for the check to be valid. The default is 5
whiteset (WhiteSet, optional) – A WhiteSet instance containing timestamps that should be excluded from outlier detection. Records matching the whiteset criteria will not be flagged as outliers. The default is an empty WhiteSet().
- Return type:
None
Notes
This method modifies the outliers in place and does not return anything. You can use the outliersdf property to view all flagged outliers.
If the minimum number of records per window is locally not met, the function logs a warning and skips the persistence check.
This function can be computationally expensive for large datasets or small time windows.
The repetitions check is similar to the persistence check, but not identical. The persistence check uses thresholds that are meteorologically based (i.e. the moving window is defined by a duration), in contrast to the repetitions check whose thresholds are instrumentally based (i.e. the “window” is defined by a number of records.)
Warning
If the minimum number of records per window is not met over the full time series, a warning is logged, and the function returns an empty DatetimeIndex.