metobs_toolkit.dataset.Dataset.repetitions_check

Dataset.repetitions_check(obstype: str = 'temp', max_N_repetitions: int = 5, whiteset: WhiteSet = WhiteSet(empty), use_mp: bool = True) → None

Test if an observation changes after a number of repetitions.

Perform a check that tests if the observation changes after a number of repetitions. If a value is repeated more than the specified number of times, all the repeated records are flagged as outliers.

Be aware that the performance of this check depends on the max_N_repetitions and the time resolution of the observations.

Parameters:

• obstype (str, optional) – The target observation to check. By default “temp”

• max_N_repetitions (int) – The maximum number of repetitions allowed before the records are flagged as outliers. If the number of repetitions exceeds this value, all repeated records are flagged as outliers. The default is 5.

• whiteset (WhiteSet, optional) – A WhiteSet instance containing timestamps that should be excluded from outlier detection. Records matching the whiteset criteria will not be flagged as outliers. The default is an empty WhiteSet().

• use_mp (bool, optional) – If True, the function will use multiprocessing to speed up the calculations. The default is False.

Return type:

None

Notes

• This method modifies the outliers in place and does not return anything. You can use the outliersdf property to view all flagged outliers.

• The repetitions check is similar to the persistence check, but not identical. The persistence check uses thresholds that are meteorologically based (i.e. the moving window is defined by a duration), in contrast to the repetitions check whose thresholds are instrumentally based (i.e. the “window” is defined by a number of records.)

Warning

If the minimum number of records per window is not met over the full time series, a warning is logged, and the function returns an empty DatetimeIndex.