Introduction

Environmental sensor data are commonly used in machine learning workflows for prediction and forecasting. However, the way observations are divided into training and testing data can introduce validation risks.

sensorLeak provides diagnostic tools for identifying several potential sources of information leakage or overly optimistic validation:

temporal ordering problems
sensors shared between training and testing data
spatially close observations
overlapping temporal windows

The package reports potential risks rather than automatically declaring a dataset invalid. Whether a detected condition represents leakage depends on the intended generalization target of the model.

Example data

The package includes a small synthetic environmental sensor dataset.

data(sensor_data)

head(sensor_data)
#>             timestamp sensor_id latitude longitude PM25 split
#> 1 2026-01-01 08:00:00        S1  13.0827   80.2707 42.1 train
#> 2 2026-01-01 09:00:00        S1  13.0827   80.2707 45.3 train
#> 3 2026-01-01 10:00:00        S1  13.0827   80.2707 47.8 train
#> 4 2026-01-01 11:00:00        S1  13.0827   80.2707 44.2  test
#> 5 2026-01-01 12:00:00        S2  13.0830   80.2710 38.5 train
#> 6 2026-01-01 13:00:00        S2  13.0830   80.2710 40.1 train

The dataset contains timestamps, sensor identifiers, geographic coordinates, PM2.5 measurements, and train/test assignments.

Running an audit

The sensor_audit() function combines several checks into a single audit object.

audit <- sensor_audit(
  sensor_data,
  time = "timestamp",
  sensor = "sensor_id",
  split = "split",
  lat = "latitude",
  lon = "longitude"
)

The audit can be printed directly:

audit
#> Environmental Sensor Leakage Audit
#> ==================================
#> 
#> Temporal risk: 3 finding(s)
#> Sensor risk:   3 sensor(s) shared
#> Spatial risk:  19 nearby pair(s)
#> 
#> Overall: potential validation risks detected.
#> 
#> Use summary(x) for detailed results.

A more detailed summary is available with:

summary(audit)
#> Sensor Leakage Audit Summary
#> ============================
#> 
#> Temporal findings: 3
#> Shared sensors:    3
#> Spatial pairs:     19

Temporal leakage

Temporal validation should respect the intended direction of time.

The temporal check identifies test observations that occur at or before the end of the training period.

check_temporal_leakage(
  sensor_data,
  time = "timestamp",
  split = "split"
)
#>   test_row           test_time        training_end
#> 1        4 2026-01-01 11:00:00 2026-01-01 17:00:00
#> 2        7 2026-01-01 14:00:00 2026-01-01 17:00:00
#> 3        8 2026-01-01 15:00:00 2026-01-01 17:00:00

A detected observation does not necessarily mean the workflow is incorrect. For example, different validation strategies may have different temporal requirements.

Shared sensors

The sensor check identifies sensors that occur in both training and testing observations.

check_sensor_leakage(
  sensor_data,
  sensor = "sensor_id",
  split = "split"
)
#>   sensor train_observations test_observations
#> 1     S1                  3                 1
#> 2     S2                  2                 2
#> 3     S3                  2                 2

Shared sensors can be appropriate when the intended task is to predict future observations from known sensors.

However, if the intended goal is to generalize to completely unseen sensors, shared sensors may represent a validation risk.

Spatial proximity

Environmental observations that are geographically close may contain similar environmental conditions.

sensorLeak can identify training/testing observation pairs within a specified geographic distance.

check_spatial_leakage(
  sensor_data,
  lat = "latitude",
  lon = "longitude",
  split = "split",
  threshold = 1
)
#>    train_row test_row distance_km
#> 1          1        4  0.00000000
#> 2          1        7  0.04656778
#> 3          1        8  0.04656778
#> 4          2        4  0.00000000
#> 5          2        7  0.04656778
#> 6          2        8  0.04656778
#> 7          3        4  0.00000000
#> 8          3        7  0.04656778
#> 9          3        8  0.04656778
#> 10         5        4  0.04656778
#> 11         5        7  0.00000000
#> 12         5        8  0.00000000
#> 13         6        4  0.04656778
#> 14         6        7  0.00000000
#> 15         6        8  0.00000000
#> 16         9       11  0.00000000
#> 17         9       12  0.00000000
#> 18        10       11  0.00000000
#> 19        10       12  0.00000000

The threshold is specified in kilometres.

Spatial proximity should be interpreted according to the intended spatial generalization of the model.

Overlapping temporal windows

Some environmental workflows use rolling, aggregation, or observation windows.

If a training window and a testing window overlap for the same sensor, information from the same time interval may occur in both datasets.

This can be checked with:

window_data <- data.frame(
  sensor_id = c("S1", "S1"),
  window_start = as.POSIXct(c(
    "2026-01-01 10:00:00",
    "2026-01-01 13:00:00"
  )),
  window_end = as.POSIXct(c(
    "2026-01-01 14:00:00",
    "2026-01-01 18:00:00"
  )),
  split = c("train", "test")
)

check_window_leakage(
  window_data,
  start = "window_start",
  end = "window_end",
  split = "split",
  sensor = "sensor_id"
)
#>   sensor train_row test_row       overlap_start         overlap_end
#> 1     S1         1        2 2026-01-01 13:00:00 2026-01-01 14:00:00
#>   overlap_duration
#> 1             3600

Interpreting audit results

sensorLeak is intended as a diagnostic tool.

A reported finding should be interpreted in the context of:

the prediction task,
the intended generalization target,
the temporal structure of the data,
the spatial structure of the study,
and the way features were constructed.

The package does not replace domain-specific decisions about how training and testing data should be separated.

Conclusion

sensorLeak provides a lightweight way to audit environmental sensor machine learning datasets for several potential validation risks.

The individual checks can be used independently, while sensor_audit() provides a convenient combined interface.

Introduction to sensorLeak

sensorLeak authors