| Title: | Spatial Cross-Validation for Machine Learning |
|---|---|
| Description: | Spatial cross-validation and model evaluation for geospatial machine learning applications. Addresses spatial dependence in observations by implementing spatial block, buffered, and clustering cross-validation methods. Includes spatial leakage detection, model performance metrics, and spatial residual diagnostics for assessing model generalization across geographic space. Methods based on Brenning (2012) <doi:10.1016/j.cageo.2012.02.001>, Pohjankukka et al. (2017) <doi:10.1016/j.isprsjprs.2017.07.001>, and Roberts et al. (2017) <doi:10.1111/ecog.02881>. |
| Authors: | Mamadou SOW [aut, cre] |
| Maintainer: | Mamadou SOW <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.1.0 |
| Built: | 2026-10-11 12:31:45 UTC |
| Source: | https://github.com/sowsalim01/spatialcvr |
Compares performance metrics across different cross-validation methods to assess the impact of spatial dependence on model evaluation.
compare_cv(results_list, methods = NULL)compare_cv(results_list, methods = NULL)
results_list |
Named list of spatial_metrics objects from different CV methods |
methods |
Character vector of method names (optional, uses names of results_list) |
The function compares metrics across different CV methods:
Creates summary table with mean and SD of metrics per method
Identifies best performing method
Provides fold-level comparison
Calculates performance differences between methods
This is useful for comparing spatial CV methods against random CV to demonstrate the impact of spatial dependence.
An object of class "cv_comparison" containing:
Data frame with summary statistics for each method
Data frame with fold-level results
Method with best performance according to RMSE
Performance comparison between methods
Other model evaluation functions:
spatial_metrics()
# Train model with different CV methods results_random <- list(RMSE = 0.5, MAE = 0.4, R2 = 0.8) results_block <- list(RMSE = 0.7, MAE = 0.6, R2 = 0.6) results_cluster <- list(RMSE = 0.6, MAE = 0.5, R2 = 0.7) comparison <- compare_cv( list(random = results_random, block = results_block, cluster = results_cluster) ) print(comparison)# Train model with different CV methods results_random <- list(RMSE = 0.5, MAE = 0.4, R2 = 0.8) results_block <- list(RMSE = 0.7, MAE = 0.6, R2 = 0.6) results_cluster <- list(RMSE = 0.6, MAE = 0.5, R2 = 0.7) comparison <- compare_cv( list(random = results_random, block = results_block, cluster = results_cluster) ) print(comparison)
Analyzes the proximity between training and test observations to detect potential spatial leakage, where training and test sets are too close spatially, leading to over-optimistic performance estimates.
detect_spatial_leakage( data, folds, x = NULL, y = NULL, threshold = NULL, risk_levels = NULL )detect_spatial_leakage( data, folds, x = NULL, y = NULL, threshold = NULL, risk_levels = NULL )
data |
Spatial observations (data.frame or sf object) |
folds |
A spatial_folds object |
x |
Name of the x coordinate column (required for data.frame, ignored for sf) |
y |
Name of the y coordinate column (required for data.frame, ignored for sf) |
threshold |
Distance threshold for considering observations "too close" (default: NULL, auto-calculated) |
risk_levels |
Custom risk level thresholds as list(min, moderate, high) (default: NULL) |
The function analyzes spatial distances between training and test observations:
Calculates minimum, mean, and median distances per fold
Identifies observations within threshold distance
Assesses overall risk level
Provides recommendations for improvement
Risk levels are based on the proportion of train/test pairs that are too close:
"low": < 10% of pairs below threshold
"moderate": 10-30% of pairs below threshold
"high": > 30% of pairs below threshold
An object of class "spatial_leakage_result" containing:
CV method used
Distance analysis for each fold
Overall summary across all folds
Overall risk assessment: "low", "moderate", or "high"
Text recommendations based on analysis
Threshold used for risk assessment
Other spatial analysis functions:
spatial_distance()
data(sample_spatial_data) folds <- spatial_folds(sample_spatial_data, "longitude", "latitude", k = 5) leakage <- detect_spatial_leakage(sample_spatial_data, folds, "longitude", "latitude") print(leakage)data(sample_spatial_data) folds <- spatial_folds(sample_spatial_data, "longitude", "latitude", k = 5) leakage <- detect_spatial_leakage(sample_spatial_data, folds, "longitude", "latitude") print(leakage)
Creates a visual representation of spatial cross-validation folds, showing the spatial distribution of training and test observations.
plot_spatial_folds(folds, data, x = NULL, y = NULL, fold = 1, main = NULL, ...)plot_spatial_folds(folds, data, x = NULL, y = NULL, fold = 1, main = NULL, ...)
folds |
A spatial_folds object |
data |
Spatial observations (data.frame or sf object) |
x |
Name of the x coordinate column (required for data.frame, ignored for sf) |
y |
Name of the y coordinate column (required for data.frame, ignored for sf) |
fold |
Fold number to plot (default: 1, or "all" for all folds) |
main |
Plot title (default: NULL, auto-generated) |
... |
Additional graphical parameters passed to plot() |
Creates a scatter plot showing training and test observations with different colors. For spatial block CV, also shows block boundaries when available.
Invisibly returns the fold object
Other visualization functions:
plot_spatial_residuals()
data(sample_spatial_data) folds <- spatial_folds(sample_spatial_data, "longitude", "latitude", k = 5) plot_spatial_folds(folds, sample_spatial_data, "longitude", "latitude", fold = 1)data(sample_spatial_data) folds <- spatial_folds(sample_spatial_data, "longitude", "latitude", k = 5) plot_spatial_folds(folds, sample_spatial_data, "longitude", "latitude", fold = 1)
Creates visualizations of spatial residuals to analyze the spatial distribution of model errors.
plot_spatial_residuals(residuals_obj, type = "scatter", main = NULL, ...)plot_spatial_residuals(residuals_obj, type = "scatter", main = NULL, ...)
residuals_obj |
A spatial_residuals object |
type |
Type of plot: "scatter", "histogram", or "qq" (default: "scatter") |
main |
Plot title (default: NULL, auto-generated) |
... |
Additional graphical parameters passed to plot() |
Creates different types of residual plots:
"scatter": Spatial scatter plot of residuals colored by magnitude
"histogram": Histogram of residual values
"qq": Q-Q plot for normality assessment
Invisibly returns the residuals object
Other visualization functions:
plot_spatial_folds()
observed <- c(1, 2, 3, 4, 5) predicted <- c(1.1, 2.2, 2.8, 4.1, 4.9) coords <- cbind(x = c(0, 1, 2, 3, 4), y = c(0, 1, 2, 3, 4)) residuals <- spatial_residuals(observed, predicted, coords) plot_spatial_residuals(residuals, type = "scatter")observed <- c(1, 2, 3, 4, 5) predicted <- c(1.1, 2.2, 2.8, 4.1, 4.9) coords <- cbind(x = c(0, 1, 2, 3, 4), y = c(0, 1, 2, 3, 4)) residuals <- spatial_residuals(observed, predicted, coords) plot_spatial_residuals(residuals, type = "scatter")
Print method for cv_comparison objects
## S3 method for class 'cv_comparison' print(x, ...)## S3 method for class 'cv_comparison' print(x, ...)
x |
A cv_comparison object |
... |
Additional arguments (ignored) |
Invisibly returns the object
Print method for spatial_distance objects
## S3 method for class 'spatial_distance' print(x, ...)## S3 method for class 'spatial_distance' print(x, ...)
x |
A spatial_distance object |
... |
Additional arguments (ignored) |
Invisibly returns the object
Print method for spatial_folds objects
## S3 method for class 'spatial_folds' print(x, ...)## S3 method for class 'spatial_folds' print(x, ...)
x |
A spatial_folds object |
... |
Additional arguments (ignored) |
Invisibly returns the object
Print method for spatial_leakage_result objects
## S3 method for class 'spatial_leakage_result' print(x, ...)## S3 method for class 'spatial_leakage_result' print(x, ...)
x |
A spatial_leakage_result object |
... |
Additional arguments (ignored) |
Invisibly returns the object
Print method for spatial_metrics objects
## S3 method for class 'spatial_metrics' print(x, ...)## S3 method for class 'spatial_metrics' print(x, ...)
x |
A spatial_metrics object |
... |
Additional arguments (ignored) |
Invisibly returns the object
Print method for spatial_residuals objects
## S3 method for class 'spatial_residuals' print(x, ...)## S3 method for class 'spatial_residuals' print(x, ...)
x |
A spatial_residuals object |
... |
Additional arguments (ignored) |
Invisibly returns the object
A synthetic dataset containing 200 spatial observations with coordinates and variables for demonstration and testing purposes.
data(sample_spatial_data)data(sample_spatial_data)
A data frame with 200 rows and 6 columns:
Unique identifier for each observation (integer)
X coordinate in projected CRS (numeric)
Y coordinate in projected CRS (numeric)
First predictor variable with spatial structure (numeric)
Second predictor variable with spatial structure (numeric)
Target variable for prediction (numeric)
The data was generated to simulate spatial autocorrelation:
Coordinates are in a UTM-like projected system (0-1000 range)
Variables show gradients in X and Y directions
Target variable combines predictors with spatial structure
Some missing values are included for testing NA handling
Synthetic data generated for package demonstration
data(sample_spatial_data) head(sample_spatial_data) summary(sample_spatial_data)data(sample_spatial_data) head(sample_spatial_data) summary(sample_spatial_data)
Creates spatial cross-validation folds by dividing the study area into rectangular blocks and assigning observations to folds based on their block membership. This method ensures spatial separation between training and test sets.
spatial_block_folds( data, x = NULL, y = NULL, k = 5, block_size = NULL, n_blocks = NULL, assignment = "systematic", seed = NULL )spatial_block_folds( data, x = NULL, y = NULL, k = 5, block_size = NULL, n_blocks = NULL, assignment = "systematic", seed = NULL )
data |
Spatial observations (data.frame or sf object) |
x |
Name of the x coordinate column (required for data.frame, ignored for sf) |
y |
Name of the y coordinate column (required for data.frame, ignored for sf) |
k |
Number of folds (default: 5) |
block_size |
Size of blocks as c(width, height) in coordinate units |
n_blocks |
Number of blocks in x and y directions as c(nx, ny) |
assignment |
Strategy for assigning blocks to folds: "systematic" or "random" |
seed |
Random seed for reproducibility (default: NULL) |
The spatial block method divides the spatial extent into a grid of rectangular blocks. Two approaches are available:
Fixed block size: Specify block_size to control block dimensions
Fixed number of blocks: Specify n_blocks to control grid resolution
If neither is specified, the function attempts to create approximately sqrt(k) blocks in each direction.
Assignment strategies:
"systematic": Assigns blocks to folds in a systematic pattern
"random": Randomly assigns blocks to folds
An object of class "spatial_folds" containing fold assignments
Other spatial cross-validation functions:
spatial_buffer_folds(),
spatial_cluster_folds(),
spatial_folds(),
spatial_split()
data(sample_spatial_data) folds <- spatial_block_folds( data = sample_spatial_data, x = "longitude", y = "latitude", k = 5, block_size = c(200, 200) )data(sample_spatial_data) folds <- spatial_block_folds( data = sample_spatial_data, x = "longitude", y = "latitude", k = 5, block_size = c(200, 200) )
Creates spatial cross-validation folds by excluding training observations within a specified buffer radius around test observations. This method ensures a minimum spatial separation between training and test sets.
spatial_buffer_folds( data, x = NULL, y = NULL, k = 5, buffer_radius = NULL, seed = NULL )spatial_buffer_folds( data, x = NULL, y = NULL, k = 5, buffer_radius = NULL, seed = NULL )
data |
Spatial observations (data.frame or sf object) |
x |
Name of the x coordinate column (required for data.frame, ignored for sf) |
y |
Name of the y coordinate column (required for data.frame, ignored for sf) |
k |
Number of folds (default: 5) |
buffer_radius |
Buffer radius in coordinate units |
seed |
Random seed for reproducibility (default: NULL) |
The buffered method ensures that for each test observation, no training observation falls within the specified buffer radius. This is particularly useful when you need strict control over the minimum distance between training and test observations.
Note: This method requires a projected CRS for accurate distance calculations. Using geographic coordinates (longitude/latitude) will produce warnings.
An object of class "spatial_folds" containing fold assignments
Other spatial cross-validation functions:
spatial_block_folds(),
spatial_cluster_folds(),
spatial_folds(),
spatial_split()
data(sample_spatial_data) folds <- spatial_buffer_folds( data = sample_spatial_data, x = "longitude", y = "latitude", k = 5, buffer_radius = 100 )data(sample_spatial_data) folds <- spatial_buffer_folds( data = sample_spatial_data, x = "longitude", y = "latitude", k = 5, buffer_radius = 100 )
Creates spatial cross-validation folds by clustering observations spatially and assigning clusters to folds. This method is useful for data with complex spatial structure.
spatial_cluster_folds( data, x = NULL, y = NULL, k = 5, n_clusters = NULL, seed = NULL )spatial_cluster_folds( data, x = NULL, y = NULL, k = 5, n_clusters = NULL, seed = NULL )
data |
Spatial observations (data.frame or sf object) |
x |
Name of the x coordinate column (required for data.frame, ignored for sf) |
y |
Name of the y coordinate column (required for data.frame, ignored for sf) |
k |
Number of folds (default: 5) |
n_clusters |
Number of spatial clusters (default: k) |
seed |
Random seed for reproducibility (default: NULL) |
The clustering method groups spatially proximate observations into clusters using k-means clustering on coordinates, then assigns clusters to folds. This ensures spatial coherence within folds while maintaining separation between folds.
An object of class "spatial_folds" containing fold assignments
Other spatial cross-validation functions:
spatial_block_folds(),
spatial_buffer_folds(),
spatial_folds(),
spatial_split()
data(sample_spatial_data) folds <- spatial_cluster_folds( data = sample_spatial_data, x = "longitude", y = "latitude", k = 5, n_clusters = 10 )data(sample_spatial_data) folds <- spatial_cluster_folds( data = sample_spatial_data, x = "longitude", y = "latitude", k = 5, n_clusters = 10 )
Computes distances between training and test observations for each fold in a spatial cross-validation setup. This helps assess the spatial separation between training and test sets.
spatial_distance(data, folds, x = NULL, y = NULL)spatial_distance(data, folds, x = NULL, y = NULL)
data |
Spatial observations (data.frame or sf object) |
folds |
A spatial_folds object |
x |
Name of the x coordinate column (required for data.frame, ignored for sf) |
y |
Name of the y coordinate column (required for data.frame, ignored for sf) |
For each fold, the function calculates Euclidean distances between all pairs of training and test observations. This provides a comprehensive view of spatial separation.
Note: Distance calculations use Euclidean distance on the provided coordinates. For accurate metric distances, ensure data is in a projected CRS. Geographic coordinates (longitude/latitude) will produce approximate distances.
A list containing distance information for each fold:
Fold number
Minimum distance between train and test observations
Mean distance between train and test observations
Median distance between train and test observations
Maximum distance between train and test observations
Standard deviation of distances
Distance quantiles (25%, 50%, 75%)
Other spatial analysis functions:
detect_spatial_leakage()
data(sample_spatial_data) folds <- spatial_folds(sample_spatial_data, "longitude", "latitude", k = 5) distances <- spatial_distance(sample_spatial_data, folds, "longitude", "latitude") print(distances)data(sample_spatial_data) folds <- spatial_folds(sample_spatial_data, "longitude", "latitude", k = 5) distances <- spatial_distance(sample_spatial_data, folds, "longitude", "latitude") print(distances)
Creates spatially separated folds for model evaluation to address spatial dependence in observations. This function serves as a wrapper for different spatial cross-validation methods.
spatial_folds( data, x = NULL, y = NULL, k = 5, method = "block", seed = NULL, ... )spatial_folds( data, x = NULL, y = NULL, k = 5, method = "block", seed = NULL, ... )
data |
Spatial observations (data.frame or sf object) |
x |
Name of the x coordinate column (required for data.frame, ignored for sf) |
y |
Name of the y coordinate column (required for data.frame, ignored for sf) |
k |
Number of folds (default: 5) |
method |
Spatial CV method: "block", "buffer", "cluster", or "random" (default: "block") |
seed |
Random seed for reproducibility (default: NULL) |
... |
Additional parameters passed to specific methods |
The function validates inputs and dispatches to the appropriate spatial CV method:
"block": Spatial block cross-validation (default)
"buffer": Buffered cross-validation
"cluster": Spatial clustering cross-validation
"random": Random spatial split (baseline)
An object of class "spatial_folds" containing:
List of train/test indices for each fold
Method used for fold creation
Number of folds
List of parameters used
Coordinate matrix of observations
Coordinate reference system
Additional metadata
Other spatial cross-validation functions:
spatial_block_folds(),
spatial_buffer_folds(),
spatial_cluster_folds(),
spatial_split()
data(sample_spatial_data) folds <- spatial_folds( data = sample_spatial_data, x = "longitude", y = "latitude", k = 5, method = "block" )data(sample_spatial_data) folds <- spatial_folds( data = sample_spatial_data, x = "longitude", y = "latitude", k = 5, method = "block" )
Calculates standard model evaluation metrics for comparing observed and predicted values. These metrics are commonly used in machine learning and spatial modeling.
spatial_metrics(observed, predicted, na.rm = TRUE)spatial_metrics(observed, predicted, na.rm = TRUE)
observed |
Numeric vector of observed values |
predicted |
Numeric vector of predicted values |
na.rm |
Logical, whether to remove NA values (default: TRUE) |
The function calculates the following metrics:
RMSE: sqrt(mean((observed - predicted)^2))
MAE: mean(abs(observed - predicted))
R2: 1 - sum((observed - predicted)^2) / sum((observed - mean(observed))^2)
MAPE: mean(abs((observed - predicted) / observed)) * 100 (only when no zeros in observed)
An object of class "spatial_metrics" containing:
Number of observations
Root Mean Square Error
Mean Absolute Error
R-squared (coefficient of determination)
Mean Absolute Percentage Error (when applicable)
Other model evaluation functions:
compare_cv()
observed <- c(1, 2, 3, 4, 5) predicted <- c(1.1, 2.2, 2.8, 4.1, 4.9) metrics <- spatial_metrics(observed, predicted) print(metrics)observed <- c(1, 2, 3, 4, 5) predicted <- c(1.1, 2.2, 2.8, 4.1, 4.9) metrics <- spatial_metrics(observed, predicted) print(metrics)
Analyzes the spatial distribution of model residuals to detect spatial patterns in prediction errors. This helps identify whether model errors are spatially autocorrelated, which may indicate missing spatial predictors or inappropriate model specification.
spatial_residuals(observed, predicted, coordinates, x = NULL, y = NULL)spatial_residuals(observed, predicted, coordinates, x = NULL, y = NULL)
observed |
Numeric vector of observed values |
predicted |
Numeric vector of predicted values |
coordinates |
Coordinate matrix or data.frame with x and y columns |
x |
Name of x coordinate column if coordinates is a data.frame |
y |
Name of y coordinate column if coordinates is a data.frame |
The function calculates residuals and provides summary statistics:
Mean, median, SD of residuals
Quantiles of residuals
Normality test statistics (if sufficient data)
Spatial autocorrelation analysis can be added in future versions when appropriate dependencies (e.g., spdep) are available.
An object of class "spatial_residuals" containing:
Residual values (observed - predicted)
Observed values
Predicted values
Coordinate matrix
Summary statistics of residuals
observed <- c(1, 2, 3, 4, 5) predicted <- c(1.1, 2.2, 2.8, 4.1, 4.9) coords <- cbind(x = c(0, 1, 2, 3, 4), y = c(0, 1, 2, 3, 4)) residuals <- spatial_residuals(observed, predicted, coords) print(residuals)observed <- c(1, 2, 3, 4, 5) predicted <- c(1.1, 2.2, 2.8, 4.1, 4.9) coords <- cbind(x = c(0, 1, 2, 3, 4), y = c(0, 1, 2, 3, 4)) residuals <- spatial_residuals(observed, predicted, coords) print(residuals)
Creates a random spatial split serving as a baseline for comparison with spatial cross-validation methods. This represents traditional random cross-validation without spatial considerations.
spatial_split(data, x = NULL, y = NULL, k = 5, seed = NULL)spatial_split(data, x = NULL, y = NULL, k = 5, seed = NULL)
data |
Spatial observations (data.frame or sf object) |
x |
Name of the x coordinate column (required for data.frame, ignored for sf) |
y |
Name of the y coordinate column (required for data.frame, ignored for sf) |
k |
Number of folds (default: 5) |
seed |
Random seed for reproducibility (default: NULL) |
This method performs standard random k-fold cross-validation without any spatial constraints. It serves as a baseline to compare against spatial cross-validation methods and demonstrate the impact of spatial dependence on model evaluation.
An object of class "spatial_folds" containing fold assignments
Other spatial cross-validation functions:
spatial_block_folds(),
spatial_buffer_folds(),
spatial_cluster_folds(),
spatial_folds()
data(sample_spatial_data) folds <- spatial_split( data = sample_spatial_data, x = "longitude", y = "latitude", k = 5 )data(sample_spatial_data) folds <- spatial_split( data = sample_spatial_data, x = "longitude", y = "latitude", k = 5 )