kdiagram.utils.compute_crps

kdiagram.utils.compute_crps(y_true, y_preds_quantiles, quantiles)[source]

Approximates the Continuous Ranked Probability Score (CRPS).

The CRPS is calculated as the average of the Pinball Loss across all provided quantiles. It is a proper scoring rule that assesses both calibration and sharpness simultaneously. A lower score is better.

Parameters:
y_truenp.ndarray

1D array of the true observed values.

y_preds_quantilesnp.ndarray

2D array of quantile forecasts.

quantilesnp.ndarray

1D array of the quantile levels.

Returns:
float

The average CRPS over all observations.

Parameters:
Return type:

float

See also

compute_pinball_loss

The underlying metric for a single quantile.

plot_crps_comparison

A visualization of this score.

Notes

The Continuous Ranked Probability Score (CRPS) is a widely used metric for evaluating probabilistic forecasts [1]. It is approximated here by averaging the Pinball Loss \(\mathcal{L}_{\tau}\) over all \(M\) provided quantile levels \(\tau\).

The Pinball Loss for a single quantile forecast \(q\) at level \(\tau\) is:

(1)\[\begin{split}\mathcal{L}_{\tau}(q, y) = \begin{cases} (y - q) \tau & \text{if } y \ge q \\ (q - y) (1 - \tau) & \text{if } y < q \end{cases}\end{split}\]

The final score is the average over all observations and all quantiles.

References

Examples

>>> import numpy as np
>>> from kdiagram.utils.mathext import compute_crps
>>>
>>> # Define true values and quantile forecasts for 2 observations
>>> y_true = np.array([10, 25])
>>> quantiles = np.array([0.1, 0.5, 0.9])
>>> y_preds = np.array([
...     [8, 11, 13],  # Forecast for y_true = 10
...     [20, 22, 26]   # Forecast for y_true = 25
... ])
>>>
>>> # Calculate the average CRPS
>>> crps_score = compute_crps(y_true, y_preds, quantiles)
>>> print(f"Average CRPS: {crps_score:.3f}")
Expected Output
Average CRPS: 1.467