kdiagram.plot.probabilistic.plot_crps_comparison

kdiagram.plot.probabilistic.plot_crps_comparison(y_true, *y_preds_quantiles, quantiles, names=None, title='Probabilistic Forecast Performance (CRPS)', figsize=(8.0, 8.0), cmap='viridis', marker='o', s=100, acov='default', show_grid=True, grid_props=None, mask_radius=False, savefig=None, dpi=300, ax=None)[source]

Plots a Polar CRPS Comparison Diagram.

This function visualizes the overall performance of one or more probabilistic forecasts using the Continuous Ranked Probability Score (CRPS). The CRPS is a proper scoring rule that assesses both calibration and sharpness simultaneously. A lower CRPS value indicates a better forecast. In this plot, models closer to the center are superior.

Parameters:
y_truenp.ndarray

1D array of observed (true) values.

*y_preds_quantilesnp.ndarray

One or more 2D arrays of quantile forecasts. Each array corresponds to a different model, with shape (n_samples, n_quantiles).

quantilesnp.ndarray

1D array of the quantile levels corresponding to the columns of the prediction arrays.

nameslist of str, optional

Display names for each of the models. If not provided, generic names like 'Model 1' will be generated.

titlestr, default=”Probabilistic Forecast Performance (CRPS)”

The title for the plot.

figsizetuple of (float, float), default=(8, 8)

The figure size in inches.

cmapstr, default=’viridis’

The colormap used to assign a unique color to each model’s marker.

markerstr, default=’o’

The marker style for the points representing each model.

sint, default=100

The size of the markers.

acov{‘default’, ‘half_circle’, ‘quarter_circle’,

‘eighth_circle’}, default=’default’ Angular coverage of the polar sector.

  • 'default' : full circle, \(2\pi\) (360°)

  • 'half_circle' : \(\pi\) (180°)

  • 'quarter_circle' : \(\pi/2\) (90°)

  • 'eighth_circle' : \(\pi/4\) (45°)

show_gridbool, default=True

Toggle the visibility of the polar grid lines.

grid_propsdict, optional

Custom keyword arguments passed to the grid for styling.

mask_radiusbool, default=False

If True, hide the radial tick labels.

savefigstr, optional

The file path to save the plot. If None, the plot is displayed interactively.

dpiint, default=300

The resolution (dots per inch) for the saved figure.

Returns:
axmatplotlib.axes.Axes

The Matplotlib Axes object containing the plot.

Parameters:
Return type:

Axes

Notes

The Continuous Ranked Probability Score (CRPS) is a widely used metric for evaluating probabilistic forecasts [1]. For a single observation \(y\) and a predictive CDF \(F\), it is defined as:

(1)\[\begin{split}\text{CRPS}(F, y) = \int_{-\infty}^{\infty}\\ (F(x) - \mathbf{1}\{x \ge y\})^2 dx\end{split}\]

where \(\mathbf{1}\) is the Heaviside step function.

When the forecast is given as a set of \(M\) quantiles \(\{q_1, ..., q_M\}\), the CRPS can be approximated by averaging the pinball loss \(\mathcal{L}_{\tau}\) over the quantile levels \(\tau \in \{ \tau_1, ..., \tau_M \}\):

(2)\[\begin{split}\text{CRPS}(F, y) \approx \frac{1}{M} \sum_{j=1}^{M} 2\\ \mathcal{L}_{\tau_j}(q_j, y)\end{split}\]

The pinball loss for a quantile \(\tau\) is:

(3)\[\begin{split}\mathcal{L}_{\tau}(q, y) = \begin{cases} (y - q) \tau & \text{if } y \ge q \\ (q - y) (1 - \tau) & \text{if } y < q \end{cases}\end{split}\]

This function calculates the average CRPS over all observations for each model and plots it as the radial coordinate.

References

Examples

>>> import numpy as np
>>> from scipy.stats import norm
>>> from kdiagram.plot.probabilistic import plot_crps_comparison
>>>
>>> # Generate synthetic data
>>> np.random.seed(42)
>>> n_samples = 1000
>>> y_true = np.random.normal(loc=10, scale=5, size=n_samples)
>>> quantiles = np.linspace(0.05, 0.95, 19)
>>>
>>> # Create forecasts for three models
>>> good_preds = norm.ppf(
...     quantiles, loc=y_true[:, np.newaxis], scale=5
... )
>>> sharp_biased_preds = norm.ppf(
...     quantiles, loc=y_true[:, np.newaxis] - 2, scale=3
... )
>>> wide_preds = norm.ppf(
...     quantiles, loc=y_true[:, np.newaxis], scale=8
... )
>>>
>>> # Generate the plot
>>> ax = plot_crps_comparison(
...     y_true,
...     good_preds,
...     sharp_biased_preds,
...     wide_preds,
...     quantiles=quantiles,
...     names=["Good", "Sharp/Biased", "Wide"]
... )