kdiagram.plot.uncertainty.plot_coverage

kdiagram.plot.uncertainty.plot_coverage(y_true, *y_preds, names=None, q=None, kind='line', cmap='viridis', pie_startangle=140, pie_autopct='%1.1f%%', radar_color='tab:blue', radar_fill_alpha=0.25, radar_line_style='o-', cov_fill=False, figsize=None, title=None, dpi=300, savefig=None, verbose=1, ax=None)[source]

Plot overall coverage scores for forecast intervals or points.

Computes and visualizes the empirical coverage rate, which is the fraction of times the true values fall within predicted quantile intervals (or match point forecasts). Supports comparing multiple models or prediction sets using various chart types.

This plot provides a high-level summary of whether prediction intervals are well-calibrated on average across the entire dataset. It’s useful for quickly comparing the overall reliability of uncertainty estimates from different models or parameter settings before diving into finer diagnostics [1].

Parameters:
y_truearray_like of shape (n_samples,)

The ground truth target values. Must be a 1D array-like object (list, NumPy array, pandas Series).

*y_predsarray_like

Variable number of prediction arrays. Each positional argument should be an array-like object corresponding to a model or prediction set. The shape determines interpretation:

  • shape (n_samples, n_quantiles): Represents quantile forecasts (e.g., lower, median, upper bounds). Coverage is calculated based on the interval defined by the minimum and maximum quantile columns provided (after sorting based on q if given, or by value otherwise). Requires q parameter for proper interval definition if quantiles are not ordered.

  • shape (n_samples,): Represents point forecasts. Coverage is calculated as the proportion of exact matches to y_true (typically near zero for continuous data).

nameslist of str, optional

Labels for each prediction set provided in *y_preds. Used in legends and axis labels. If None or shorter than the number of prediction sets, default names like “Model_1”, “Model_2”, etc., are generated for the missing ones. Default is None.

qlist of float, optional

Quantile levels corresponding to the columns in 2D y_preds arrays. Values must be between 0 and 1. Used to identify and sort quantile columns correctly before determining the interval bounds (minimum and maximum specified quantile) for coverage calculation. Required if y_preds contains 2D arrays representing unordered quantiles. Default is None.

kind{‘line’, ‘bar’, ‘pie’, ‘radar’}, optional

The type of plot to generate for visualizing coverage scores. Default is 'line'.

  • 'line': Line chart connecting coverage scores for each prediction set.

  • 'bar': Bar chart showing the coverage score for each prediction set.

  • 'pie': Pie chart where each slice represents a model’s coverage score as a fraction of the sum of all scores.

  • 'radar': Radar (spider) chart where each axis represents a prediction set, and the distance along the axis indicates its coverage score.

cmapstr, optional

Matplotlib colormap name used for coloring slices in the ‘pie’ chart or the gradient fill in single-model radar plots when cov_fill=True. Default is 'viridis'.

pie_startanglefloat, optional

Starting angle in degrees for the first slice in the ‘pie’ chart. Rotates the chart orientation. Default is 140.

pie_autopctstr or None, optional

String formatting pattern (e.g., '%1.1f%%') used to label pie slices with their numeric value (percentage). If None, no numeric labels are shown. Default is '%1.1f%%'.

radar_colorstr, optional

Color specification for the main line and optional fill in the ‘radar’ chart. Accepts any valid Matplotlib color. Default is 'tab:blue'.

radar_fill_alphafloat, optional

Transparency level (alpha) for the filled area in the ‘radar’ chart when cov_fill=True. Value between 0 (transparent) and 1 (opaque). Default is 0.25.

radar_line_stylestr, optional

Matplotlib line and marker style string for the radar chart outline (e.g., 'o-', '--', ':'). Default is 'o-'.

cov_fillbool, optional

If True and kind='radar', fills the area under the radar plot lines. For a single model, creates a radial gradient; for multiple models, fills polygons. Default is False.

figsizetuple of (float, float), optional

Figure size (width, height) in inches. If None, uses Matplotlib’s default figure size.

titlestr, optional

Optional title for the plot. Default is None.

savefigstr, optional

If not None, specifies the full path (including filename and extension, e.g., ‘coverage.png’) to save the figure. If None, the plot is displayed interactively. Default is None.

verbose{0, 1}, optional

Controls printing of coverage scores to the console. - 0: Silent. - 1: Prints a formatted summary of coverage scores. Default is 1.

Returns:
None

This function generates and displays or saves a Matplotlib plot but does not return any object.

Raises:
ValueError

If q contains values outside (0, 1), or if kind is not one of the allowed options, or if dimensions of y_preds and q are incompatible.

TypeError

If inputs y_true or y_preds contain non-numeric data.

Parameters:

See also

plot_coverage_diagnostic

Plot point-wise coverage status.

numpy.mean

Used to calculate the average coverage rate.

Notes

The empirical coverage rate is a key metric for assessing the calibration of probabilistic forecasts, particularly prediction intervals [1].

Calculation:

  • For quantile intervals (2D y_preds), the interval for each sample \(i\) is defined by the minimum and maximum predicted values across the specified quantiles for that sample,

    \([\hat{y}_{i}^{(\ell)}, \hat{y}_{i}^{(u)}]\). Coverage is:

    (1)\[\text{Coverage} = \frac{1}{N}\sum_{i=1}^{N} \mathbf{1}\{\hat{y}_{i}^{(\ell)} \leq y_i \leq \hat{y}_{i}^{(u)}\}\]

    where \(\mathbf{1}\{\cdot\}\) is 1 if true, 0 otherwise.

  • For point forecasts (1D y_preds), coverage is the proportion of exact matches:

    (2)\[\text{Coverage} = \frac{1}{N}\sum_{i=1}^{N} \mathbf{1}\{ \hat{y}_i = y_i \}\]

    This is typically very low for continuous \(y_true\).

Interpretation: Compare the calculated coverage score(s) to the nominal coverage rate implied by the quantiles (e.g., 80% for a Q10-Q90 interval). Scores far from the nominal rate suggest miscalibration (intervals too wide or too narrow on average) [2].

Radar Plot Fill (`cov_fill=True`):

  • Single model: Shows a circular gradient fill from the center out to the calculated coverage score radius, with a solid reference line at the coverage score.

  • Multiple models: Fills the polygon defined by the coverage scores for all models with a semi-transparent color.

References

Examples

>>> import numpy as np
>>> from kdiagram.plot.uncertainty import plot_coverage
>>> # True values
>>> y_true = np.random.rand(100) * 10
>>> # 3-quantile predictions (Q10, Q50, Q90) for two models
>>> y_pred_A = np.sort(np.random.normal(
...    loc=9, scale=2, size=(100, 3)), axis=1)
>>> y_pred_B = np.sort(np.random.normal(
...    loc=11, scale=3, size=(100, 3)), axis=1)
>>> q_levels = [0.1, 0.5, 0.9]

>>> # Bar chart comparison
>>> plot_coverage(y_true, y_pred_A, y_pred_B, q=q_levels,
...               names=['Model A', 'Model B'], kind='bar',
...               title='Coverage Comparison (Bar)')

>>> # Radar chart comparison with fill
>>> plot_coverage(y_true, y_pred_A, y_pred_B, q=q_levels,
...               names=['Model A', 'Model B'], kind='radar',
...               title='Coverage Comparison (Radar)',
...               cov_fill=True, cmap='Set2')