kdiagram.utils.pivot_q_data

kdiagram.utils.pivot_q_data(df, value_prefix, dt_col='dt_col', q=None, spatial_cols=None, error='raise', verbose=0)[source]

Convert a long/tidy-wide quantile table back to a time-embedded wide format with columns named like {value_prefix}_{t}_q{α}.

This function expects one column per quantile in the input (e.g., subs_q0.1, subs_q0.5) plus a temporal column dt_col and optional spatial columns. It reconstructs the original wide layout by creating a column for each pair (t,α) and moving values into {value_prefix}_{t}_q{α}.

(1)\[\mathcal{S}=\text{spatial indices},\quad \mathcal{T}=\text{times},\quad \mathcal{Q}=\text{quantiles}\]

Input frame \(\mathbf{L}\) has columns

(2)\[\{\text{spatial}\}\ \cup\ \{\texttt{dt\_col}\}\ \cup\ \{\ \texttt{f"{value\_prefix}\_q\{\alpha\}"}\ :\ \alpha\in\mathcal{Q}\ \}.\]

The output frame \(\mathbf{W}\) has columns

(3)\[\{\text{spatial}\}\ \cup\ \{\ \texttt{f"{value\_prefix}\_\{t\}\_q\{\alpha\}"}\ :\ t\in\mathcal{T},\,\alpha\in\mathcal{Q}\ \}.\]

For each location \(s\in\mathcal{S}\), time \(t\), and quantile \(\alpha\),

(4)\[\mathbf{W}\big[s,\ \texttt{f"{value\_prefix}\_\{t\}\_q\{\alpha\}"}\big] \ =\ \mathbf{L}\big[(s,t),\ \texttt{f"{value\_prefix}\_q\{\alpha\}"}\big].\]
Parameters:
dfpandas.DataFrame

Input table with:

  • Spatial columns (optional)

  • Temporal column dt_col

  • One column per quantile: {value_prefix}_q{α}

value_prefixstr

Base measurement name (e.g., 'subs'). Used to reconstruct wide column names.

dt_colstr, default=’dt_col’

Temporal column from which tokens t are taken.

qlist of {float, str}, optional

Quantiles to include in the output. If None, all detected quantiles are used.

spatial_colstuple[str, …] or list[str], optional

Names of spatial coordinate columns. If provided, used as the index during pivot and preserved in the output.

error{‘raise’, ‘warn’, ‘ignore’}, default=’raise’

Handling for missing components:

  • 'raise' : raise ValueError with details

  • 'warn' : warn and return an empty frame or partial

  • 'ignore': silently return partial results

verbose{0,1,2,3,4,5}, default=0

Detail level for progress messages.

Returns:
pandas.DataFrame

Wide DataFrame with columns:

  • Spatial columns (if provided)

  • For each \(t\in\mathcal{T}\) and \(\alpha\in\mathcal{Q}\), a column named {value_prefix}_{t}_q{α}.

Parameters:
Return type:

DataFrame

See also

melt_q_data

Forward reshape: parse time-embedded quantile columns and produce one column per quantile.

pandas.DataFrame.pivot_table

Core reshaping used in the wide reconstruction.

Notes

  • Quantile columns in the input must follow the pattern {value_prefix}_q{α}. The temporal values are taken from dt_col.

  • Column names in the output are normalized with compact fixed-point quantile formatting (e.g., q0.1, q0.25).

  • Duplicate entries for the same (spatial, t, α) are aggregated with aggfunc='first' by default. Adjust upstream data to avoid ambiguity if necessary.

  • The transformation is the (approximate) inverse of melt_q_data():

    (5)\[\mathbf{L}\ \xrightarrow{\ \text{pivot}\ }\ \mathbf{W} \quad\text{and}\quad \mathbf{W}\ \xrightarrow{\ \text{melt+pivot}\ }\ \mathbf{L}.\]

References

[1]

Wickham, H. (2014). Tidy Data. J. Stat. Software, 59(10).

[2]

McKinney, W. (2013). Python for Data Analysis. O’Reilly.

Examples

From long/tidy-wide to time-embedded wide:

>>> long_df = pd.DataFrame({
...   'lon': [-118.25, -118.25, -118.30],
...   'lat': [34.05, 34.05, 34.10],
...   'year': [2022, 2023, 2022],
...   'subs_q0.1': [1.2, 1.7, 1.3],
...   'subs_q0.5': [1.5, 1.9, 1.6]
... })
>>> wide_df = pivot_q_data(long_df, 'subs', dt_col='year')
>>> sorted(c for c in wide_df.columns if c.startswith('subs_'))
['subs_2022_q0.1', 'subs_2022_q0.5',
 'subs_2023_q0.1', 'subs_2023_q0.5']

Filtering quantiles:

>>> wide_df2 = pivot_q_data(long_df, 'subs', dt_col='year',
...                         q=[0.1])
>>> sorted(c for c in wide_df2.columns if c.startswith('subs_'))
['subs_2022_q0.1', 'subs_2023_q0.1']