shap_enhanced.explainers.TimeSHAP

TimeSHAP Explainer: Pruning-Enhanced SHAP for Sequential Models

Theoretical Explanation

TimeSHAP is a SHAP-style feature attribution method specifically designed for sequential, temporal, or event-based models. It builds upon the principles of KernelSHAP but introduces pruning to handle the combinatorial complexity of time-series input structures.

Rather than sampling all possible feature–timestep or event subsets, TimeSHAP first performs a rough importance scan, then prunes the space down to the top-k most relevant events, windows, or timesteps. This makes it scalable to long sequences while maintaining fidelity.

Key Concepts

  • Pruned Coalition Sampling:

    Performs an initial round of random sampling to estimate rough feature/event importance. Only the top-k units are retained for precise SHAP estimation.

  • Event/Window Attribution:
    Supports attribution across:

    • Individual timesteps (fine-grained),

    • Features (vertical slices),

    • Event windows (e.g., rolling sequences).

  • Flexible Masking:
    Masked features can be:

    • Set to zero (hard masking), or

    • Replaced with the mean from background data (soft masking).

  • Additivity Normalization:

    Final SHAP attributions are normalized so that their total equals the model output difference between the original and fully-masked inputs.

Algorithm

  1. Initialization:
    • Accepts a target model, background data, event window size, masking strategy, pruning parameter (e.g., top-k),

      and device context.

  2. Rough Importance Estimation:
    • For each unit (feature, timestep, or window):

      • Sample random coalitions excluding the unit.

      • Compute model outputs with and without the unit masked.

      • Estimate marginal contribution based on output difference.

  3. Pruning:
    • If pruning is enabled:

      • Retain only the top-k most important units from the rough scan.

      • Discard lower-importance units from further evaluation.

  4. Refined Attribution:
    • For each selected unit:

      • Sample coalitions and compute more precise SHAP values.

      • Assign contributions to the appropriate location in the attribution map

        (e.g., timestep, feature, or window).

  5. Normalization:
    • Rescale all SHAP values so that their sum equals the difference between

      the model prediction on the original and fully-masked input.

Use Case

TimeSHAP is ideal for:

  • Event sequences, medical time-series, or log data.

  • Models where full SHAP computation is infeasible due to input length.

  • Explaining model behavior over time, including “when” and “what” drove a prediction.

References

  • Lundberg & Lee (2017), “A Unified Approach to Interpreting Model Predictions” [SHAP foundation—coalitional feature attribution framework]

  • Bento et al. (2021), “TimeSHAP: Explaining Recurrent Models through Sequence Perturbations” [Introduces pruning-enhanced KernelSHAP for sequence models; supports feature-, event-, and cell-level attribution in long sequences] :contentReference[oaicite:1]{index=1}

  • Nayebi et al. (2022), “WindowSHAP: An Efficient Framework for Explaining Time‑Series Classifiers based on Shapley Values” [Partitions sequences into windows to reduce computation—related strategy to manage long temporal inputs] :contentReference[oaicite:2]{index=2}

Classes

TimeSHAPExplainer(model, background[, ...])

TimeSHAPExplainer: Pruned SHAP Attribution for Sequential Models
class shap_enhanced.explainers.TimeSHAP.TimeSHAPExplainer(model, background, mask_strategy='mean', event_window=None, prune_topk=None, device=None)[source]

Bases: BaseExplainer

TimeSHAPExplainer: Pruned SHAP Attribution for Sequential Models

Implements a SHAP-style explainer for time-series and sequential data using pruning to efficiently estimate per-(t, f) or event-level attributions.

Combines:

  • Masking strategy (zero or mean-based).

  • Optional event windowing (for segment-level attribution).

  • Top-k pruning to reduce the coalition space before final SHAP estimation.

Parameters:

  • model (Any) – The model to be explained.

  • background (np.ndarray or torch.Tensor) – Background dataset for imputation and mean estimation.

  • mask_strategy (str) – Masking method, either ‘zero’ or ‘mean’.

  • event_window (int or None) – Optional window size for event-based attribution.

  • prune_topk (int or None) – If specified, retain only top-k units (based on rough attribution) for refinement.

  • device (str) – Computation device (‘cpu’ or ‘cuda’).

property expected_value

Optional property returning the expected model output on the background dataset.

Returns:

Expected value if defined by the subclass, else None.

Return type:

float or None

explain(X, **kwargs)

Alias to shap_values for flexibility and API compatibility.

Parameters:

  • X (Union[np.ndarray, torch.Tensor, list]) – Input samples to explain.

  • kwargs – Additional arguments.

Returns:

SHAP values.

Return type:

Union[np.ndarray, list]

shap_values(X, nsamples=100, level='timestep', check_additivity=True, random_seed=42, **kwargs)[source]

Compute SHAP values for sequential input with optional pruning and window-based attribution.

Note

Pruned estimation uses an initial coarse pass to identify important units (features, timesteps, or windows), followed by refined SHAP estimation over that subset.

Parameters:

  • X (Union[np.ndarray, torch.Tensor]) – Input tensor or array of shape (T, F) or (B, T, F).

  • nsamples (int) – Number of coalitions to sample per unit.

  • level (str) – Attribution level: ‘timestep’, ‘feature’, or ‘event’.

  • check_additivity (bool) – If True, print additivity diagnostics.

  • random_seed (int) – Random seed for reproducibility.

Returns:

SHAP values with shape (T, F) or (B, T, F).

Return type:

np.ndarray