xai_compare.comparisons.consistency

Classes

Consistency(model, data, target[, mode, ...])

A class to evaluate the consistency of different explainers on a specified model.

class xai_compare.comparisons.consistency.Consistency(model, data: DataFrame, target: DataFrame | Series | ndarray, mode: str = 'regression', random_state: int = 42, verbose: bool = False, n_splits: int = 5, use_stratified_folds: bool = False, shuffle: bool = False, default_explainers: List[str] = ['shap', 'lime', 'permutations'], custom_explainer: Type[Explainer] | List[Type[Explainer]] | None = None)

A class to evaluate the consistency of different explainers on a specified model.

Attributes:

model (Model):

The machine learning model to be evaluated.

data (pd.DataFrame):

The feature dataset used for model training and explanation.

target (Union[pd.DataFrame, pd.Series, np.ndarray]):

The target variables associated with the data.

mode (str):

The mode of operation, either ‘REGRESSION’ or ‘CLASSIFICATION’.

random_state (int):

Seed used by the random number generator.

verbose (bool):

If True, prints additional information during the function’s execution.

n_splits (int):

The number of splits for cross-validation.

use_stratified_folds (bool):

If True, uses stratified folds for cross-validation.

shuffle (bool):

If True, shuffles the data before splitting into batches.

default_explainers (List[str]):

List of default explainers to be used.

custom_explainer (Union[Type[Explainer], List[Type[Explainer]], None]):

Custom explainer(s) provided by the user.

consistency_scores_df (pd.DataFrame, optional):

DataFrame containing the summary statistics of feature impact standard deviations.

scores (Any, optional):

Placeholder for the scores obtained during evaluation.

summary (Any, optional):

Placeholder for the summary of results.

results (Any, optional):

Placeholder for the detailed results.

comparison_plot (Any, optional):

Placeholder for the comparison plot.

apply()

Applies the consistency measurement if it has not been done yet.

consistency_measurement(use_stratified_folds)

Measures the consistency of feature explanations across different folds.

Parameters:

use_stratified_folds (bool): Whether to use StratifiedKFold instead of KFold.

Returns:

DataFrame: DataFrame containing summary statistics of feature impact standard deviations.

create_list_explainers(custom_explainer: Type[Explainer] | List[Type[Explainer]] | None) → List[Explainer]

Creates a list of explainer classes from default and custom explainers.

Attributes:

custom_explainer (Union[Type[Explainer], List[Type[Explainer]], None]):

Custom explainer or a list of custom explainer classes.

Returns:

List[Explainer]:

A list of initialized explainer classes.

display()

Displays the consistency measurement results.

visualize_consistency()

Visualizes the mean and standard deviation of feature impacts for different explainers.