Fact-checked by Grok 2 weeks ago

Explainable artificial intelligence

Explainable artificial intelligence (XAI) is a subfield of artificial intelligence that develops techniques to render the predictions, decisions, and internal workings of AI models comprehensible to human users, countering the opacity inherent in complex systems like deep neural networks.^[1]^[2] This pursuit arises from the trade-off in modern machine learning between predictive accuracy and interpretability, where "black-box" models excel in performance but obscure causal mechanisms, complicating trust, debugging, and deployment in high-stakes domains such as medicine, finance, and autonomous systems.^[3]^[4] Prominent approaches encompass intrinsically interpretable models (e.g., linear regressions or decision trees that expose decision rules directly) and post-hoc explanation methods (e.g., feature attribution techniques like SHAP values, which quantify input contributions to outputs, or local surrogates like LIME that approximate model behavior around specific instances).^[5]^[6] Achievements include enhanced regulatory adherence under frameworks like the EU AI Act, improved model robustness through interpretability-driven refinements, and empirical validations in sectors like healthcare where XAI aids clinicians in verifying diagnostic rationales.^[7]^[8] Yet controversies endure: critics argue many XAI tools yield superficial or misleading proxies rather than genuine causal insights into model reasoning, potentially fostering overconfidence in flawed systems, while debates rage over whether scalable explanations for nonlinear deep learning are fundamentally unattainable without sacrificing performance.^[9]^[10]

Definitions and Fundamentals

Core Concepts and Distinctions

Explainable artificial intelligence (XAI) encompasses techniques designed to elucidate the decision-making processes of machine learning models, addressing the opacity inherent in many high-performance algorithms. Central to XAI are distinctions between model types and explanation scopes, which inform the choice of interpretability methods. Black-box models, such as deep neural networks, exhibit complex internal structures where input-output mappings are not directly observable, limiting human comprehension of causal pathways.^[11] In contrast, white-box models, including linear regression or decision trees, feature transparent architectures that allow direct inspection of feature contributions and decision rules.^[12] This dichotomy highlights a performance trade-off: black-box models often achieve superior predictive accuracy on intricate datasets, while white-box models prioritize inherent understandability at potential cost to precision.^[13] Explanations in XAI further divide into intrinsic (ante-hoc) and post-hoc categories. Intrinsic explanations arise from models designed for interpretability from inception, where the algorithm's logic—such as rule-based splits in decision trees—naturally reveals feature importance and prediction rationale without additional processing.^[14] Post-hoc explanations, conversely, apply to trained models regardless of complexity, generating approximations or surrogates to probe behavior; examples include feature perturbation methods like LIME, which localize explanations around specific instances.^[15] Post-hoc approaches enable flexibility for black-box systems but risk fidelity issues, as surrogate models may not perfectly capture the original's nuances.^[14] Explanations also vary by scope: local versus global. Local explanations target individual predictions, attributing outcomes to feature values for a single input, as in SHAP values that decompose a prediction's deviation from baseline.^[16] Global explanations, by comparison, aggregate insights across the dataset to describe overall model tendencies, such as average feature impacts or decision boundaries, aiding in bias detection or generalization assessment.^[14] These scopes are not mutually exclusive; hybrid methods increasingly combine them for comprehensive diagnostics.^[17] Overlapping terms like transparency, interpretability, and explainability lack universal formalization, complicating XAI discourse. Transparency typically denotes openness of model components and data flows, interpretability the ease of discerning decision causes, and explainability the provision of human-readable rationales—yet usages vary across literature, with explainability often encompassing post-hoc tools for non-interpretable systems.^[7] This conceptual fluidity underscores the field's emphasis on context-specific utility over rigid taxonomy.^[18]

Taxonomy of Explainability Approaches

Explainability approaches in artificial intelligence are classified along multiple dimensions to capture their design, applicability, and output characteristics, as surveyed in recent literature. A core distinction lies between intrinsic (ante-hoc) methods, which employ models designed to be interpretable from the outset—such as linear models, decision trees, or rule-based systems—and post-hoc methods, which generate explanations for opaque "black-box" models after training, including techniques like surrogate models or attribution methods.^[19] ^[20] This dichotomy addresses the trade-off between model performance and transparency, with intrinsic approaches prioritizing simplicity at potential cost to accuracy on complex tasks.^[21] Another fundamental axis is scope: local explanations focus on individual predictions or instances, elucidating why a specific input yields a particular output (e.g., via Local Interpretable Model-agnostic Explanations (LIME), which approximates a black-box locally with a simple model), whereas global explanations describe the model's overall behavior across the input space, such as through feature importance rankings or partial dependence plots.^[19] ^[20] Local methods dominate for debugging single cases, as evidenced by their prevalence in applications like medical diagnostics, while global methods aid in auditing systemic biases or regulatory compliance.^[22] Methods are further differentiated by applicability: model-specific techniques leverage the internal structure of particular architectures, such as Layer-wise Relevance Propagation (LRP) for neural networks, which decomposes predictions via backpropagation of relevance scores, or saliency maps that highlight gradient-based sensitivities in convolutional layers.^[20] In contrast, model-agnostic approaches, like SHapley Additive exPlanations (SHAP), apply universally by treating models as oracles and using game-theoretic values to assign feature contributions, enabling portability across algorithms but often at higher computational expense.^[19] ^[22] Taxonomies also categorize by methodology or functioning, encompassing perturbation-based techniques that probe inputs (e.g., LIME's sampling around instances or counterfactual generation, which identifies minimal changes to alter outcomes), gradient-based methods reliant on differentiability (e.g., Integrated Gradients, which accumulate gradients along a baseline-to-input path for stable attributions), and others like attention mechanisms in transformers or example-based retrievals.^[20] Output forms vary correspondingly, from visualizations (heatmaps, decision paths) to textual rules or prototypes, with selection guided by domain needs—e.g., rule extraction for legal interpretability.^[19] These dimensions often intersect, yielding hybrid classifications; for instance, SHAP can be local and post-hoc yet adaptable globally via kernel approximations. Challenges in unification persist due to overlapping terms and context-dependent validity, as no single taxonomy fully resolves ambiguities like the fidelity-interpretability trade-off, prompting ongoing refinements in surveys up to 2024.^[20] Empirical validation remains sparse, with many methods evaluated via proxy metrics rather than real-world causal impacts.^[22]

Motivations and Objectives

Technical and Practical Drivers

Technical drivers for explainable artificial intelligence (XAI) primarily stem from the need to diagnose and enhance the internal workings of complex models, particularly black-box systems like deep neural networks, where opacity hinders identification of errors or inefficiencies. Explanations enable developers to pinpoint failure modes, such as reliance on spurious correlations in training data, facilitating targeted debugging that improves generalization and robustness. For instance, XAI techniques like feature attribution methods reveal how models weigh inputs, allowing iterative refinements that address biases or overfitting without retraining from scratch.^[23]^[24] Empirical evidence underscores these benefits: in controlled studies, integrating XAI into model development pipelines has yielded accuracy gains of 15% to 30% by exposing and mitigating flawed decision pathways, as observed in platforms designed for iterative AI refinement. Moreover, XAI supports performance optimization by quantifying the impact of hyperparameters or architectural changes on predictions, bridging the gap between high-level metrics like accuracy and causal mechanisms underlying model behavior. This is particularly vital for supervised learning tasks, where transparency aids in validating assumptions about data distributions and prevents degradation in deployment scenarios differing from training environments.^[23]^[3] Practical drivers arise from deployment imperatives in regulated or high-stakes domains, where unexplained decisions impede accountability and integration with human oversight. In industries like finance and healthcare, XAI ensures traceability for auditing loan approvals or diagnostic recommendations, reducing liability risks by clarifying AI contributions to outcomes. Regulatory frameworks amplify this: the European Union's AI Act, effective from August 2024 with phased enforcement through 2027, mandates transparency and explainability for high-risk systems, requiring providers to disclose decision logic to avoid prohibited opacity in areas like credit scoring or medical devices.^[25]^[26] Beyond compliance, practical adoption addresses end-user trust and operational efficiency; for autonomous driving, XAI elucidates real-time object detection rationales, enabling engineers to intervene in edge cases and regulators to verify safety claims. Industry reports highlight that without explanations, AI deployment stalls due to skepticism from stakeholders, whereas interpretable outputs foster adoption by aligning machine reasoning with verifiable human intuition, as seen in cybersecurity applications where XAI unpacks intrusion detection to preempt false positives. These drivers collectively prioritize causal insight over mere predictive power, ensuring AI systems scale reliably in production environments.^[27]^[28]^[29]

Ethical and Societal Rationales

The push for explainable artificial intelligence (XAI) stems from ethical imperatives to ensure accountability in AI-driven decisions, particularly where opaque "black-box" models obscure the causal pathways leading to outcomes that affect human lives. In high-stakes domains such as healthcare and criminal justice, unexplainable models hinder the ability to audit decisions for errors or unintended harms, making it challenging to hold developers, deployers, or users responsible for discriminatory or unjust results.^[30] For instance, black-box systems in predictive policing or loan approvals have been empirically linked to perpetuating societal biases embedded in training data, as decisions cannot be readily traced to specific inputs or algorithmic logic, exacerbating inequalities without recourse for affected individuals.^[31] XAI techniques, by contrast, facilitate post-hoc scrutiny to identify and mitigate such biases, aligning AI outputs more closely with ethical standards of fairness and non-discrimination.^[32] Societally, the opacity of advanced AI models erodes public trust, as users and regulators lack verifiable insight into how systems process data or prioritize factors, fostering skepticism toward widespread adoption in critical infrastructure like autonomous vehicles or medical diagnostics. Empirical studies indicate that explainability enhances perceived trustworthiness by allowing stakeholders to validate decision rationales against real-world expectations, thereby supporting broader societal acceptance and reducing risks of misuse or over-reliance on unverified predictions.^[32] This is particularly salient in regulatory contexts, where transparent AI enables oversight bodies to enforce compliance with legal norms, such as detecting unfair data representations that under- or over-represent demographic groups, which could otherwise amplify minority biases at scale.^[7] However, while XAI promotes these goals, it does not inherently guarantee fairness, as interpretable models can still encode biased logic if not rigorously vetted, underscoring the need for complementary empirical validation beyond mere transparency.^[33] From a first-principles perspective, ethical rationales for XAI emphasize causal realism: understanding the mechanistic "why" behind predictions counters the pitfalls of correlational black-box outputs, which may mimic intelligence without genuine alignment to human values or verifiable causality. This is evidenced in frameworks advocating XAI integration throughout the AI lifecycle to embed responsibility, where explainability tools aid in auditing for ethical alignment, such as ensuring decisions in resource allocation prioritize equitable outcomes over opaque efficiency gains.^[34] Societally, such approaches mitigate risks of democratic erosion, as unexplainable AI in governance or policy advising could entrench power imbalances by shielding influential actors from scrutiny, whereas explainable variants empower informed public discourse and policy calibration based on auditable evidence.^[35] Overall, these rationales drive XAI not as a panacea but as a necessary safeguard against the societal costs of deploying powerful yet inscrutable systems, with ongoing research quantifying improvements in accountability metrics like bias detection rates in controlled deployments.^[32]

Relation to AI Safety and Reliability

Explainable artificial intelligence (XAI) contributes to AI safety by enabling the detection of biases, failures, and unintended behaviors in machine learning models, allowing developers to audit decision-making processes and mitigate risks before deployment.^[36] For instance, XAI techniques facilitate the identification of model vulnerabilities, such as discriminatory patterns in predictive algorithms, which could otherwise lead to harmful outcomes in high-stakes applications like healthcare or autonomous systems.^[37] This transparency supports proactive safety measures, including the validation of model fairness and the correction of erroneous predictions, thereby reducing the potential for systemic errors or adversarial exploits.^[3] In the context of AI alignment—ensuring systems pursue intended objectives without deviation—XAI, particularly through mechanistic interpretability, provides insights into internal representations and causal pathways within neural networks, aiding efforts to verify goal-directed behavior.^[38] Researchers argue that such interpretability is essential for scaling oversight of advanced models, as it allows humans to probe for misaligned incentives or emergent capabilities that opaque "black-box" systems obscure.^[39] However, limitations exist; interpretability methods may fail to reliably detect sophisticated deception in trained models, where deceptive alignments could evade superficial explanations, underscoring that XAI is a necessary but insufficient tool for comprehensive safety guarantees.^[40] Regarding reliability, XAI enhances system dependability by supporting debugging and empirical validation of model robustness against distributional shifts or adversarial inputs, fostering verifiable performance in real-world scenarios.^[41] Techniques like post-hoc explanations and surrogate models enable stakeholders to assess consistency and generalize predictions, which is critical for domains requiring high assurance, such as safety-critical engineering.^[41] Empirical studies demonstrate that integrating XAI improves fault detection rates, with interpretable components reducing downtime in deployed systems by clarifying failure modes.^[42] Despite these benefits, over-reliance on explanations risks a false sense of security if metrics for explainability lack rigorous grounding, potentially masking underlying unreliability in complex models.

Historical Evolution

Pre-2010 Foundations in Interpretable Machine Learning

The foundations of interpretable machine learning prior to 2010 were rooted in symbolic artificial intelligence and statistical modeling traditions that emphasized transparency through explicit rules and simple structures. Expert systems, prominent from the 1970s to the 1980s, relied on human-engineered knowledge bases of production rules and logical inference, enabling explanations via traces of reasoning steps, such as forward or backward chaining.^[43] A seminal example was MYCIN, developed in the 1970s and formalized in 1984, which diagnosed bacterial infections using approximately 450 rules and provided justifications for recommendations by citing evidential rules and confidence factors. These systems prioritized comprehensibility for domain experts, though they suffered from knowledge acquisition bottlenecks and limited scalability to complex, data-driven domains.^[43] In parallel, statistical machine learning advanced inherently interpretable models like linear regression and generalized linear models, where parameter coefficients directly quantified feature contributions to predictions, facilitating causal and predictive insights since the early 20th century but gaining ML prominence in the 1980s.^[44] Decision trees emerged as a cornerstone for classification and regression tasks, offering visual tree structures that traced decision paths from root to leaf nodes, thus providing global interpretability. Leo Breiman and colleagues introduced Classification and Regression Trees (CART) in 1984, employing recursive partitioning with Gini impurity or mean squared error criteria to build trees amenable to pruning for generalization and explanation.^[45] J. Ross Quinlan's ID3 algorithm (1986) and subsequent C4.5 (1993) further refined this by using information gain from entropy to select splits, enabling rule extraction from trees for propositional logic representations. These methods balanced predictive accuracy with human-readable hierarchies, influencing applications in fields like medicine and finance where decision rationale was essential.^[44] As neural networks gained traction in the late 1980s and 1990s following backpropagation's popularization, their black-box nature prompted early post-hoc interpretability efforts to approximate or decompose complex models. Techniques included sensitivity analysis, which measured output changes to input perturbations, and visualization of hidden unit activations to infer learned representations.^[43] Rule extraction methods treated neural networks as oracles, distilling them into surrogate decision trees or lists; for instance, Andrews et al. (1995) proposed decompositional and pedagogical approaches to derive symbolic rules from trained connectionist systems, evaluating fidelity via accuracy preservation. Craven and Shavlik's Trepan (1996) extended this by querying neural networks to induce oblique decision trees, prioritizing fidelity to the original model over pedagogical simplicity. These foundations underscored a trade-off between model complexity and interpretability, favoring simpler, transparent alternatives unless post-hoc surrogates could reliably bridge the gap, as evidenced in domains requiring regulatory compliance or error tracing.^[44]

2010s Revival and DARPA's Role

The resurgence of interest in explainable artificial intelligence during the 2010s was driven by the rapid adoption of deep learning models, which achieved state-of-the-art performance in tasks such as image recognition and natural language processing but operated as opaque "black boxes," complicating trust and accountability in high-stakes applications like autonomous systems and decision support.^[46]^[47] This shift contrasted with earlier emphases on inherently interpretable models, as the predictive power of neural networks—exemplified by AlexNet's 2012 ImageNet victory with an error rate of 15.3% versus prior bests over 25%—prioritized accuracy over transparency, prompting renewed focus on methods to elucidate model internals without sacrificing capability. Early 2010s publications, such as those exploring feature visualization in convolutional networks, laid groundwork, but systematic efforts coalesced mid-decade amid growing deployment in defense and healthcare domains where erroneous decisions could yield catastrophic outcomes. The U.S. Defense Advanced Research Projects Agency (DARPA) catalyzed this revival through its Explainable Artificial Intelligence (XAI) program, formulated in 2015 to develop techniques enabling humans to comprehend, trust, and effectively manage AI outputs in operational contexts.^[48]^[49] Launched with initial funding announcements in 2016 and broader solicitations by 2017, the program allocated approximately $50 million across basic research, applied development, and evaluation thrust areas, targeting both local explanations (e.g., for individual predictions) and global model behaviors.^[46]^[50] DARPA program manager David Gunning emphasized creating "glass box" models compatible with human-in-the-loop oversight, particularly for military applications like tactical decision aids, where unexplained AI recommendations risked mission failure or ethical lapses.^[51] DARPA's XAI initiative influenced broader academia and industry by funding over 20 performers, including universities and firms like Boeing and Raytheon, to prototype tools such as scalable visualizations and causal inference hybrids that preserved deep learning performance—e.g., achieving explanation fidelity scores above 90% in benchmark tests—while advancing standards for user-centric validation.^[49]^[52] Retrospective analyses credit the program with shifting XAI from ad-hoc techniques to rigorous engineering, though challenges persisted in scaling explanations for non-expert end-users and verifying causal validity beyond correlative patterns.^[48] By program's end around 2021, it had spurred open-source libraries and interdisciplinary collaborations, embedding explainability as a core requirement in subsequent AI governance frameworks.^[53]

2020s Developments and Integration with Deep Learning

The 2020s marked a pivotal shift in explainable artificial intelligence (XAI) toward deeper integration with deep learning architectures, driven by the dominance of transformer-based models in large language models (LLMs). Researchers increasingly focused on mechanistic interpretability, aiming to reverse-engineer internal computations to uncover causal mechanisms rather than relying solely on post-hoc approximations. This approach treats neural networks as interpretable circuits, enabling precise interventions and debugging.^[54] A foundational effort was the 2022 Transformer Circuits project, which identified modular components like induction heads in attention layers, responsible for in-context learning patterns.^[54] Key advancements included the study of grokking, a phenomenon where overparameterized models abruptly transition from memorization to generalization after prolonged training on small datasets. Observed in modular addition tasks, grokking revealed discrete phases in deep learning optimization, informing interpretability by highlighting how circuits form gradually before sudden performance leaps. This integration extended to sparse autoencoders (SAEs), applied from 2023 onward to decompose activations into human-interpretable features, such as monosemantic concepts in LLMs, mitigating superposition where neurons encode multiple abstract features. Anthropic's 2023 dictionary learning techniques scaled SAEs to billion-parameter models, extracting thousands of interpretable directions aligned with topics like safety or deception. Further developments emphasized hybrid methods combining local explanations with global circuit analysis. For instance, automated interpretability pipelines in 2024 used causal tracing to verify feature contributions across layers, enhancing fidelity in transformer explanations.^[55] These techniques addressed deep learning's opacity by enabling scalable interventions, such as editing specific circuits to alter model behavior without retraining. Despite progress, challenges persist in scaling to frontier models, where computational costs for circuit discovery grow superlinearly, prompting ongoing research into efficient approximation methods.^[55] Regulatory pressures, including the EU AI Act's requirements for high-risk systems effective from 2024, accelerated practical integrations of these tools in deployed deep learning applications.

Core Techniques

Inherently Interpretable Models

Inherently interpretable models, also termed intrinsically interpretable or white-box models, are machine learning algorithms designed such that their internal structure and prediction mechanisms are directly understandable by humans, obviating the need for post-hoc explanation tools applied to opaque systems.^[56]^[57] These models achieve transparency through properties like simulatability, where users can mentally replicate decisions in limited time, and decomposability, enabling intuitive grasp of inputs, parameters, and outputs.^[57] Unlike black-box models such as deep neural networks, which require surrogate explanations, inherently interpretable models embed comprehensibility in their architecture from the outset.^[58] Classic examples include linear and logistic regression, where feature coefficients quantify the magnitude and direction of each variable's influence on outcomes, allowing direct assessment of importance and causality assumptions under linearity.^[56] Decision trees, particularly shallow or optimal variants like Optimal Classification Trees (OCTs), represent decisions as hierarchical if-then rules tracing paths from root to leaf nodes, with splits based on feature thresholds that users can inspect for logical consistency.^[58] Naive Bayes classifiers offer probabilistic interpretations via conditional independence assumptions, decomposing predictions into feature likelihoods.^[56] These models suit domains demanding accountability, as predictions can be audited without computational intermediaries.^[57] More advanced variants extend interpretability to nonlinear data while preserving transparency. Generalized additive models (GAMs) decompose predictions into additive sums of univariate nonlinear functions per feature, visualized as shape plots to reveal interactions without full additivity violations.^[57] Supersparse linear integer models (SLIMs) enforce integer coefficients and sparsity for concise, rule-like expressions, as in medical risk scoring where few terms dominate.^[57] Falling rule lists (FRLs) generate monotonic sequences of if-then rules, prioritizing higher-risk conditions first for ordinal outcomes like disease severity.^[57] Such extensions balance expressiveness with human oversight, though they impose constraints like sparsity or monotonicity to maintain comprehensibility.^[57] Despite advantages in trust-building and regulatory compliance—evident in healthcare applications where OCTs achieve area under the curve (AUC) values of 0.638–0.675 for cancer prognostication, rivaling complex models like XGBoost (AUC 0.654–0.690)—these models often trade predictive power for simplicity, underperforming on intricate, high-dimensional datasets with nonlinearities or interactions.^[58] Evaluations highlight fidelity via functional-grounded metrics (e.g., matching oracle predictions) but reveal challenges in universal definitions and human-grounded assessments, where perceived utility varies by expertise.^[57] In practice, selection favors them when accuracy thresholds permit, prioritizing causal insight over marginal gains in opaque alternatives.^[56]^[58]

Post-Hoc Local Explanation Methods

Post-hoc local explanation methods generate instance-level interpretations for black-box machine learning models after training, focusing on approximating the model's decision boundary near a specific input prediction without altering the model's architecture or parameters. These approaches prioritize locality by emphasizing explanations valid in the neighborhood of the instance, enabling users to understand why a particular output was produced for that case, which is particularly useful for high-stakes domains requiring per-prediction accountability. Unlike global methods, they trade broader model insights for detailed, context-specific rationales, often using surrogate approximations that balance interpretability and fidelity to the original prediction.^[59] A foundational technique is Local Interpretable Model-agnostic Explanations (LIME), introduced by Ribeiro, Singh, and Guestrin in 2016. LIME operates by perturbing the input instance to create a dataset of synthetic samples, querying the black-box model for predictions on these perturbations, and then fitting a simple interpretable surrogate model—typically linear regression—weighted by proximity to the original instance to ensure local fidelity. The resulting feature weights indicate contributions to the prediction, visualized as bar charts or heatmaps for tabular, text, or image data. This model-agnostic method applies to classifiers like random forests or neural networks, with empirical evaluations on datasets such as those from the UCI repository showing it approximates predictions within 5-10% error locally in many cases.^[60] SHapley Additive exPlanations (SHAP), proposed by Lundberg and Lee in 2017, extends cooperative game theory's Shapley values to attribute prediction outcomes additively to input features. For a given instance, SHAP computes exact or approximate marginal contributions of each feature by considering all possible coalitions of features, marginalizing over the model's behavior, and ensures consistency properties like efficiency (attributions sum to the prediction) and local accuracy (explaining deviations from expected output). Kernel SHAP approximates these values efficiently via weighted linear regression on sampled coalitions, while TreeSHAP leverages decision tree structures for exact computation in polynomial time. Evaluations on benchmarks like ImageNet subsets demonstrate SHAP's attributions correlate strongly with human-annotated importance, outperforming LIME in consistency across perturbations by up to 20% in some ablation studies. Other variants include permutation-based methods like feature permutation importance localized via repeated sampling around the instance, which measures prediction degradation upon feature shuffling while preserving correlations, though they risk confounding effects in high-dimensional spaces. Counterfactual local explanations generate minimal input changes yielding alternative predictions, optimized via gradient descent or genetic algorithms to highlight decision boundaries, with studies on loan approval models showing they reveal actionable insights missed by additive methods. These techniques share advantages in flexibility across model types but face challenges: LIME's explanations can vary unstably with sampling seeds (up to 15% variance in feature rankings per 2019 robustness analyses), SHAP's exact computation scales exponentially with features (mitigated by approximations introducing bias), and both may overemphasize spurious correlations if perturbations inadequately capture the model's inductive biases. Validation often relies on metrics like local accuracy (prediction match) and stability (consistency under noise), with comparative benchmarks indicating SHAP generally achieves higher faithfulness at greater computational expense—e.g., 10-100x slower than LIME for deep networks.^[61]^[62]^[63]

Post-Hoc Global Explanation Methods

Post-hoc global explanation methods apply interpretive techniques to already-trained machine learning models, focusing on their overall predictive patterns across an entire dataset rather than individual instances. These model-agnostic approaches generate approximations or visualizations that reveal aggregate feature influences and decision boundaries without modifying the original black-box predictor, enabling stakeholders to understand systemic behaviors such as dominant feature interactions or bias patterns. Unlike local methods, which probe specific predictions, global methods prioritize comprehensiveness, though they risk oversimplification if the black-box exhibits high non-linearity or heterogeneity.^[64]^[65] Global surrogate models represent a core technique, wherein an interpretable proxy—such as linear regression, decision trees, or rule-based systems—is trained to replicate the black-box model's outputs using the same input features and target predictions. Fidelity is quantified through metrics like mean squared error or accuracy on held-out data, with higher surrogate performance indicating reliable insights into the black-box's logic; for instance, a decision tree surrogate might yield hierarchical feature rules mirroring the complex model's priorities. This method, applicable to any black-box, traces origins to early efforts in approximating neural networks but gained prominence in XAI for its balance of transparency and scalability, as evidenced in benchmarks where tree surrogates achieved over 90% fidelity on tabular datasets. Limitations include potential loss of subtle interactions if the surrogate class is overly simplistic, prompting hybrid selections based on domain knowledge.^[64]^[66] Permutation feature importance provides another post-hoc global metric, evaluating each feature's aggregate contribution by randomly shuffling its values in the validation set and measuring the resulting degradation in model performance, such as increased out-of-bag error or AUC drop. Features causing the largest error spikes rank highest in importance, offering a baseline-agnostic view independent of model internals; Breiman originally applied this in random forests in 2001, but it extends post-hoc via implementations in libraries like scikit-learn, where it has been validated on datasets like UCI benchmarks to identify spurious correlations missed by embedded methods. Critics note sensitivity to dataset noise and multicollinearity, which can inflate or deflate scores, necessitating multiple permutations—typically 10–100—for stability. Partial dependence plots (PDPs) visualize the marginal effect of one or two features on predictions by averaging the model's output over all other features' distributions, effectively isolating average trends while assuming feature independence. Introduced by Friedman in 2001 for tree ensembles, PDPs extend post-hoc to any model and reveal non-linear relationships, such as monotonic increases or thresholds; for example, in credit risk models, a PDP might show loan approval probability plateauing beyond income levels of $100,000. Individual conditional expectation (ICE) plots extend this by plotting per-instance curves, allowing detection of heterogeneous effects when aggregated into fan-like visuals. Both techniques, implemented in tools like scikit-learn since 2010, falter with strongly correlated features, leading to extrapolated artifacts, as demonstrated in simulations where PDPs misrepresented interactions by up to 20% in high-dimensional data. Accumulated local effects (ALE) plots mitigate this by conditioning on local neighborhoods, preserving correlation handling while maintaining global scope.^[67]^[68] Prototypes and counterfactuals can aggregate globally by clustering data into representative exemplars or generating high-level rules from perturbation analyses, though these often blend local insights; for instance, SHAP values, derived from game-theoretic axioms, can summarize into global importance rankings via mean absolute values across instances, correlating strongly with permutation scores in empirical tests on ImageNet subsets (r > 0.8). Validation remains challenging, with studies showing surrogate fidelity dropping below 70% for deep neural networks on image tasks due to distributional shifts, underscoring the need for domain-specific benchmarks.^[69]

Emerging Hybrid and Causal Approaches

Hybrid approaches in explainable artificial intelligence (XAI) integrate elements of inherently interpretable models, such as decision trees or linear regressions, with high-performance black-box models like deep neural networks to achieve a balance between predictive accuracy and human-understandable explanations. This strategy addresses the limitations of purely interpretable models, which often sacrifice performance on complex tasks, by leveraging the strengths of opaque models while approximating their decisions through transparent proxies or distillation techniques. For instance, a 2020 study proposed a hybrid framework that distills explanations from deep learning predictions into rule-based forms, enabling post-hoc interpretability without retraining the core model.^[70] Recent advancements, documented in 2024 reviews, classify these hybrids by interpretability focus, such as local versus global explanations, and highlight applications in domains requiring regulatory compliance, where black-box accuracy is augmented by symbolic reasoning layers.^[71]^[72] Causal approaches emphasize modeling cause-and-effect relationships to provide explanations grounded in interventions and counterfactuals, moving beyond correlational feature attributions common in traditional XAI methods. Drawing from Judea Pearl's causal hierarchy—which distinguishes association, intervention, and counterfactual reasoning—these methods construct directed acyclic graphs (DAGs) or structural causal models (SCMs) to infer how changes in inputs would affect outcomes, offering verifiable insights into model behavior under hypothetical scenarios. A 2023 analysis of over 100 studies found that causality enhances XAI by enabling robust explanations resilient to confounding variables, with applications in bias detection and policy simulation.^[73] For example, counterfactual explanations generate minimal input perturbations that flip predictions, quantifying causal contributions more reliably than saliency maps, as validated in controlled experiments on tabular and image data.^[74] Emerging hybrid causal frameworks combine these paradigms to yield "truly explainable" systems that maintain high fidelity to causal structures while scaling to large datasets. In 2025, the Holistic-XAI (H-XAI) framework integrated causal rating mechanisms—assessing intervention effects via do-calculus—with feature attribution tools like SHAP, demonstrating improved explanation stability in dynamic environments such as healthcare diagnostics.^[75] Neuro-symbolic hybrids further blend neural networks for pattern recognition with symbolic causal engines for logical inference, as explored in 2025 prototypes for agent-based decision-making, where causal graphs constrain neural outputs to ensure interventions align with real-world mechanics.^[76] These developments, often tested on benchmarks like causal discovery tasks from the IHDP dataset, report up to 20% gains in counterfactual accuracy over non-causal baselines, underscoring their potential for reliable AI deployment in high-stakes settings.^[77] However, challenges persist in automating causal discovery from observational data, where assumptions like Markov faithfulness must be empirically validated to avoid spurious inferences.^[78]

Evaluation and Validation

Metrics for Explanation Fidelity and Comprehensibility

Explanation fidelity metrics evaluate the alignment between an explanation and the black-box model's actual predictions, often through perturbation tests that measure prediction changes when features are altered based on the explanation's attributions. Faithfulness, a core fidelity metric, quantifies this by assessing how well removing or masking features ranked by importance affects model output; for instance, deletion-based faithfulness computes the correlation between attribution scores and the drop in prediction confidence as high-importance features are sequentially removed.^[79] Insertion AUC, conversely, measures fidelity by progressively adding features from least to most important per the explanation and tracking rising model accuracy, with higher AUC values indicating stronger alignment.^[80] Faithfulness correlation, another perturbation metric, calculates the Pearson correlation between feature importance scores from the explanation and corresponding changes in model predictions under masking, achieving near-perfect scores on linear models but varying across complex architectures.^[81] These metrics reveal limitations, such as sensitivity to perturbation strategies; for example, ground-truth faithfulness assumes access to true model internals, which is infeasible for opaque models, while predictive faithfulness relies on proxy behaviors like output shifts.^[79] Studies using decision trees as transparent proxies have verified that metrics like faithfulness estimate and correlation yield consistent rankings of explanation methods, though they underperform on non-monotonic relationships without causal adjustments.^[81] Comprehensive reviews classify fidelity under representational metrics, emphasizing its distinction from stability, where explanations should remain consistent across similar inputs.^[82] Comprehensibility metrics assess human interpretability, prioritizing subjective and objective proxies for how easily users grasp explanations. User satisfaction and mental model accuracy, evaluated via surveys or tasks where participants predict model outputs from explanations, gauge perceived clarity; for example, comprehension tests in controlled studies score users' ability to infer feature influences correctly.^[83] Objective measures include explanation sparsity (e.g., number of highlighted features) or syntactic simplicity (e.g., rule length), which correlate with faster human processing in domains like tabular data.^[84] Datasets from user studies on XAI methods, such as LIME or SHAP visualizations, quantify comprehensibility through Likert-scale ratings of understandability and transparency, revealing domain-specific variances like higher scores for visual over textual formats in image tasks.^[85] Trade-offs persist, as high-fidelity explanations (e.g., dense attribution maps) often reduce comprehensibility due to cognitive overload, necessitating hybrid evaluations combining automated fidelity with human-centered proxies.^[86] Standardization lags, with taxonomies proposing multi-aspect frameworks including effectiveness and trust, but empirical validation shows inter-metric correlations below 0.7, underscoring the need for context-aware benchmarks.^[82]

Human-Centered Assessment Challenges

Human-centered assessments in explainable artificial intelligence (XAI) seek to measure how explanations influence human users' understanding, trust, and decision-making processes, often through empirical user studies that gauge subjective outcomes like perceived utility and mental models formed.^[87] These evaluations prioritize end-user perspectives over purely technical metrics, yet they encounter persistent difficulties in establishing reliable, objective benchmarks due to the interplay of human cognition and contextual factors.^[88] A core challenge stems from the subjectivity inherent in human judgments, where explanations' perceived fidelity and helpfulness vary widely based on users' prior knowledge, cognitive biases, and social influences, complicating consensus on what constitutes an effective explanation.^[87] Without a universal ground truth for explanations—unlike verifiable model predictions—assessors struggle to differentiate genuine insight from superficial or illusory comprehension, often relying on self-reported data prone to overconfidence or anchoring effects.^[88] Standardization remains elusive, as studies employ ad-hoc protocols and metrics (e.g., Likert-scale surveys for trust or task performance proxies for understanding), yielding incomparable results across domains and precluding meta-analyses or broad validation.^[87] This fragmentation is exacerbated by sparse incorporation of cognitive science principles, such as mental model theory or bounded rationality, which could ground evaluations but are rarely operationalized systematically.^[87] Participant diversity poses further hurdles, with many studies drawing from convenience samples like university students or AI experts, underrepresenting end-users such as clinicians, policymakers, or non-technical stakeholders whose needs differ in expertise and cognitive load tolerance.^[89] Evaluations thus often overlook variations in user backgrounds, leading to designs that fail in real-world deployment where heterogeneous groups interact with AI.^[89] The logistical demands of user studies—requiring ethical oversight, controlled experiments, and sufficient power for statistical significance—limit their scale and frequency, resulting in sparse evidence bases that hinder reproducibility and long-term tracking of explanation efficacy.^[87] Consequently, human-centered assessments risk prioritizing narrow, context-bound findings over robust, generalizable insights, potentially misguiding XAI development toward superficial transparency rather than causal or mechanistic understanding.^[88]

Benchmarks and Standardization Efforts

Benchmarks in explainable artificial intelligence (XAI) aim to provide standardized frameworks for evaluating the fidelity, robustness, and comprehensibility of explanation methods, addressing the absence of universal metrics in the field. These benchmarks typically involve synthetic or real-world datasets paired with ground-truth explanations or controlled model behaviors to test post-hoc methods like feature attribution. For instance, the M4 benchmark, introduced in 2023, unifies faithfulness evaluation across modalities such as images, text, and graphs using consistent metrics like sufficiency and comprehensiveness scores.^[90] Similarly, XAI-Units, released in 2025, employs unit-test-like evaluations on datasets with known causal mechanisms to assess feature attribution methods against diverse failure modes, revealing inconsistencies in popular techniques like SHAP and LIME.^[91] Several open-source toolkits facilitate large-scale benchmarking. BenchXAI, a comprehensive package from 2025, evaluates 15 post-hoc XAI methods on criteria including robustness to perturbations and suitability for tabular data, highlighting limitations such as sensitivity to hyperparameter choices.^[92] The BEExAI framework, proposed in 2024, enables comparisons via metrics like explanation stability and alignment with human judgments on image classification tasks.^[93] Visual XAI benchmarks often draw from curated datasets, such as the eight-domain collection covering object classification and medical imaging, which tests explanation faithfulness against perturbation-based proxies.^[94] These efforts underscore a shift toward modular, extensible platforms, though surveys note persistent gaps in toolkit interoperability and coverage of global surrogates.^[95] Standardization efforts focus on establishing principles and protocols to mitigate evaluation inconsistencies, driven by regulatory pressures for trustworthy AI. The U.S. National Institute of Standards and Technology (NIST) outlined four principles for XAI systems in 2021—explanation, meaning, validity, and soundness—to guide development and assessment, emphasizing empirical validation over subjective interpretations.^[4] In Europe, initiatives like CEN workshop agreements promote metadata standards and procedural guidelines for XAI-FAIR data practices, aiming to harmonize explainability across AI/ML pipelines.^[96] Despite these, full standardization remains elusive due to domain-specific challenges, such as varying notions of "faithfulness" in high-stakes applications, prompting calls for unified metrics in peer-reviewed benchmarks.^[3] Ongoing work, including open benchmarks like OpenXAI, seeks to enforce rigorous, reproducible evaluations to support regulatory compliance.^[97]

Key Applications

Healthcare and Biomedical Decision Support

Explainable artificial intelligence (XAI) plays a critical role in healthcare by elucidating the reasoning behind AI models used in clinical decision support systems (CDSS), where opaque predictions can undermine clinician trust and patient safety.^[98] In biomedical applications, XAI techniques such as SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations) attribute feature importance to model outputs, enabling verification of diagnostic or prognostic decisions against medical knowledge.^[99] For instance, in medical imaging analysis for cancer detection, post-hoc methods like Grad-CAM generate heatmaps highlighting regions influencing classifications, allowing radiologists to cross-check AI suggestions with visual evidence.^[100] In treatment planning and personalized medicine, XAI supports outcome prediction by revealing causal factors in patient data, such as genetic markers or comorbidities driving therapy recommendations. A 2024 study on optimizing clinical alerts used XAI to refine alert criteria in electronic health records, identifying key variables like vital signs and lab results that reduced false positives by prioritizing interpretable features over black-box performance alone.^[101] Similarly, in drug discovery and biomarker identification, XAI has been applied to omics data for ovarian cancer, where models explained predictions by linking gene expression patterns to disease progression, aiding validation of potential therapeutic targets.^[102] Empirical evidence indicates XAI enhances adoption: a review of 5 studies found that clear, relevant explanations increased clinicians' trust in AI over unexplainable models, particularly in high-stakes scenarios like sepsis prediction or surgical risk assessment.^[103] In traumatic brain injury (TBI) forecasting, a case study comparing methods deemed SHAP most stable and faithful to model behavior, while rule-based anchors provided the highest clinician comprehensibility for tabular clinical data.^[104] However, challenges persist, including ensuring explanations align with domain expertise—e.g., avoiding misleading attributions in heterogeneous biomedical datasets—and validating fidelity through clinician feedback loops.^[105] Biomedical decision support also leverages XAI for epidemic response, as seen in COVID-19 prognosis models where explanations traced predictions to symptoms and biomarkers, improving triage accuracy during the 2020-2022 pandemic.^[106] Regulatory bodies like the FDA emphasize explainability in approved AI devices, mandating transparency for high-risk uses such as cardiovascular risk stratification, though integration requires balancing model accuracy with interpretive depth.^[107] Overall, XAI mitigates risks of over-reliance on AI by empowering evidence-based overrides, though ongoing research stresses human-AI collaboration to address biases in training data from diverse populations.^[108]

Financial Risk Assessment and Compliance

In financial risk assessment, explainable artificial intelligence (XAI) techniques enable the interpretation of opaque machine learning models used for predicting credit defaults, market volatility, and operational risks, revealing feature contributions such as borrower debt-to-income ratios or macroeconomic indicators that drive predictions.^[109] For instance, SHAP (SHapley Additive exPlanations) values can quantify how specific variables, like transaction velocity in fraud models, influence risk scores, allowing analysts to trace causal pathways from inputs to outputs without relying on black-box approximations.^[110] This interpretability supports empirical validation against historical data, where studies have shown XAI-enhanced models reducing unexplained variances in credit risk forecasts by up to 20% compared to non-interpretable counterparts.^[111] Regulatory compliance in finance increasingly demands such transparency, as black-box AI decisions risk violating mandates for auditability and non-discrimination; under the EU AI Act, effective August 1, 2024, high-risk systems in creditworthiness evaluation and risk management must provide explanations of decision logic to users, with phased enforcement starting February 2025 for general obligations and August 2027 for high-risk compliance.^[112]^[113] In anti-money laundering (AML) applications, XAI elucidates flagged transactions by highlighting indicators like transfers from high-risk jurisdictions or anomalous patterns, facilitating demonstrable adherence to standards such as the U.S. Bank Secrecy Act or FATF recommendations, where unexplained alerts have historically led to regulatory fines exceeding $10 billion annually across global banks.^[114]^[115] XAI also mitigates compliance risks in algorithmic trading and stress testing, where global surrogates or counterfactual explanations justify portfolio risk allocations under frameworks like Basel III, which require institutions to articulate model assumptions for supervisory review.^[116] Empirical deployments, such as those in European banks post-2022, have integrated local explanation methods like LIME to comply with GDPR's right to explanation, reducing dispute rates in automated lending decisions by providing borrower-specific rationales tied to verifiable data points.^[117] However, while XAI enhances accountability, its effectiveness hinges on robust validation against adversarial inputs, as unaddressed biases in explanation proxies could undermine regulatory trust, with peer-reviewed analyses noting persistent gaps in global model fidelity for high-dimensional financial datasets.^[118]^[25] In public policy, explainable artificial intelligence (XAI) supports decision-making processes by providing interpretable models for policy simulation, resource allocation, and impact forecasting, enabling policymakers to audit causal pathways and mitigate unintended biases. For instance, AI-driven tools for predicting policy outcomes, such as economic stimulus effects or environmental regulation impacts, incorporate techniques like LIME or SHAP to decompose predictions into feature contributions, fostering accountability in governmental applications.^[119] ^[120] Empirical studies demonstrate that supplying explanations for AI-generated policy recommendations enhances stakeholder trust and acceptance, with one experiment showing improved attitudes toward automated government decisions when rationales were provided, though the effect varied by explanation type such as feature-based versus counterfactual.^[121] However, integrating XAI into public policy reveals trade-offs, as demands for interpretability can constrain model complexity and accuracy, potentially undermining effective governance in high-stakes scenarios like welfare distribution or crisis response. Brookings analysis highlights that while explainability counters risks of opaque AI reinforcing biases, it may conflict with policy objectives requiring nuanced, non-linear predictions, such as in adaptive fiscal planning where black-box models outperform interpretable ones in forecast precision.^[122] Moral arguments emphasize XAI's role in upholding democratic legitimacy, arguing that transparent algorithms in policy tools prevent arbitrary power exercises and align with principles of procedural justice.^[123] In social choice mechanisms, XAI addresses challenges in aggregating heterogeneous preferences for collective decisions, such as voting systems or fair resource division, by rendering algorithmic aggregators auditable to detect manipulation or inequity. Randomized voting rules enhanced with explainability, for example, use post-hoc techniques to justify probabilistic outcomes, ensuring voters comprehend how individual rankings influence final tallies and reducing perceptions of arbitrariness in multi-winner elections.^[124] Frameworks drawing from learning theory propose representative social choice models where AI aligns with diverse voter preferences through interpretable generalization bounds, applicable to policy referenda or participatory budgeting, though empirical validation remains limited to simulated environments as of 2024.^[125] These approaches prioritize causal transparency over mere correlational outputs, aiding verification of incentive compatibility in mechanisms like approval voting adaptations. Despite potential, scalability issues persist, as explaining intricate preference profiles in large electorates demands computationally efficient XAI methods without sacrificing fidelity to ground-truth utilities.

Regulatory and Policy Dimensions

Existing Frameworks and Mandates

The European Union's Artificial Intelligence Act (Regulation (EU) 2024/1689), published on July 12, 2024, and entering into force on August 1, 2024, establishes the world's first comprehensive binding regulatory framework for AI, with phased applicability starting February 2, 2025, and full enforcement by August 2, 2027. It adopts a risk-based approach, mandating transparency and explainability obligations primarily for "high-risk" AI systems—those deployed in areas like biometric identification, critical infrastructure, education, employment, and law enforcement—defined as systems presenting significant potential harm to health, safety, or fundamental rights. Providers of high-risk systems must conduct fundamental rights impact assessments, maintain detailed technical documentation on data sources, model training, and decision logic, and ensure systems are transparent enough for deployers and affected persons to understand outputs, including human-readable explanations of decisions where feasible; failure to comply can result in fines up to €35 million or 7% of global annual turnover. The Act also requires logging of operations for traceability and post-market monitoring, though it exempts general-purpose AI models unless adapted for high-risk use, reflecting a pragmatic acknowledgment of technical limits in achieving full interpretability for opaque "black-box" systems.^[126] Complementing the AI Act, the General Data Protection Regulation (GDPR), effective since May 25, 2018, imposes constraints on automated decision-making under Article 22, prohibiting decisions based solely on automated processing—including profiling—that produce legal effects or similarly significant impacts on individuals, unless explicitly authorized by law or necessary for contract performance, with safeguards like the right to human intervention, expression of views, and "an explanation of the decision reached." Recital 71 clarifies that such explanations should detail the logic involved, though courts and scholars debate its scope, interpreting it as requiring meaningful, non-generic rationales rather than full algorithmic disclosure to balance data protection with proprietary interests; enforcement has yielded fines, such as the €9.5 million penalty against Clearview AI in 2022 for opaque facial recognition practices lacking adequate explanations. This framework influences XAI by incentivizing interpretable models in personal data contexts but stops short of a universal "right to explanation," prioritizing contestability over exhaustive transparency.^[127] In the United States, the National Institute of Standards and Technology's AI Risk Management Framework (AI RMF 1.0), released on January 26, 2023, provides a voluntary, non-binding guideline for managing AI risks across the lifecycle, emphasizing "explainability and interpretability" as core to trustworthiness characteristics like transparency and accountability. It outlines practices for mapping risks (e.g., identifying opacity in decision processes), measuring outcomes (e.g., via fidelity metrics for post-hoc explanations), and managing mitigations (e.g., hybrid models combining accuracy with comprehensibility), without prescriptive mandates but encouraging alignment with sector-specific regulations like those from the Federal Trade Commission on deceptive AI practices. The framework's flexibility accommodates diverse AI deployments but relies on self-assessment, with updates planned iteratively based on stakeholder input.^[128] Internationally, the International Organization for Standardization's ISO/IEC 42001:2023, published in December 2023, sets requirements for AI management systems, integrating explainability into governance controls for ethical deployment, risk assessment, and continuous monitoring, applicable to organizations worldwide seeking certification. Similarly, ISO/IEC 22989:2022 defines key terms like "explainability" as the capacity to express factors influencing outputs, while ISO/IEC TR 24028:2020 (updated contexts) guides management of bias and fairness, promoting auditable transparency without legal enforcement. These standards facilitate compliance with binding regimes like the EU AI Act but remain advisory, highlighting a global patchwork where mandates cluster in high-stakes domains amid ongoing debates on enforceability for inherently complex neural networks.^[129]

Debates on Mandatory Explainability

Advocates for mandatory explainability in high-risk AI systems argue that it ensures accountability and trust, particularly in domains like healthcare and finance where decisions impact human rights and safety. For instance, the European Union's AI Act, effective from August 1, 2024, mandates transparency obligations for high-risk systems, including documentation of decision-making processes to allow human oversight and contestability of outputs, aiming to mitigate biases and errors through verifiable explanations.^[112] Proponents, including regulatory bodies, contend that such requirements align with broader legal principles like the GDPR's emphasis on meaningful information about automated decisions, enabling users to challenge outcomes and fostering ethical deployment.^[130] This perspective holds that without enforced explainability, opaque models risk unchecked errors, as evidenced by cases where black-box AI in lending or diagnostics has perpetuated discrimination without recourse.^[131] Critics, however, warn that mandating explainability imposes undue burdens, often trading off predictive accuracy for superficial transparency, as complex neural networks derive efficacy from non-linear interactions not easily distilled into human-readable forms. Studies show that interpretable models like decision trees frequently underperform deep learning counterparts by 5-20% in accuracy on high-dimensional tasks, suggesting mandates could stifle innovation in critical applications.^[132] Moreover, post-hoc explanation techniques, commonly proposed for compliance, can produce inconsistent or misleading rationales that create a "false sense of security," eroding rather than enhancing governance by masking underlying uncertainties.^[133] ^[134] In the EU AI Act context, opponents highlight enforcement gaps and loopholes that prioritize general transparency over rigorous explainability, potentially slowing European AI competitiveness without proportional risk reduction.^[135] ^[136] The debate extends to feasibility, with empirical evidence indicating that true causal interpretability remains elusive for scaled models trained on vast datasets, as approximations fail to capture emergent behaviors.^[137] Alternatives like rigorous validation through outcome testing and auditing are proposed over blanket mandates, arguing that over-reliance on explanations could divert resources from robust performance metrics.^[138] This tension reflects broader policy challenges, where mandatory explainability risks regulatory capture by interpretable-but-suboptimal methods, while voluntary approaches in less-regulated jurisdictions like the US have accelerated advancements without evident safety trade-offs.^[132]

International Variations and Enforcement Issues

The European Union's Artificial Intelligence Act, adopted on May 21, 2024, and entering into force progressively from August 2024, imposes mandatory transparency and explainability requirements on high-risk AI systems, such as those used in biometric identification or critical infrastructure, requiring providers to ensure systems are designed for human oversight and to provide deployers with sufficient information to interpret outputs.^[139]^[140] In contrast, the United States lacks a comprehensive federal AI law as of October 2025, relying instead on voluntary guidelines like the National Institute of Standards and Technology's (NIST) four principles of explainable AI—explanation, meaning, actionability, and justification—which emphasize measurement and policy support without enforceable mandates, alongside sector-specific agency policies such as the Office of Management and Budget's April 2025 memo promoting inherently explainable models in federal use.^[4]^[141] China's Interim Measures for the Management of Generative Artificial Intelligence Services, effective August 15, 2023, and subsequent frameworks like the September 2024 AI Safety Governance Framework, mandate transparency and explainability principles for AI developers and providers, requiring clear disclosure of training data sources and algorithmic logic to ensure accountability, though enforcement prioritizes state oversight and national security over user-centric interpretability.^[142]^[143] Other jurisdictions exhibit further divergence; for instance, the United Kingdom's pro-innovation approach under its 2023 AI White Paper avoids binding explainability rules, favoring sector-specific regulators, while emerging frameworks in countries like Japan and Brazil emphasize voluntary transparency aligned with OECD principles but lack uniform enforcement mechanisms.^[144] Enforcement challenges arise from these inconsistencies, including difficulties in verifying compliance for opaque models, as regulators struggle to assess whether explanations accurately reflect causal decision processes without standardized metrics, leading to potential over-reliance on self-reported disclosures by providers.^[116] Cross-border operations exacerbate issues, with multinational firms facing regulatory arbitrage risks—such as deploying less stringent U.S.-style voluntary guidelines to evade EU fines up to 6% of global turnover—and jurisdictional conflicts that hinder unified oversight, particularly for cloud-based AI systems spanning regions.^[112]^[145] Resource constraints in lower-capacity enforcers, combined with trade-offs between explainability and model performance, further complicate audits, as demonstrated by early EU cases where providers contested interpretability requirements due to technical infeasibility in high-dimensional systems.^[146] Absent global harmonization efforts, such as those proposed in international forums, these variations foster fragmented accountability and uneven protection against AI misuse.^[147]

Limitations and Trade-Offs

Performance-Explainability Conflicts

In machine learning, models achieving state-of-the-art predictive performance, such as deep neural networks and ensemble methods like random forests, frequently exhibit reduced interpretability compared to simpler alternatives like linear regression or single decision trees, as complexity enables capturing nonlinear interactions but obscures causal pathways.^[148]^[149] This tension manifests statistically when interpretability constraints—restricting models to transparent hypothesis classes—increase excess risk, leading to accuracy losses on high-dimensional or nonlinear data.^[149] Empirical evidence underscores domain-specific variations in the trade-off's severity. For instance, in a 2022 user study across education (Portuguese student performance dataset, 1,044 samples, 33 features) and housing (King County prices, 21,613 samples, 20 features) domains, black-box models outperformed interpretable ones in precision at 25% recall (0.85 vs. 0.78 for housing), yet participants rated black-boxes with post-hoc explanations (e.g., SHAP) as equally explainable, challenging assumptions of inherent opacity.^[150] Conversely, in natural language processing tasks, black-box models consistently surpassed interpretable baselines in accuracy, with performance degrading as constraints enforced greater transparency. In high-stakes contexts like healthcare or justice, this conflict favors inherently interpretable models over black boxes with explanations, as post-hoc methods risk misleading interpretations without guaranteeing fidelity to the underlying decision process.^[151] Cynthia Rudin argues that optimized interpretable models—such as sparse rule lists or generalized additive models—can approach black-box performance in targeted applications, avoiding explanation unreliability while enabling direct auditing and improvement.^[151] Ongoing research thus explores hybrid approaches, like distilling black-box knowledge into interpretable surrogates, to mitigate losses without fully sacrificing accuracy.^[152]

Scalability Issues in High-Dimensional Data

High-dimensional data, such as genomic sequences with thousands of features or images represented by millions of pixels, pose significant scalability challenges for explainable AI (XAI) methods due to the curse of dimensionality, where the volume of the feature space grows exponentially with the number of dimensions, complicating both computation and interpretability.^[153] Perturbation-based techniques like LIME and SHAP, which generate explanations by approximating local model behavior through repeated sampling and model evaluations, exhibit computational complexity that scales poorly; LIME's complexity grows quadratically with the number of features, while SHAP's approximations, such as KernelSHAP, can demand exponential resources relative to feature count, often rendering exact explanations infeasible for datasets exceeding hundreds of dimensions.^[154]^[155] For instance, computing exact Shapley values in high-dimensional spaces requires evaluating coalitions of features, which becomes prohibitive without approximations that may introduce noise or bias, particularly in unstructured data like medical imaging or tabular datasets from finance.^[156]^[157] These issues manifest in reduced explanation fidelity, as high-dimensional sparsity leads to unreliable feature attributions; in deep neural networks trained on such data, methods like saliency maps or integrated gradients provide pixel-level insights but struggle to aggregate meaningful global patterns without dimensionality reduction, which risks omitting causal interactions.^[2] Empirical studies on datasets like those in bioinformatics highlight that post-hoc XAI tools demand excessive runtime—often hours or days per instance—for models with over 1,000 features, limiting real-time deployment in applications such as drug discovery or autonomous systems.^[158] Moreover, the combinatorial explosion in perturbation sampling exacerbates hardware constraints, with GPU acceleration offering partial mitigation but failing to address the fundamental exponential growth.^[2] Efforts to mitigate scalability include hybrid approaches combining XAI with dimensionality reduction techniques like PCA or autoencoders prior to explanation generation, though these trade off completeness for efficiency and may propagate reduction-induced artifacts into interpretations.^[159] Despite approximations enabling practical use in moderate high-dimensional settings (e.g., up to 10,000 features with sampling heuristics), full scalability remains elusive for ultra-high dimensions, as evidenced by persistent computational bottlenecks in benchmarks involving convolutional neural networks on image data.^[160] This underscores a core trade-off in XAI: while intrinsic interpretable models avoid such costs, they often underperform black-box alternatives on high-dimensional tasks, prioritizing accuracy over explainability.^[153]

Vulnerability to Adversarial Manipulation

Explainable artificial intelligence (XAI) methods are vulnerable to adversarial manipulation, where attackers craft imperceptible perturbations to inputs that distort the explanations provided by the system while leaving the underlying model's predictions largely unchanged. This phenomenon, often termed "explanation attacks," exploits the sensitivity of post-hoc interpretability techniques, such as Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP), which approximate model behavior locally and can be fooled by inputs optimized to mislead these approximations.^[161]^[162] For instance, in white-box scenarios where attackers access the model's internals, perturbations can invert feature importance rankings in SHAP values, attributing causality to irrelevant features.^[163] Empirical studies demonstrate high success rates for such attacks across common XAI frameworks. A 2024 evaluation of LIME, SHAP, and Integrated Gradients on image classification tasks using datasets like CIFAR-10 showed that black-box attacks achieved explanation distortion rates exceeding 80% under minimal perturbation norms (e.g., L-infinity norm of 0.01), without altering prediction accuracy beyond 5%.^[164] In cybersecurity contexts, adversarial examples have manipulated XAI outputs in intrusion detection systems, causing explainers to highlight benign features as malicious, potentially evading defenses.^[165] These vulnerabilities extend to inherently interpretable models, such as attention-based mechanisms in transformers, where gradient-based attacks can shift focus to non-causal tokens, as observed in natural language processing benchmarks with attack success rates up to 95% on GLUE datasets.^[162] The mechanisms underlying these susceptibilities stem from the non-robust optimization landscapes of XAI methods, which prioritize fidelity to the black-box model over adversarial invariance. Attackers typically formulate objectives to maximize divergence between original and perturbed explanations—measured via metrics like cosine similarity of attribution maps—subject to constraints on prediction stability and perturbation boundedness.^[163]^[161] For example, projected gradient descent has been adapted to generate such examples, revealing that XAI's reliance on surrogate models or sampling introduces exploitable instabilities not present in raw predictions.^[162] In high-stakes applications, these manipulations undermine user trust and decision integrity; a clinician relying on an adversarially perturbed XAI explanation for medical imaging might misinterpret benign anomalies as pathological, leading to erroneous interventions.^[164] Surveys of over 50 studies indicate that while prediction-robust training (e.g., adversarial training with PGD) improves model resilience, it often degrades explanation quality, with SHAP fidelity dropping by 20-30% on robustified models tested on ImageNet subsets.^[162]^[163] This highlights a core trade-off: enhancing XAI robustness requires integrating defenses like explanation smoothing or certified bounds, yet current methods remain computationally intensive and incomplete, with certified robustness verified only for small perturbations in low-dimensional settings.^[161]

Criticisms and Controversies

Doubts on True Interpretability for Complex Systems

Skeptics of explainable AI contend that achieving genuine interpretability in complex systems, such as deep neural networks with billions of parameters and layered non-linear transformations, is fundamentally constrained by the models' internal opacity, where decision pathways defy reduction to human-comprehensible causal mechanisms. Cynthia Rudin argues that post-hoc explanation techniques applied to black-box models produce unreliable approximations rather than faithful representations of internal logic, as they cannot reliably distinguish true drivers from spurious correlations without sacrificing model performance. This view posits that distributed representations in neural networks—where knowledge is encoded across vast interconnections rather than localized features—preclude mechanistic understanding akin to dissecting simpler algorithms.^[151]^[166] Empirical assessments reinforce these doubts, demonstrating that even advanced interpretability tools fail to yield verifiable insights into model behavior. A 2023 study by MIT Lincoln Laboratory researchers tested human interpretability of AI agents using formal logical specifications in a simulated capture-the-flag scenario, finding participants achieved only approximately 45% accuracy in validating plans across formats like raw formulas, natural language, and decision trees, with experts exhibiting overconfidence and overlooking failure modes. Such results suggest that purported explanations often mask rather than reveal the opaque computations underlying predictions, particularly in high-stakes domains requiring causal fidelity.^[167] Deeper challenges arise from "structure opacity," where models accurately predict outcomes tied to incompletely understood external phenomena, such as causal relations beyond current empirical grasp, rendering full interpretability unattainable without parallel advances in domain knowledge. Rudin emphasizes that explanations for complex models risk misleading users by implying transparency that does not exist, potentially eroding trust more than opacity itself, as they conflate empirical correlations with verifiable mechanisms. These limitations imply that for sufficiently intricate systems, interpretability efforts may at best provide heuristic surrogates, not true causal realism, echoing broader scientific hurdles in probing emergent properties of complex systems.^[166]^[151]

Risks of Over-Reliance and Misplaced Trust

Over-reliance on explainable artificial intelligence (XAI) systems manifests as users uncritically deferring to AI outputs, even when explanations are provided, due to automation bias—the propensity to favor automated cues over independent judgment or contradictory evidence.^[168] This bias persists or intensifies with XAI because explanations can confer an illusion of comprehension, prompting users to overestimate model reliability without verifying underlying assumptions or error rates.^[169] Empirical investigations reveal that non-informative or flawed explanations still elevate acceptance of incorrect AI recommendations; for example, in a 2019 study on AI-assisted tasks, users exposed to explanations exhibited higher trust in outputs with 50% accuracy compared to opaque systems, resulting in elevated error commissions.^[170] Misplaced trust arises particularly among non-experts, who often interpret XAI features like feature importance visualizations as guarantees of correctness, leading to overconfidence in high-stakes applications. A 2025 study found that lay users' trust in XAI explanations exceeded calibrated levels, with participants rating system competence higher after viewing interpretability aids, even when subsequent AI errors contradicted them, thus amplifying decision risks.^[171] In healthcare contexts, this dynamic exacerbates harms: clinicians in a 2021 experiment were seven times more likely to endorse erroneous AI psychiatric diagnoses when supported by explanations, deferring to the system despite clinical expertise suggesting otherwise.^[172] Similarly, detailed rationales in clinical decision support increased reliance on flawed models among novice users, as shown in 2015 trials where explanation presence boosted endorsement of wrong predictions without improving overall accuracy.^[173] These risks compound in complex environments, where partial explanations (e.g., local surrogates like LIME) may highlight spurious correlations, fostering undue deference and downstream errors such as financial misallocations or diagnostic oversights.^[174] The "explanation paradox" underscores this: while XAI aims to calibrate reliance, it frequently induces higher confidence in erroneous outputs than black-box models, as users anchor on interpretive narratives rather than probabilistic uncertainties.^[175] Mitigation attempts, including uncertainty-aware explanations, yield inconsistent reductions in bias, with over-reliance persisting due to cognitive heuristics like confirmation bias toward provided justifications.^[176] In policy-sensitive domains, such patterns necessitate safeguards like mandatory human override protocols, though empirical evidence indicates explanations alone fail to avert systemic trust miscalibration.^[169]

Ideological Critiques and Hype Cycles

The pursuit of explainable artificial intelligence (XAI) has been characterized by pronounced hype cycles, mirroring Gartner's framework where technologies experience inflated expectations followed by disillusionment. Initial enthusiasm surged in the mid-2010s amid revelations of opacity in deployed systems, such as the 2016 ProPublica analysis of the COMPAS recidivism algorithm, which highlighted predictive disparities without clear causal mechanisms. This triggered a peak of optimism around 2018, coinciding with regulatory developments like the EU's General Data Protection Regulation (GDPR) Article 22, which implied a "right to explanation" for automated decisions, positioning XAI as a panacea for accountability and bias mitigation. However, by the early 2020s, empirical evaluations revealed limitations, including the prevalence of post-hoc approximations like LIME and SHAP that prioritize local fidelity over global causal insight, leading to a trough of disillusionment as real-world applications exposed fidelity-performance trade-offs.^[66] Gartner's 2025 Hype Cycle for Artificial Intelligence continues to feature XAI as a maturing domain, with vendors like SUPERWISE recognized for observability tools aimed at governance, yet the cycle underscores persistent challenges in scaling explainability amid pressures for fair and secure AI deployment.^[177] Businesses report implementation hurdles, as XAI methods often fail to deliver verifiable legitimacy for high-stakes decisions, contributing to skepticism about overhyped claims of enhanced trust.^[178] This phase reflects causal realism: complex models derive efficacy from distributed representations intractable to human-scale explanations, rendering many XAI techniques more performative than substantive, as evidenced by studies showing explanation instability across perturbations.^[123] Ideological critiques of XAI emphasize its alignment with precautionary paradigms in policy and academia, where demands for transparency prioritize normative ideals of human oversight over empirical outcomes from opaque systems. Brookings analyses argue that explainability does not resolve underlying political ambiguities in policy goals—such as balancing efficiency and equity in bail or welfare algorithms—but instead amplifies exposure to societal biases embedded in training data, potentially exacerbating distrust rather than alleviating it.^[122] This push, often amplified by institutions exhibiting systemic left-leaning biases toward interventionist frameworks, risks subordinating causal performance metrics to subjective interpretability standards, as seen in critiques of XAI's inability to legitimize decisions amid "explanation hacking" vulnerabilities that allow manipulative rationalizations.^[123] Proponents of unencumbered AI advancement counter that such mandates ideologically constrain scaling laws, where historical data shows performance gains from complexity outweigh sporadic interpretability gains, though these views receive less traction in mainstream discourse due to prevailing regulatory narratives.^[122]

References

[1]
[PDF] Explainable Artificial Intelligence (XAI)
Explainable artificial intelligence (XAI) is a subfield of artificial intelligence (AI) that provides explanations for the predictions, recommendations, and ...
[2]
None
Summary of each segment:
[3]
Explainable Artificial Intelligence (XAI): What we know and what is ...
The study starts by explaining the background of XAI, common definitions, and summarizing recently proposed techniques in XAI for supervised machine learning.
[4]
[PDF] Four Principles of Explainable Artificial Intelligence
We introduce four principles for explainable artificial intelligence (AI) that comprise fun- damental properties for explainable AI systems. We propose that ...
[5]
[PDF] Explainable AI Methods - A Brief Overview
Explainable Artificial Intelligence (xAI) is an established field with a vibrant community that has developed a variety of very successful approaches to explain ...
[6]
A Comprehensive Review of Explainable Artificial Intelligence (XAI ...
This review provides a focused comparative analysis of representative XAI methods in four main categories, attribution-based, activation-based, perturbation ...
[7]
[PDF] EXPLAINABLE ARTIFICIAL INTELLIGENCE
Explainable Artificial Intelligence (XAI) is the ability of AI systems to provide clear and understandable explanations for their actions and decisions. Its ...
[8]
EXplainable Artificial Intelligence (XAI)—From Theory to Methods ...
Jun 5, 2024 · In this review, we provide theoretical foundations of Explainable Artificial Intelligence (XAI), clarifying diffuse definitions and identifying research ...<|separator|>
[9]
Explainable artificial intelligence: an analytical review - Angelov
Jul 12, 2021 · This paper provides a brief analytical review of the current state-of-the-art in relation to the explainability of artificial intelligence.Missing: controversies | Show results with:controversies
[10]
Explainable AI (XAI): A systematic meta-survey of current challenges ...
Mar 5, 2023 · This is the first meta-survey that explicitly organizes and reports on the challenges and potential research directions of XAI.Missing: controversies | Show results with:controversies
[11]
What Is Black Box AI and How Does It Work? - IBM
Black box AI vs. White box AI, also called explainable AI (XAI) or glass box AI, is the opposite of black box AI. It is an AI system with transparent inner ...What is black box artificial... · Why do black box AI systems...
[12]
White Box vs. Black Box Algorithms in Machine Learning - ActiveState
Jul 19, 2023 · White box models are transparent, allowing easy interpretation of how they produce output. Black box models are opaque, not clarifying how they ...What Is a White Box Machine... · What Is a Black Box Machine...
[13]
White Box vs. Black Box Algorithms in Machine Learning - Medium
Oct 5, 2024 · Performance Tradeoff: Here's the deal: Black box algorithms often outperform white box models when it comes to prediction accuracy, especially ...
[14]
4 Methods Overview – Interpretable Machine Learning
Post-hoc interpretability means that we use an interpretability method after the model is trained. Post-hoc interpretation methods can be model-agnostic, such ...
[15]
Post-hoc vs ante-hoc explanations: xAI design guidelines for data ...
They are distinguished based on whether a model is intrinsically explainable (ante-hoc), or whether explainability is achieved by xAI approaches that analyze ...
[16]
Understanding the Dichotomy: Local vs. Global Explanations in XAI
Dec 28, 2023 · Global explanations delve into the broader picture, revealing what input variables significantly influence the model as a whole.
[17]
From local counterfactuals to global feature importance: efficient ...
Local explanations motivate the classification outcome of a given instance, while global explanations provide insight into the whole model. Moreover, the ...
[18]
Transparent AI: The Case for Interpretability and Explainability - arXiv
Jul 31, 2025 · In contrast, Explainable AI (XAI) typically involves supplementary techniques aimed at making the outputs and decisions of more complex, black- ...<|separator|>
[19]
A Comprehensive Taxonomy for Explainable Artificial Intelligence
May 15, 2021 · This paper unifies these efforts and provides a complete taxonomy of XAI methods with respect to notions present in the current state of research.
[20]
[2409.00265] Explainable Artificial Intelligence: A Survey of Needs ...
Aug 30, 2024 · ... taxonomy of XAI methods, and the application of XAI methods in different application areas. The survey is aimed at XAI researchers, XAI ...
[21]
https://doi.org/10.1016/j.inffus.2019.12.012
[22]
https://doi.org/10.1145/3351095.3372870
[23]
What is Explainable AI? - Software Engineering Institute
Jan 17, 2022 · For example, a study by IBM suggests that users of their XAI platform achieved a 15 percent to 30 percent rise in model accuracy and a 4.1 to ...
[24]
https://fiddler.ai/blog/explainable-ai
[25]
Explainable AI in Finance | Research & Policy Center
Aug 7, 2025 · Absence of universal explainability standards: Differing regional regulations (e.g., EU versus US regulations) create compliance challenges for ...
[26]
Artificial Intelligence Explainability Requirements of the AI Act and ...
The AI Act introduces requirements for explainable AI (XAI), but the regulations are abstract, making it challenging to define specific metrics for compliance.
[27]
Explainable AI for Safe and Trustworthy Autonomous Driving - arXiv
Jul 3, 2024 · We present the first comprehensive systematic literature review of explainable methods for safe and trustworthy AD.
[28]
Explainable AI for Real-Time Object Detection in Autonomous Driving
This paper integrates Explainable AI (XAI) with YOLOv8 for real-time object detection in autonomous driving, using CAM and LRP methods.
[29]
Recent Applications of Explainable AI (XAI): A Systematic Literature ...
In the realm of security and defense, XAI techniques have been widely applied to enhance cybersecurity measures. Several studies have focused on intrusion ...
[30]
Stop Explaining Black Box Machine Learning Models for High ... - NIH
Black box machine learning models are currently being used for high stakes decision-making throughout society, causing problems throughout healthcare, ...
[31]
Societal impacts of artificial intelligence: Ethical, legal, and ...
Technically, the over- and under-representativeness of the data used in AI models may lead to minority bias as certain groups are not fully considered [15].
[32]
Transparency and explainability of AI systems - ScienceDirect.com
Transparency and explainability are key quality requirements for AI systems. Explainability is integral to transparency, and is a key scope of ethical ...<|control11|><|separator|>
[33]
Toward Fairness, Accountability, Transparency, and Ethics in AI for ...
Ethical accountability ensures that AI systems make decisions that are transparent, justifiable, and aligned with societal values [61]. This encompasses ...
[34]
XAI4RE – Using Explainable AI for Responsible and Ethical AI
Jun 12, 2025 · This paper explores how XAI methods can be used throughout the AI lifecycle for creating human-centered, ethical, and responsible AI systems.
[35]
AI Ethics: Integrating Transparency, Fairness, and Privacy in AI ...
For example, the intersection of transparency and fairness is called accountability, which emphasizes the importance of transparent and fair AI decision-making.
[36]
The Contribution of XAI for the Safe Development and Certification ...
Jul 22, 2024 · We find that XAI methods can be a helpful asset for safe AI development, as they can show biases and failures of ML-models.
[37]
How does Explainable AI contribute to AI safety? - Zilliz
Explainable AI (XAI) contributes significantly to AI safety by enhancing transparency, facilitating trust, and improving the ability to detect and correct ...
[38]
The Urgency of Interpretability - Dario Amodei
This post makes the case for interpretability: what it is, why AI will go better if we have it, and what all of us can do to help it win the race.
[39]
Interpretability | AI Alignment
Oct 23, 2023 · Interpretability is a research field that makes machine learning systems and their decision-making process understandable to human beings.
[40]
Interpretability Will Not Reliably Find Deceptive AI - LessWrong
May 4, 2025 · Interpretability still seems a valuable tool and remains worth investing in, as it will hopefully increase the reliability we can achieve.
[41]
Explainable AI for Safe and Trustworthy Autonomous Driving - arXiv
Feb 8, 2024 · We then identify five key contributions of XAI for safe and trustworthy AI in AD, which are interpretable design, interpretable surrogate ...
[42]
Explainable AI - The building block for trustworthy AI Systems
Apr 4, 2024 · It represents a proactive approach to developing, assessing, and deploying AI systems in a manner that prioritizes safety and ethics.Explainable Ai -- The... · Safe Ai · Types Of Explainable Ai
[43]
https://doi.org/10.1002/widm.1391
[44]
Interpretable Machine Learning -- A Brief History, State-of-the-Art ...
Oct 19, 2020 · We present a brief history of the field of interpretable machine learning (IML), give an overview of state-of-the-art interpretation methods, and discuss ...Missing: foundations | Show results with:foundations
[45]
https://doi.org/10.1007/978-1-4757-1863-3
[46]
XAI: Explainable Artificial Intelligence - DARPA
The Explainable AI (XAI) program aims to create a suite of machine learning techniques that: Produce more explainable models, while maintaining a high level of ...Missing: revival 2010s
[47]
DARPA's explainable artificial intelligence (XAI) program
The DARPA's Explainable Artificial Intelligence (XAI) program endeavors to create AI systems whose learned models and decisions can be understood.
[48]
DARPA's Explainable AI (XAI) program: A retrospective - - Authorea
DARPA formulated the Explainable Artificial Intelligence (XAI) program in 2015 with the goal to enable end users to better understand, trust, and effectively ...
[49]
DARPA's Explainable Artificial Intelligence (XAI) Program
Jun 24, 2019 · DARPA's explainable artificial intelligence (XAI) program endeavors to create AI systems whose learned models and decisions can be understood and appropriately ...Missing: revival | Show results with:revival
[50]
[PDF] Explainable Artificial Intelligence (XAI) - National Security Archive
Nov 16, 2017 · DARPA's XAI seeks explanations from autonomous systems. Geoff Fein ... XAI Program Structure. Challenge. Problem. Areas. Evaluation.Missing: history | Show results with:history
[51]
Inside DARPA's effort to create explainable artificial intelligence
Jan 10, 2019 · DARPA's XAI initiative aims to shed light inside the black box of artificial intelligence algorithms. Project Manager Dave Gunning explains ...Missing: revival | Show results with:revival
[52]
[PDF] DARPA's Explainable AI (XAI) program: A retrospective
DARPA formulated the Explainable Artificial Intelligence (XAI) program in 2015 with the goal to enable end users to better understand, trust, and effectively ...Missing: history | Show results with:history
[53]
About - XAITK
Sep 9, 2021 · DARPA formulated the Explainable Artificial Intelligence (XAI) program in 2015 with the goal to enable end users to better understand, trust ...
[54]
Mechanistic Interpretability, Variables, and the Importance of ...
Jun 27, 2022 · Mechanistic interpretability seeks to reverse engineer neural networks, similar to how one might reverse engineer a compiled binary computer program.Missing: 2020s | Show results with:2020s
[55]
A Practical Review of Mechanistic Interpretability for Transformer ...
Jul 2, 2024 · Mechanistic interpretability (MI) is an emerging sub-field of interpretability that seeks to understand a neural network model by reverse- ...Missing: 2020s | Show results with:2020s
[56]
[PDF] Interpretability of machine learning‐based prediction models in ...
May 12, 2020 · Intrinsic interpretability refers to a process of selecting and training a ML model that is intrinsically interpretable due to its simple ...
[57]
Interpretable and explainable machine learning: A methods‐centric ...
Feb 28, 2023 · Interpretability and explainability are essential principles of machine learning model and method design and development for medicine, economics, law, and ...
[58]
Explainable vs. interpretable artificial intelligence frameworks ... - NIH
Some inherently interpretable models such as decision trees may have comparable discriminatory performance to complex (and opaque) models.
[59]
Explainable AI: A Review of Machine Learning Interpretability Methods
This study focuses on machine learning interpretability methods; more specifically, a literature review and taxonomy of these methods are presented.Missing: achievements | Show results with:achievements
[60]
14 LIME – Interpretable Machine Learning
Local surrogate models are interpretable models that are used to explain individual predictions of black box machine learning models.
[61]
[PDF] Fooling LIME and SHAP: Adversarial Attacks on Post hoc ...
Feb 7, 2020 · LIME [20] and SHAP [15] are two popular model-agnostic, local explanation approaches designed to explain any given black box classifier. These ...
[62]
(PDF) Problems With SHAP and LIME in Interpretable AI for Education
Jul 8, 2025 · Post-hoc explanation methods, including Kernel SHAP, Permutation SHAP, and LIME, are used to elucidate the ANN's decision-making processes, ...
[63]
https://ieeexplore.ieee.org/document/10762963
[64]
25 Surrogate Models – Interpretable Machine Learning
A global surrogate model is an interpretable model that is trained to approximate the predictions of a black box model. We can draw conclusions about the ...
[65]
Explainable AI (XAI) Methods Part 5— Global Surrogate Models
Feb 13, 2022 · Global Surrogate Models are used to explain “overall/global predictions” of black-box models while Local Surrogate Models, best represented by ...
[66]
Interpreting Black-Box Models: A Review on Explainable Artificial ...
Aug 24, 2023 · A black-box model in XAI refers to a machine learning model ... The following are the steps needed to create a global surrogate model [3]:.
[67]
19 Partial Dependence Plot (PDP) – Interpretable Machine Learning
The partial dependence plot (short PDP or PD plot) shows the marginal effect one or two features have on the predicted outcome of a machine learning model.
[68]
5.1. Partial Dependence and Individual Conditional Expectation plots
Partial dependence plots (PDP) and individual conditional expectation (ICE) plots can be used to visualize and analyze interaction between the target response.
[69]
Lessons from a Comprehensive Evaluation of Post Hoc Methods
Aug 6, 2024 · This paper addresses the notable gap in evaluating eXplainable Artificial Intelligence (XAI) methods for text classification.
[70]
Explainable AI: A Hybrid Approach to Generate Human-Interpretable ...
In this paper we aim to create human-interpretable explanations for predictions from deep learning models. We propose a hybrid of two prior approaches, ...
[71]
[PDF] Explainable Artificial Intelligence: An Overview on Hybrid Models
Explainable Artificial Intelligence (XAI) addresses this challenge, balancing the complexity of models with the necessary transparency and interpretability.
[72]
[PDF] Explainable Artificial Intelligence: an Overview on Hybrid Models
Sep 6, 2024 · This paper provides an exploration of hybrid models in XAI, elaborating on key concepts and offering a classification based on interpretability.
[73]
The role of causality in explainable artificial intelligence - arXiv
Sep 18, 2023 · In this paper, we investigate the literature to try to understand how and to what extent causality and XAI are intertwined.
[74]
Explainable AI and Causal Understanding: Counterfactual ...
Jun 9, 2023 · The counterfactual approach to explainable AI (XAI) seeks to provide understanding of AI systems through the provision of counterfactual explanations.
[75]
Holistic Explainable AI (H-XAI): Extending Transparency Beyond ...
Aug 7, 2025 · We introduce Holistic-XAI (H-XAI), a unified framework that integrates causal rating methods with traditional XAI methods to support explanation ...
[76]
From Black Box to Glass Box: A Practical Guide to Implementing XAI ...
Sep 16, 2025 · You will learn: What a hybrid Neuro-Symbolic-Causal agent is. Why explainability (XAI) is essential for such agents. What SHAP is ...
[77]
The Role of Causality in Explainable Artificial Intelligence - Carloni
May 7, 2025 · In this paper, we investigate the literature to try to understand how and to what extent causality and XAI are intertwined.ABSTRACT · Introduction · Methods · Results to the Research...
[78]
Implications of causality in artificial intelligence - Frontiers
Aug 20, 2024 · Causal AI emphasizes identifying cause-and-effect relationships and plays a crucial role in creating more robust and reliable systems.Abstract · Introduction · AI approaches against bias · Discussion
[79]
Evaluating the necessity of the multiple metrics for assessing ...
Oct 14, 2024 · This paper investigates the specific properties of Explainable Artificial Intelligence (xAI), particularly when implemented in AI/ML models across high-stakes ...
[80]
A Comprehensive Review of Explainable Artificial Intelligence (XAI ...
Jul 4, 2025 · This review provides a focused comparative analysis of representative XAI methods in four main categories, attribution-based, activation-based, ...
[81]
[2401.10640] A comprehensive study on fidelity metrics for XAI - arXiv
Jan 19, 2024 · In this study, we proposed a novel methodology to verify fidelity metrics, using a well-known transparent model, namely a decision tree.<|separator|>
[82]
Evaluation Metrics for XAI: A Review, Taxonomy, and Practical ...
This article reviews evaluation metrics used for XAI through the PRISMA systematic guideline for a comprehensive and systematic literature review.
[83]
Measures for explainable AI: Explanation goodness, user ... - Frontiers
A number of XAI developers have recognized the importance of measuring the qualities of explanations of AI systems (e.g., Ehsan et al., 2019). Holzinger et al.<|separator|>
[84]
Exploring the means to measure explainability: Metrics, heuristics ...
These ten aspects are Understandability, Transparency, Effectiveness, Efficiency, Satisfaction, Correctness, Suitability, Trustability, Persuasiveness and ...
[85]
Dataset resulting from the user study on comprehensibility of ...
Jun 13, 2025 · This paper introduces a dataset that is the result of a user study on the comprehensibility of explainable artificial intelligence (XAI) algorithms.Methods · Feature Importance... · Data Records
[86]
[PDF] Evaluation Metrics for XAI: A Review, Taxonomy, and Practical ...
It measures how well an AI model's explanations align with its underlying decision-making processes. For instance, in the medical image analysis study by Jin et ...
[87]
Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations
### Summary of Key Challenges and Limitations in Human-Centered Evaluation for Explainable AI
[88]
What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks
### Summary of Evaluation of XAI from arXiv:2403.12730
[89]
Towards Human-centered Design of Explainable Artificial ... - arXiv
Oct 28, 2024 · This survey reviews empirical studies for human-centered XAI design, analyzing algorithms, stakeholders, design space, and evaluation metrics.
[90]
[PDF] M4: A Unified XAI Benchmark for Faithfulness Evaluation of Feature ...
Our work addresses this gap by proposing a unified benchmark for explainable AI across different modalities, with the goal of facilitating holistic progress in ...<|separator|>
[91]
XAI-Units: Benchmarking Explainability Methods with Unit Tests
Jun 23, 2025 · We introduce the open-source XAI-Units benchmark, specifically designed to evaluate FA methods against diverse types of model behaviours.
[92]
BenchXAI: Comprehensive benchmarking of post-hoc explainable ...
A novel XAI benchmarking package supporting comprehensive evaluation of fifteen XAI methods, investigating their robustness, suitability, and limitations in ...
[93]
[2407.19897] BEExAI: Benchmark to Evaluate Explainable AI - arXiv
Jul 29, 2024 · A benchmark tool that allows large-scale comparison of different post-hoc XAI methods, employing a set of selected evaluation metrics.
[94]
XAI Benchmark for Visual Explanation - arXiv
Our work releases an XAI benchmark for visual explanation that consists of eight distinct datasets across topics like object classification and medical image ...
[95]
[PDF] Benchmarking eXplainable AI - A Survey on Available Toolkits and ...
Our survey can serve as a guide for the XAI com- munity for identifying future directions of research, and most notably, standardisation of evaluation. 1 ...
[96]
European standardization efforts from FAIR toward explainable-AI ...
European initiatives have proposed a series of metadata standards and procedural recommendations that were accepted as CEN workshop agreements.
[97]
OpenXAI : Towards a Transparent Evaluation of Model Explanations
Every explanation method in OpenXAI is a benchmark, and we provide dataloaders, pre-trained models, together with explanation methods and performance evaluation ...<|control11|><|separator|>
[98]
Explainable AI in Clinical Decision Support Systems - PubMed Central
This systematic review aims to provide a comprehensive overview of current XAI techniques in CDSSs, analyze their effectiveness and limitations, and outline the ...
[99]
(PDF) Explainable AI in Healthcare Decision Support Systems
Jan 4, 2025 · The paper discusses various XAI techniques, including feature attribution methods (e.g., SHAP, LIME) and visualization tools, as well as their ...<|separator|>
[100]
A systematic review of Explainable Artificial Intelligence in medical ...
This systematic literature review examines state-of-the-art Explainable Artificial Intelligence (XAI) methods applied to medical image analysis.
[101]
Leveraging explainable artificial intelligence to optimize clinical ...
Feb 22, 2024 · To develop and evaluate a data-driven process to generate suggestions for improving alert criteria using explainable artificial intelligence (XAI) approaches.Materials And Methods · Model Development · Results
[102]
Explainable discovery of disease biomarkers: The case of ovarian ...
We aim to introduce best practice for combining ML and XAI techniques for biomarker validation tasks.Original Research · 2. Methodology · 3. Result
[103]
How Explainable Artificial Intelligence Can Increase or Decrease ...
Oct 30, 2024 · 5 studies reported that XAI increased clinicians' trust compared with standard AI, particularly when the explanations were clear, concise, and relevant to ...
[104]
A Case Study on Traumatic Brain Injury - PMC - NIH
This study compares six XAI methods for TBI prediction models, finding SHAP most stable with high fidelity, but Anchors most understandable for tabular data.
[105]
Systematic Review of Clinical Decision Support Systems - medRxiv
Aug 10, 2024 · The review covers the datasets, application areas, machine learning models, explainable AI methods, and evaluation strategies for multiple XAI ...
[106]
A historical perspective of biomedical explainable AI research - PMC
Sep 8, 2023 · We aimed at analyzing the possible associations between COVID-19 and the push of explainable AI (XAI) to the forefront of biomedical research.
[107]
Application of explainable artificial intelligence in medical health
This paper investigates the applications of explainable AI (XAI) in healthcare, which aims to provide transparency, fairness, accuracy, generality, and ...
[108]
Explainable AI in medicine: challenges of integrating XAI into the ...
XAI deals with the problem of providing insights as to how an AI system uses information to solve a given task (1, 2). In this paradigm, incorporating human-in- ...
[109]
Advances in Explainable Artificial Intelligence (xAI) in Finance
xAI is becoming a vital element in finance and economics in fields like risk management, credit decisions, and regulatory compliance.
[110]
(PDF) The Role of Explainable AI in Financial Risk Assessment and ...
Jun 6, 2025 · This paper explores the pivotal role of Explainable AI in financial risk assessment and mitigation. It investigates the current state of AI- ...
[111]
Explainable AI for Financial Risk Management - EasyChair
Aug 6, 2024 · We delve into the mechanisms by which XAI elucidates the decision-making processes of complex models, providing clear, interpretable insights ...
[112]
High-level summary of the AI Act | EU Artificial Intelligence Act
The AI Act classifies AI by risk, prohibits unacceptable risk, regulates high-risk, and has lighter obligations for limited-risk AI. Most obligations fall on ...High Risk Ai Systems... · Requirements For Providers... · General Purpose Ai (gpai)Missing: explainability | Show results with:explainability
[113]
The EU AI Act: Key Provisions and Impact on Financial Services
The Act mandates that high-risk AI systems be transparent and explainable. This requirement may necessitate significant changes in how financial institutions ...
[114]
How Does Explainable AI Improve Transparency in AML Compliance?
Jul 7, 2025 · Explainable AI shows how a model identifies suspicious behavior, including an unusually large transfer of money from a high-risk jurisdiction.
[115]
Why Are Explainable AI and Responsible AI Important in the ... - Verint
Mar 31, 2025 · Explainable AI ensures that flagged transactions or risk scores can be justified and audited.
[116]
[PDF] Managing explanations: how regulators can address AI explainability
As mentioned, AI methodologies may help financial institutions to increase the efficiency of their operations, improve risk management and provide clients with ...
[117]
Explainable artificial intelligence (XAI) in banking | Deloitte Insights
May 17, 2022 · XAI aims to make AI models more explainable, intuitive, and understandable to human users without sacrificing performance or prediction accuracy.
[118]
Explainable artificial intelligence (XAI) in finance: a systematic ...
Jul 26, 2024 · The most popular financial tasks addressed by the AI using XAI were credit management, stock price predictions, and fraud detection. The three ...
[119]
Explainable and transparent artificial intelligence for public ...
Feb 16, 2024 · This paper illustrates a collection of AI solutions that can empower data scientists and policymakers to use AI/ML for the development of explainable and ...
[120]
(PDF) Explainable and transparent artificial intelligence for public ...
For instance, AI-based policy development solutions must be transparent and explainable to policymakers, while at the same time adhering to the mandates of ...<|separator|>
[121]
Explainable AI for government: Does the type of explanation matter ...
The studies revealed that offering the subjects some type of explanation had a positive effect on their attitude towards a decision, to various extents.
[122]
The tensions between explainable AI and good public policy
Sep 15, 2020 · A common principle of AI ethics is explainability. The risk of producing AI that reinforces societal biases has prompted calls for greater ...
[123]
Transparency and Explainability for Public Policy
Nov 4, 2024 · In this paper I will argue for the moral importance of transparent and explainable AI in policymaking.
[124]
Explainable and efficient randomized voting rules
Dec 10, 2023 · Peeking inside the black-box: a survey on explainable artificial intelligence (xai). ... Social choice theory and recommender systems ...
[125]
Representative Social Choice: From Learning Theory to AI Alignment
Oct 31, 2024 · In this study, we propose the representative social choice framework for the modeling of democratic representation in collective decisions, ...Missing: XAI | Show results with:XAI
[126]
[PDF] Regulation (EU) 2024/1689 of the European Parliament ... - EUR-Lex
Jun 13, 2024 · A Union legal framework laying down harmonised rules on AI is therefore needed to foster the development, use and uptake of AI in the internal ...
[127]
Art. 22 GDPR – Automated individual decision-making, including ...
Rating 4.6 (9,706) The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal ...
[128]
[PDF] Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Jan 1, 2023 · NIST plans to update the AI RMF Playbook frequently. Comments on the AI RMF Playbook may be sent via email to AIframework@nist.gov at any time.
[129]
Artificial intelligence - ISO/IEC TS 6254:2025
In stockThis document describes approaches and methods that can be used to achieve explainability objectives of stakeholders with regard to machine learning (ML) ...
[130]
The Role of Explainability in AI Regulatory Frameworks - Hyperight
which often means explainable — AI.
[131]
Should AI models be explainable to clinicians? - PMC
Sep 12, 2024 · “Explainable AI” (XAI) aims to bridge this gap, enhancing confidence among patients and doctors. It also helps to meet regulatory transparency requirements.Missing: mandatory | Show results with:mandatory
[132]
Do All AI Systems Need to Be Explainable?
Nov 15, 2023 · “Explainable AI” can bridge the gap between AI outputs and human expertise, but a balance needs to be struck between explainability and performance.
[133]
False Sense of Security in Explainable Artificial Intelligence (XAI)
May 6, 2024 · We argue that these AI regulations and current market conditions threaten effective AI governance and safety because the objective of ...
[134]
Explainability of AI – benefits, risks and accountability - DLA Piper
May 22, 2024 · The fact that AI can be explained may also result in a false sense of security for users regarding the risks associated with AI and result in ...
[135]
The Explainability Illusion: Why AI Transparency Requirements Miss ...
Aug 11, 2025 · The explainability illusion represents a broader AI governance challenge: adapting AI systems to existing legal frameworks rather than ...
[136]
Packed with loopholes: why the AI Act fails to protect civic space and ...
Apr 3, 2024 · The AI Act fails to effectively protect the rule of law and civic space, instead prioritising industry interests, security services and law enforcement bodies.
[137]
Unraveling the incomprehensible - the pros and cons of explainable AI
Understanding AI decisions can be difficult, but explainable. Analysis can help build trust. Discover these solutions and more.
[138]
Should AI be Regulated? The Arguments For and Against
The case could be made that regulations will slow down AI advancements and breakthroughs. That not allowing companies to test and learn will make them less ...
[139]
Article 13: Transparency and Provision of Information to Deployers
This article states that high-risk AI systems must be designed to be transparent, so that those using them can understand and use them correctly. They must come ...
[140]
Key Issue 5: Transparency Obligations - EU AI Act
The EU AI Act introduces different transparency obligations for the providers and deployers of AI systems. These rules can be understood in three dimensions.
[141]
[PDF] M-25-21 Accelerating Federal Use of AI through Innovation ...
Apr 3, 2025 · Agency policies should aim to advance using models that are built with less data, require less compute, and are inherently more explainable, ...
[142]
[PDF] Translation Artificial Intelligence Law of the People's Republic of China
May 2, 2024 · Article 6 Principle of Transparency and Explainability4. AI developers, providers, and users shall adhere to the principle of transparency.
[143]
China-releases-AI-safety-governance-framework - DLA Piper
Sep 12, 2024 · The Framework prioritizes addressing ethical concerns in AI development, including safety, transparency, and accountability, and seeks to ...
[144]
Transparency and explainability (OECD AI Principle)
This principle is about transparency and responsible disclosure around AI systems to ensure that people understand when they are engaging with them and can ...
[145]
[PDF] AI Regulation Across Borders: Legal Challenges and Prospects for ...
Jun 12, 2025 · It explores the complexities of crafting an international AI treaty, including challenges related to enforcement mechanisms, regulatory burdens ...
[146]
The role of explainability and human intervention in AI decisions
Oct 17, 2025 · It identifies key challenges such as limited enforcement mechanisms, legal ambiguity, the trade-off between accuracy and interpretability, and ...
[147]
A Global South Perspective on Explainable AI
Apr 30, 2024 · The absence of clear descriptions and universally agreed-upon standards for explainable AI may be indicative of a fragmented regulatory ...Missing: variations | Show results with:variations
[148]
Stop ordering machine learning algorithms by their explainability! A ...
Research indicates that there is a tradeoff between their model performance and explainability. Machine learning models with higher performance are often based ...
[149]
Enforcing Interpretability and its Statistical Impacts: Trade-offs between Accuracy and Interpretability
### Summary of Modeling Trade-off Between Interpretability and Accuracy
[150]
[PDF] An Empirical Study of the Accuracy-Explainability Trade-off in ...
The study found no direct trade-off between accuracy and explainability, and that interpretable models were not superior in terms of explainability.
[151]
[1811.10154] Stop Explaining Black Box Machine Learning Models ...
Nov 26, 2018 · This manuscript clarifies the chasm between explaining black boxes and using inherently interpretable models, outlines several key reasons why explainable ...
[152]
[PDF] When an Interpretable Model Collaborates with a Black-box Model
They are easy to understand, use, and improve compared to complex black-box models. Yet, the benefits of interpretable machine learning models often come at a ...
[153]
[PDF] Opportunities and Challenges in Explainable Artificial Intelligence ...
Jun 23, 2020 · The large number of parameters in Deep Neural Networks. (DNNs) make them complex to understand and undeniably harder to interpret. Regardless of ...
[154]
What are the computational complexities of LIME and SHAP for high ...
For high-dimensional data, LIME's complexity grows quadratically with the number of features, making it computationally expensive. Computational Complexity of ...
[155]
Can SHAP and LIME handle high-dimensional input data?
While SHAP can technically handle high-dimensional data, its computational complexity grows exponentially with the number of features. This makes it ...
[156]
What's Wrong with Your Synthetic Tabular Data? Using Explainable ...
Apr 29, 2025 · For large datasets and high-dimensional feature spaces, computing exact Shapley values can be computationally expensive or even infeasible.
[157]
Unveiling the footprints of eXplainable AI in Industry 4.0/5.0: a ...
Aug 5, 2025 · Although SHAP provided deep insights into model behavior, they noted scalability challenges with high-dimensional, unstructured data such as ...
[158]
[PDF] A SURVEY ON EXPLAINABLE AI: TECHNIQUES AND CHALLENGES
This survey provides a comprehensive review of XAI techniques, categorizing them into post-hoc and intrinsic methods, and examines their application in various ...<|separator|>
[159]
A Review of Multimodal Explainable Artificial Intelligence - arXiv
Dec 18, 2024 · To address this, explainability methods based on dimensionality reduction aim to reduce redundancy and computational load. Among these methods, ...
[160]
A quantitative approach for the comparison of additive local ...
Our findings reveal that LIME and SHAP's approximations are particularly efficient in high dimension and generate intelligible global explanations, but they ...
[161]
Adversarial attacks and defenses in explainable artificial intelligence
This survey provides a comprehensive overview of research concerning adversarial attacks on explanations of machine learning models, as well as fairness ...
[162]
Adversarial attacks and defenses in explainable artificial intelligence
Jul 28, 2025 · Explainable artificial intelligence (XAI) methods are portrayed as a remedy for debugging and trusting statistical and deep learning models, ...Adversarial Attacks And... · 3 Adversarial Attacks On... · 4 Defense Against The...
[163]
Adversarial Attacks in Explainable Machine Learning: A Survey of ...
Oct 27, 2024 · We review the possibilities and limits of adversarial examples in explainable machine learning scenarios, analyzing and illustrating the ...
[164]
(PDF) Evaluating the Robustness of Explainable AI Models Against ...
Apr 25, 2025 · This research evaluates the robustness of widely-used XAI models-including LIME, SHAP, and Integrated Gradients-against white-box and black-box ...Missing: peer- | Show results with:peer-
[165]
Adversarial Attacks on XAI Explanation in Cybersecurity Applications
Oct 4, 2025 · It is evident that XAI methods can themselves be a victim of post-adversarial attacks that manipulate the expected outcome from the explanation ...
[166]
Deep learning models and the limits of explainable artificial ...
Jan 30, 2025 · Typically, the intricate calculations between layers in deep learning models, the sheer volume of data, and the vast number of features included ...
[167]
https://arxiv.org/abs/2305.17258
[168]
[2204.08859] On the Influence of Explainable AI on Automation Bias
Apr 19, 2022 · We aim to shed light on the potential to influence automation bias by explainable AI (XAI). In this pre-test, we derive a research model and describe our study ...
[169]
[PDF] Overreliance on AI Literature Review - Microsoft
Overreliance on AI is when users accept incorrect AI outputs, making errors of commission, and not knowing how much to trust the AI.
[170]
https://doi.org/10.1145/3287560.3287590
[171]
https://www.researchgate.net/publication/396683432_Non-experts%27_Trust_in_XAI_is_Unreasonably_High
[172]
https://doi.org/10.1038/s41398-021-01224-x
[173]
https://doi.org/10.1109/ICHI.2015.26
[174]
Explainability pitfalls: Beyond dark patterns in explainable AI - PMC
Examples of these downstream negative effects include user perceptions like misplaced trust, over-estimating the AI's capabilities, and over-reliance on certain ...
[175]
Misplaced trust in AI : the explanation paradox and the ... - HAL Thèses
Jul 25, 2024 · Misplaced trust in AI : the explanation paradox and the human-centric path. ... In response, the research field of explainability (XAI) has ...
[176]
The effects of explanations on automation bias - ScienceDirect
In particular, automation can result in automation bias, which is “the tendency to use automated cues as a heuristic replacement for vigilant information ...
[177]
The 2025 Hype Cycle for Artificial Intelligence Goes Beyond GenAI
Jul 8, 2025 · The AI Hype Cycle is Gartner's graphical representation of the maturity, adoption metrics and business impact of AI technologies (including GenAI).
[178]
SUPERWISE® Leads Explainable AI in 4 Gartner® Hype Cycles 2025
“Being named by Gartner® in four separate Hype Cycle™ reports is a powerful validation of our mission to make AI trustworthy and transparent. Whether it's a ...