AI must do more than predict: New framework pushes for context-aware transparency

To test the real-world efficacy of explainable AI, the study focused on fraud detection in financial institutions. Fraud analysts frequently rely on AI models to flag suspicious transactions, but they often struggle to understand why a transaction is considered fraudulent.

Artificial Intelligence (AI) is transforming decision-making across industries, from finance to healthcare. However, the widespread adoption of AI-driven systems has raised concerns about transparency, interpretability, and trust. AI models, particularly those used in high-stakes environments, often function as "black boxes", making it difficult for users to understand how decisions are made. This lack of explainability can hinder adoption and lead to resistance from stakeholders.

A recent study, "Real-World Efficacy of Explainable Artificial Intelligence using the SAGE Framework and Scenario-Based Design," authored by Eleanor Mill, Wolfgang Garn, and Chris Turner, and published in Applied Artificial Intelligence (2024), explores an innovative approach to explainable AI (XAI). The research introduces the SAGE framework (Settings, Audience, Goals, and Ethics), which aims to contextualize AI explanations based on real-world requirements. By integrating scenario-based design (SBD), the study demonstrates how AI models can be tailored to specific user needs, enhancing trust and usability.

The SAGE framework: Bridging AI explainability gaps

Traditional AI models often prioritize performance over transparency, leading to explanations that are either too technical for non-experts or insufficient for real-world applications. The SAGE framework was developed to bridge this gap by structuring AI explanations around four key dimensions:

1. Settings: The operating environment of the AI model, including industry-specific constraints and risks.
2. Audience: The end-users who rely on AI explanations, ensuring that outputs are understandable and actionable.
3. Goals: The purpose of the explanation, whether for auditing, decision support, or user trust-building.
4. Ethics: The fairness, accountability, and reliability of AI decisions.

The study highlights that most current XAI models lack real-world usability because they do not consider these contextual factors. By applying the SAGE framework, researchers were able to fine-tune AI explanations to align with user needs, ultimately making AI-driven decisions more interpretable and trustworthy.

Scenario-based design for real-world AI applications

To test the real-world efficacy of explainable AI, the study focused on fraud detection in financial institutions. Fraud analysts frequently rely on AI models to flag suspicious transactions, but they often struggle to understand why a transaction is considered fraudulent.

Using scenario-based design (SBD), the study created a realistic fraud investigation workflow with a fictional fraud analyst, "Patrick." The scenario outlined Patrick's daily tasks, the decision-making challenges he faced, and the types of explanations he required to effectively perform his job. This approach helped researchers align AI explanations with actual industry workflows, ensuring that the model's outputs were useful, interpretable, and aligned with operational demands.

For instance, the fraud detection model needed to:

• Provide clear justifications for why a transaction was flagged.
• Generate templated text-based reports that could be easily shared.
• Be efficient and scalable for handling large transaction volumes.

By embedding realistic user requirements into AI model development, the study demonstrated how SBD enhances the practicality of XAI solutions.

Selecting the right XAI model: TreeSHAP for fraud detection

Choosing an appropriate explainability method is crucial in high-stakes applications like fraud detection. The study evaluated several XAI techniques, including SHAP (SHapley Additive Explanations) and LIME (Local Interpretable Model-Agnostic Explanations).

After a detailed comparison, TreeSHAP was selected as the most suitable model. Unlike traditional SHAP methods, TreeSHAP is optimized for tree-based models (such as random forests and gradient boosting machines), making it ideal for fraud detection. It offers:

• Faster computations while maintaining high accuracy.
• Feature importance scores, helping analysts understand why specific transactions were flagged.
• Scalability, enabling financial institutions to process thousands of transactions in real time.

However, TreeSHAP had two key limitations:

1. It lacked textual explanations, making it difficult for non-technical users to interpret results.
2. It did not provide a faithfulness score, meaning users had no way of knowing how reliable its explanations were.

To address these issues, the researchers enhanced TreeSHAP by integrating:

• Text-based explanations, converting complex visual outputs into clear, structured reports.
• A faithfulness score, using Rank Biased Overlap (RBO) to measure how well the AI's explanation aligned with the actual model's decision logic.

These improvements significantly boosted trust and usability, ensuring that AI-generated fraud alerts were both actionable and transparent.

Implications for AI adoption and future research

The study provides valuable insights into how explainable AI can be practically integrated into real-world decision-making. By applying the SAGE framework and scenario-based design, AI models can be tailored to better serve human decision-makers, reducing resistance to adoption and improving trust.

One of the key takeaways is that AI explanations should not be one-size-fits-all. Instead, they must be adaptable to different industries, user expertise levels, and ethical considerations. The research also underscores the importance of faithfulness in AI explanations, as users need to know how much confidence they can place in an AI-generated decision.

Future research should focus on:

• Expanding the SAGE framework to other high-risk domains like healthcare, cybersecurity, and legal decision-making.
• Refining faithfulness metrics, ensuring AI explanations are not just interpretable but also accurate representations of model logic.
• Integrating AI explainability with large language models (LLMs) to further enhance text-based explanations.

Conclusion

The study by Mill, Garn, and Turner presents a major advancement in explainable AI, demonstrating how the SAGE framework and scenario-based design can bridge the gap between technical AI explanations and real-world usability. By focusing on user-centric design, AI models can become more trustworthy, transparent, and effective in high-stakes environments like fraud detection.

As AI adoption continues to expand, ensuring explainability and accountability will be critical. This research lays a strong foundation for more interpretable, user-aligned AI systems, paving the way for greater public and industry trust in AI-driven decision-making.