DiFACE2: Generating Diverse, Feasible, and Actionable Counterfactual Explanations for Post-Conflict Primary Education Interventions Using Causal DAGs
DOI:
https://doi.org/10.62411/faith.3048-3719-283Keywords:
Causal DAG, Counterfactual Explanations, Diverse Feasible and Actionable, Educational Intervention, Explainable Artificial Intelligence, Feasibility, Machine Learning, Structural Causal ModelAbstract
We introduce DiFACE2 causal DAG, a framework for generating Diverse, Feasible, and Actionable Counterfactual Explanations under causal Directed Acyclic Graph (DAG) constraints. Leveraging background knowledge from an educational intervention program in post-conflict Nigeria, we construct an application-specific causal DAG to identify causal relationships and immutable features during counterfactual generation for black-box ML predictions. Using this DAG, we develop a causally constrained sparse ML model that integrates prior frameworks to generate diverse, feasible, and actionable counterfactuals for the Strengthening Education in Northeast – Early Grade Reading Assessment (SENSE-EGRA) dataset. DiFACE2 produces up to four counterfactuals per prediction, demonstrating high feasibility and actionability for categorical variables. While numeric features occasionally yield infeasible recommendations, this is mitigated by additional non-causal constraints. Although the sparse ML model yields lower accuracy and F1 scores than benchmark frameworks, this stems from the higher complexity and noisiness of the real-world dataset rather than methodological weakness. This trade-off underscores that meaningful, interpretable counterfactuals are preferable to near-perfect accuracy on simplified benchmarks. Therefore, when fairly evaluated on the same complex dataset, DiFACE2 outperforms prior methods across all counterfactual quality metrics.
Downloads
References
F. Doshi-Velez and B. Kim, “Towards A Rigorous Science of Interpretable Machine Learning,” arXiv. Feb. 27, 2017. [Online]. Available: http://arxiv.org/abs/1702.08608
M. J. Kusner, J. R. Loftus, C. Russell, and R. Silva, “Counterfactual Fairness,” in 31st Conference on Neural Information Processing Systems (NIPS 2017, Mar. 2018. [Online]. Available: http://arxiv.org/abs/1703.06856
Z. C. Lipton, “The Mythos of Model Interpretability,” Queue, vol. 16, no. 3, pp. 31–57, Jun. 2018, doi: 10.1145/3236386.3241340.
Association for Computing Machinery US Public Policy Council, “Statement on Algorithmic Transparency and Accountability.” 2017. [Online]. Available: https://www.acm.org/binaries/content/assets/public-policy/2017_usacm_statement_algorithms.pdf
D. Gunning, “Explainable Artificial Intelligence (XAI),” National Security Archive. 2017. [Online]. Available: https://nsarchive.gwu.edu/sites/default/files/documents/5794867/National-Security-Archive-David-Gunning-DARPA.pdf
B. Goodman and S. Flaxman, “European Union Regulations on Algorithmic Decision Making and a ‘Right to Explanation,’” AI Mag., vol. 38, no. 3, pp. 50–57, Sep. 2017, doi: 10.1609/aimag.v38i3.2741.
M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why Should I Trust You?’ : Explaining the Predictions of Any Classifier,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Aug. 2016, vol. 13-17-Augu, pp. 1135–1144. doi: 10.1145/2939672.2939778.
S. Wachter, B. Mittelstadt, and C. Russell, “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR,” SSRN Electron. J., 2017, doi: 10.2139/ssrn.3063289.
D. Buchsbaum, S. Bridgers, D. Skolnick Weisberg, and A. Gopnik, “The power of possibility: causal learning, counterfactual reasoning, and pretend play,” Philos. Trans. R. Soc. B Biol. Sci., vol. 367, no. 1599, pp. 2202–2212, Aug. 2012, doi: 10.1098/rstb.2012.0122.
S. R. Beck, K. J. Riggs, and S. L. Gorniak, “Relating developments in children’s counterfactual thinking and executive functions,” Think. Reason., vol. 15, no. 4, pp. 337–354, Nov. 2009, doi: 10.1080/13546780903135904.
D. S. Weisberg and A. Gopnik, “Pretense, Counterfactuals, and Bayesian Causal Models: Why What Is Not Real Really Matters,” Cogn. Sci., vol. 37, no. 7, pp. 1368–1381, Sep. 2013, doi: 10.1111/cogs.12069.
D. Garreau and U. von Luxburg, “Explaining the Explainer: A First Theoretical Analysis of LIME,” in Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS) 2020, Jan. 2020. [Online]. Available: http://arxiv.org/abs/2001.03447
C. Molnar, Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. LeanPub, 2020. [Online]. Available: https://christophm.github.io/interpretable-ml-book/
A. Fisher, C. Rudin, and F. Dominici, “All Models are Wrong, but Many are Useful: Learning a Variable’s Importance by Studying an Entire Class of Prediction Models Simultaneously,” arXiv. Dec. 23, 2019. [Online]. Available: http://arxiv.org/abs/1801.01489
R. K. Mothilal, A. Sharma, and C. Tan, “Explaining machine learning classifiers through diverse counterfactual explanations,” in Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, Jan. 2020, pp. 607–617. doi: 10.1145/3351095.3372850.
G. T. Ayem, A. Ajibesin, A. Iorliam, and A. S. Nsang, “A mixed framework for causal impact analysis under confounding and selection biases: a focus on Egra dataset,” Int. J. Inf. Technol., vol. 17, no. 5, pp. 2731–2748, Oct. 2023, doi: 10.1007/s41870-023-01490-6.
G. T. Ayem, O. Asilkan, and A. Iorliam, “Design and Validation of Structural Causal Model: A Focus on EGRA Dataset,” J. Comput. Theor. Appl., vol. 1, no. 2, pp. 86–103, Nov. 2023, doi: 10.33633/jcta.v1i2.9304.
G. T. Ayem, O. Asilkan, A. Iorliam, R. Ibrahim, S. George, and S. G. Thandekkattu, “Causal Inference Estimates with Backdoor Adjustment Condition vs. the Unconfoundedness Assumption: A Comparative Analysis Study of the Structural Causal Model and the Potential Outcome Frameworks,” ESP J. Eng. Technol. Adv., vol. 3, no. 4, pp. 1–22, 2023, doi: 10.56472/25832646/JETA-V3I8P101.
G. T. Ayem, A. S. Nsang, B. I. Igoche, and G. Naankang, “Design and Validation of Structural Causal Model: A focus on SENSE-EGRA Datasets,” Int. J. Adv. Sci. Comput. Eng., vol. 5, no. 3, pp. 257–268, Dec. 2023, doi: 10.62527/ijasce.5.3.177.
J. Pearl, “Causal inference in statistics: An overview,” Stat. Surv., vol. 3, no. none, Jan. 2009, doi: 10.1214/09-SS057.
S. Greenland, J. Pearl, and J. M. Robins, “Causal diagrams for epidemiologic research.,” Epidemiology, vol. 10, no. 1, pp. 37–48, Jan. 1999, doi: 10.1097/00001648-199901000-00008.
J. Pearl, Causality. Cambridge University Press, 2009. doi: 10.1017/CBO9780511803161.
J. Pearl and D. Mackenzie, The Book of Why: The New Science of Cause and Effect. 2020.
P. W. G. Tennant et al., “Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: review and recommendations,” Int. J. Epidemiol., vol. 50, no. 2, pp. 620–632, May 2021, doi: 10.1093/ije/dyaa213.
A. R. Nogueira, J. Gama, and C. A. Ferreira, “Causal discovery in machine learning: Theories and applications,” J. Dyn. Games, vol. 8, no. 3, p. 203, 2021, doi: 10.3934/jdg.2021008.
S. Balgi, A. Daoud, J. Pena, G. Wodtke, and J. Zhou, “Deep Learning with DAGs,” osf.io. Jan. 12, 2024. doi: 10.31235/osf.io/6375y.
B. Schölkopf, “Causality for Machine Learning,” in Probabilistic and Causal Inference, Probabilistic and C. Inference, Eds. New York, NY, USA: ACM, 2022, pp. 765–804. doi: 10.1145/3501714.3501755.
A. H. Petersen, J. Ramsey, C. T. Ekstrøm, and P. Spirtes, “Causal Discovery for Observational Sciences Using Supervised Machine Learning,” J. Data Sci., pp. 255–280, Mar. 2023, doi: 10.6339/23-JDS1088.
S. Garg, “Robust Machine Learning: Detection, Evaluation and Adaptation Under Distribution Shift,” Carnegie Mellon University, 2024. [Online]. Available: https://ml.cmu.edu/research/phd-dissertation-pdfs/sgarg2_mld_phd_2024.pdf
T. Freiesleben and T. Grote, “Beyond generalization: a theory of robustness in machine learning,” Synthese, vol. 202, no. 4, p. 109, Sep. 2023, doi: 10.1007/s11229-023-04334-9.
A. Aggarwal, P. Lohia, S. Nagar, K. Dey, and D. Saha, “Black box fairness testing of machine learning models,” in Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Aug. 2019, pp. 625–635. doi: 10.1145/3338906.3338937.
V. Hassija et al., “Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence,” Cognit. Comput., vol. 16, no. 1, pp. 45–74, Jan. 2024, doi: 10.1007/s12559-023-10179-8.
R. Guidotti, “Counterfactual explanations and how to find them: literature review and benchmarking,” Data Min. Knowl. Discov., vol. 38, no. 5, pp. 2770–2824, Sep. 2024, doi: 10.1007/s10618-022-00831-6.
R. Poyiadzi, K. Sokol, R. Santos-Rodriguez, T. De Bie, and P. Flach, “FACE: Feasible and Actionable Counterfactual Explanations,” in Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Feb. 2020, pp. 344–350. doi: 10.1145/3375627.3375850.
G. T. Ayem, A. Ajibesin, A. Iorliam, and A. S. Nsang, “Using Causal Constraints to Achieve Diverse Feasible and Actionable Counterfactual Explanations Predictions with SENSE-EGRA Dataset,” in American University of Nigeria, 1st International Conference Proceeding, 2023, pp. 209–224.
H. Zhang, “The Optimality of Naive Bayes,” in Proceedings of the Seventeenth International Florida Artificial Intelligence Research Society Conference, 2004. [Online]. Available: https://cdn.aaai.org/FLAIRS/2004/Flairs04-097.pdf
D. J. Hand and K. Yu, “Idiot’s Bayes—Not So Stupid After All?,” Int. Stat. Rev., vol. 69, no. 3, pp. 385–398, Dec. 2001, doi: 10.1111/j.1751-5823.2001.tb00465.x.
P. Domingos and M. Pazzani, “On the Optimality of the Simple Bayesian Classifier under Zero-One Loss,” Mach. Learn., vol. 29, no. 2–3, pp. 103–130, Nov. 1997, doi: 10.1023/A:1007413511361.
American University of Nigeria (AUN), "SENSE project evaluation: reading proficiency assessment, results of the hausa early grade reading assessment (EGRA) in Adamawa and Gombe," a report, 2021. [Online]. Available: https://github.com/Sadaju-Codes/SENSE-EGRA_Project/blob/main/SENSE_EGRA%20Evaluation%20report_20211026.pdf .
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Gabriel T. Ayem, Aondowase J. Shiwua, Peter T. Atagher, Terfa J. Tivkaa
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
