Why the Proof Fails in Different Versions of Theorem Provers: An Empirical Study of Compatibility Issues in Isabelle
File version
Version of Record (VoR)
Author(s)
Sanan, David
Hou, Zhe
Xu, Qiyuan
Liu, Chengwei
Cai, Yufan
Liu, Yang
Sun, Meng
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
Size
File type(s)
Location
Trondheim, Norway
Abstract
Proof assistants are software tools for formal modeling and verification of software, hardware, design, and mathematical proofs. Due to the growing complexity and scale of formal proofs, compatibility issues frequently arise when using different versions of proof assistants. These issues result in broken proofs, disrupting the maintenance of formalized theories and hindering the broader dissemination of results within the community. Although existing works have proposed techniques to address specific types of compatibility issues, the overall characteristics of these issues remain largely unexplored. To address this gap, we conduct the first extensive empirical study to characterize compatibility issues, using Isabelle as a case study. We develop a regression testing framework to automatically collect compatibility issues from the Archive of Formal Proofs, the largest repository of formal proofs in Isabelle. By analyzing 12,079 collected issues, we identify their types and symptoms and further investigate their root causes. We also extract updated proofs that address these issues to understand the applied resolution strategies. Our study provides an in-depth understanding of compatibility issues in proof assistants, offering insights that support the development of effective techniques to mitigate these issues.
Journal Title
Proceedings of the ACM on Software Engineering
Conference Title
Proceedings of the ACM on Software Engineering
Book Title
Edition
Volume
2
Issue
FSE
Thesis Type
Degree Program
School
Publisher link
DOI
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement
© 2025 Copyright held by the owner/author(s). This work is licensed under a Creative Commons Attribution 4.0 International License.
Item Access Status
Note
Access the data
Related item(s)
Subject
Persistent link to this record
Citation
Luan, X; Sanan, D; Hou, Z; Xu, Q; Liu, C; Cai, Y; Liu, Y; Sun, M, Why the Proof Fails in Different Versions of Theorem Provers: An Empirical Study of Compatibility Issues in Isabelle, Proceedings of the ACM on Software Engineering, 2025, 2 (FSE), pp. 1499-1521