Why the Proof Fails in Different Versions of Theorem Provers: An Empirical Study of Compatibility Issues in Isabelle

Loading...
Thumbnail Image
File version

Version of Record (VoR)

Author(s)
Luan, Xiaokun
Sanan, David
Hou, Zhe
Xu, Qiyuan
Liu, Chengwei
Cai, Yufan
Liu, Yang
Sun, Meng
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
2025
Size
File type(s)
Location

Trondheim, Norway

Abstract

Proof assistants are software tools for formal modeling and verification of software, hardware, design, and mathematical proofs. Due to the growing complexity and scale of formal proofs, compatibility issues frequently arise when using different versions of proof assistants. These issues result in broken proofs, disrupting the maintenance of formalized theories and hindering the broader dissemination of results within the community. Although existing works have proposed techniques to address specific types of compatibility issues, the overall characteristics of these issues remain largely unexplored. To address this gap, we conduct the first extensive empirical study to characterize compatibility issues, using Isabelle as a case study. We develop a regression testing framework to automatically collect compatibility issues from the Archive of Formal Proofs, the largest repository of formal proofs in Isabelle. By analyzing 12,079 collected issues, we identify their types and symptoms and further investigate their root causes. We also extract updated proofs that address these issues to understand the applied resolution strategies. Our study provides an in-depth understanding of compatibility issues in proof assistants, offering insights that support the development of effective techniques to mitigate these issues.

Journal Title

Proceedings of the ACM on Software Engineering

Conference Title

Proceedings of the ACM on Software Engineering

Book Title
Edition
Volume

2

Issue

FSE

Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement

© 2025 Copyright held by the owner/author(s). This work is licensed under a Creative Commons Attribution 4.0 International License.

Item Access Status
Note
Access the data
Related item(s)
Subject
Persistent link to this record
Citation

Luan, X; Sanan, D; Hou, Z; Xu, Q; Liu, C; Cai, Y; Liu, Y; Sun, M, Why the Proof Fails in Different Versions of Theorem Provers: An Empirical Study of Compatibility Issues in Isabelle, Proceedings of the ACM on Software Engineering, 2025, 2 (FSE), pp. 1499-1521