Reference. PRoofster: Automated Formal Verification
Cite
Cited by (2)
Baldur: Whole-Proof Generation and Repair with Large Language Models first-2023-baldur
Passport: Improving Automated Formal Verification Using Identifiers sanchezstern-2023-passport
Formally verifying system properties is one of the most effective ways of improving system quality, but its high manual effort requirements often render it prohibitively expensive. Tools that automate formal verification by learning from proof corpora to synthesize proofs have just begun to show their promise. These tools are effective because of the richness of the data the proof corpora contain. This richness comes from the stylistic conventions followed by communities of proof developers, together with the powerful logical systems beneath proof assistants. However, this richness remains underexploited, with most work thus far focusing on architecture rather than on how to make the most of the proof data. This article systematically explores how to most effectively exploit one aspect of that proof data: identifiers. We develop the Passport approach, a method for enriching the predictive Coq model used by an existing proof-synthesis tool with three new encoding mechanisms for identifiers: category vocabulary indexing, subword sequence modeling, and path elaboration. We evaluate our approach’s enrichment effect on three existing base tools: ASTactic, Tac, and Tok. In head-to-head comparisons, Passport automatically proves 29% more theorems than the best-performing of these base tools. Combining the three tools enhanced by the Passport approach automatically proves 38% more theorems than combining the three base tools. Finally, together, these base tools and their enhanced versions prove 45% more theorems than the combined base tools. Overall, our findings suggest that modeling identifiers can play a significant role in improving proof synthesis, leading to higher-quality software.
Cites 54 works (4 here)
With notes (4)
Passport: Improving Automated Formal Verification Using Identifiers sanchezstern-2023-passport
Formally verifying system properties is one of the most effective ways of improving system quality, but its high manual effort requirements often render it prohibitively expensive. Tools that automate formal verification by learning from proof corpora to synthesize proofs have just begun to show their promise. These tools are effective because of the richness of the data the proof corpora contain. This richness comes from the stylistic conventions followed by communities of proof developers, together with the powerful logical systems beneath proof assistants. However, this richness remains underexploited, with most work thus far focusing on architecture rather than on how to make the most of the proof data. This article systematically explores how to most effectively exploit one aspect of that proof data: identifiers. We develop the Passport approach, a method for enriching the predictive Coq model used by an existing proof-synthesis tool with three new encoding mechanisms for identifiers: category vocabulary indexing, subword sequence modeling, and path elaboration. We evaluate our approach’s enrichment effect on three existing base tools: ASTactic, Tac, and Tok. In head-to-head comparisons, Passport automatically proves 29% more theorems than the best-performing of these base tools. Combining the three tools enhanced by the Passport approach automatically proves 38% more theorems than combining the three base tools. Finally, together, these base tools and their enhanced versions prove 45% more theorems than the combined base tools. Overall, our findings suggest that modeling identifiers can play a significant role in improving proof synthesis, leading to higher-quality software.
Proof repair across type equivalences ringer-2021-proof
QED at Large: A Survey of Engineering of Formally Verified Software ringer-2019-qed
Development of formal proofs of correctness of programs can increase actual and perceived reliability and facilitate better understanding of program specifications and their underlying assumptions. Tools supporting such development have been available for over 40 years, but have only recently seen wide practical use. Projects based on construction of machine-checked formal proofs are now reaching an unprecedented scale, comparable to large software projects, which leads to new challenges in proof development and maintenance. Despite its increasing importance, the field of proof engineering is seldom considered in its own right; related theories, techniques, and tools span many fields and venues. This survey of the literature presents a holistic understanding of proof engineering for program correctness, covering impact in practice, foundations, proof automation, proof organization, and practical proof development.
Formal verification of a realistic compiler leroy_formal_2009
This paper reports on the development and formal verification (proof of semantic preservation) of CompCert, a compiler from Clight (a large subset of the C programming language) to PowerPC assembly code, using the Coq proof assistant both for programming the compiler and for proving its correctness. Such a verified compiler is useful in the context of critical software and its formal verification: the verification of the compiler guarantees that the safety properties proved on the source code hold for the executable compiled code as well.
External (50)
- Better automatic program repair by using bug reports and tests together (2023)
- Seldonian Toolkit: Building Software with Safe and Fair Machine Learning (2023)
- Diversity-driven automated formal verification (2022)
- Fairness guarantees under demographic shift (2022)
- Fairness testing: A comprehensive survey and analysis of trends (2022)
- Thor: Wielding hammers to integrate language models and automated theorem provers (2022)
- Bias mitigation for machine learning classifiers: A comprehensive survey (2022)
- LISA: Language models of ISAbelle proofs (2021)
- Proof Repair (PhD thesis, Ringer) (2021)
- TacTok: semantics-aware proof synthesis (2020)
- Generating correctness proofs with neural networks (2020)
- Tactic learning and proving for the Coq proof assistant (2020)
- Quality of Automated Program Repair on Real-World Defects (2020)
- Untangling mechanized proofs (2020)
- Performance improvements via formally-verified cryptography in Firefox (2020)
- The Cost of Poor Software Quality in the US: A 2020 Report (2020)
- Preventing undesirable behavior of intelligent machines (2019)
- Simple High-Level Code for Cryptographic Arithmetic - With Proofs, Without Compromises (2019)
- Learning to prove theorems via interacting with proof assistants (2019)
- Automated program repair (2019)
- Offline contextual bandits with high probability fairness guarantees (2019)
- Automatically Generating Precise Oracles from Structured Natural Language Specifications (2019)
- SOSRepair: Expressive Semantic Search for Real-World Program Repair (2019)
- GamePad: A learning environment for theorem proving (2019)
- Hammer for Coq: Automation for Dependent Type Theory (2018)
- On the naturalness of proofs (2018)
- Software fairness (2018)
- Themis: Automatically testing software for discrimination (2018)
- Deep Contextualized Word Representations (2018)
- Automatic formal verification for EPICS (2018)
- Front-end tooling for building and maintaining dependently-typed functional programs (PhD thesis, Robert) (2018)
- jsCoq: Towards hybrid theorem proving interfaces (2017)
- Fairness testing: testing software for discrimination (2017)
- A Type System for Privacy Properties (2017)
- The debugging mindset: Understanding the psychology of learning strategies leads to effective problem-solving skills (2017)
- FairSquare: probabilistic verification of program fairness (2017)
- The Coq Proof Assistant, version 8.7 (2017)
- Automatic generation of oracles for exceptional behaviors (2016)
- Verdi: a framework for implementing and formally verifying distributed systems (2015)
- Improved semantic representations from tree-structured long short-term memory networks (2015)
- Is the cure worse than the disease? overfitting in automated program repair (2015)
- An analysis of patch plausibility and correctness for generate-and-validate patch generation systems (2015)
- Industrial use of CompCert on a safety-critical software product (2014)
- Recycling Proof Patterns in Coq: Case Studies (2014)
- seL4: From General Purpose to a Proof of Information Flow Enforcement (2013)
- Machine learning in Proof General: Interfacing interfaces (2012)
- D3.js: Data-driven documents (Bostock) (2012)
- A Brief Overview of HOL4 (2008)
- Isabelle/HOL: A Proof Assistant for Higher-Order Logic (2002)
- AWS Provable Security