Reference. AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement

Automated code generation with large language models has gained significant traction, but there remains no guarantee on the correctness of generated code. We aim to use formal verification to provide mathematical guarantees that the generated code is correct. However, generating formally verified code with LLMs is hindered by the scarcity of training data and the complexity of formal proofs. To tackle this challenge, we introduce AlphaVerus, a self-improving framework that bootstraps formally verified code generation by iteratively translating programs from a higher-resource language and leveraging feedback from a verifier. AlphaVerus operates in three phases: exploration of candidate translations, Treefinement – a novel tree search algorithm for program refinement using verifier feedback, and filtering misaligned specifications and programs to prevent reward hacking. Through this iterative process, AlphaVerus enables a LLaMA-3.1-70B model to generate verified code without human intervention or model finetuning. AlphaVerus shows an ability to generate formally verified solutions for HumanEval and MBPP, laying the groundwork for truly trustworthy code-generation agents.

Cite

Cite as @aggarwal-2024-alphaverus (helia, typst) · \cite{aggarwal-2024-alphaverus} (LaTeX)
BibTeX
bibtex · 8 lines
@misc{aggarwal-2024-alphaverus,
  author = {Pranjal Aggarwal and Bryan Parno and Sean Welleck},
  title = {AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement},
  year = {2024},
  month = {12},
  eprint = {2412.06176},
  archiveprefix = {arXiv}
}
hayagriva YAML (typst)
yaml · 10 lines
aggarwal-2024-alphaverus:
  type: misc
  title: 'AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement'
  author:
  - Aggarwal, Pranjal
  - Parno, Bryan
  - Welleck, Sean
  date: 2024-12
  serial-number:
    arxiv: '2412.06176'
Cites 61 works (2 here)
With notes (2)

Baldur: Whole-Proof Generation and Repair with Large Language Models first-2023-baldur

PDF · DOI · pldb

Verus: Verifying Rust Programs using Linear Ghost Types lattuada-2023-verus

The Rust programming language provides a powerful type system that checks linearity and borrowing, allowing code to safely manipulate memory without garbage collection and making Rust ideal for developing low-level, high-assurance systems. For such systems, formal verification can be useful to prove functional correctness properties beyond type safety. This paper presents Verus, an SMT-based tool for formally verifying Rust programs. With Verus, programmers express proofs and specifications using the Rust language, allowing proofs to take advantage of Rust’s linear types and borrow checking. We show how this allows proofs to manipulate linearly typed permissions that let Rust code safely manipulate memory, pointers, and concurrent resources. Verus organizes proofs and specifications using a novel mode system that distinguishes specifications, which are not checked for linearity and borrowing, from executable code and proofs, which are checked for linearity and borrowing. We formalize Verus’ linearity, borrowing, and modes in a small lambda calculus, for which we prove type safety and termination of specifications and proofs. We demonstrate Verus on a series of examples, including pointer-manipulating code (an xor-based doubly linked list), code with interior mutability, and concurrent code.
PDF · DOI · arXiv · pldb
External (59)
aggarwal-2024-alphaverus reference entries/refs/aggarwal-2024-alphaverus/aggarwal-2024-alphaverus.hel