Reference. Fair enumeration combinators
Enumerations represented as bijections between the natural numbers and elements of some given type have recently garnered interest in property-based testing because of their efficiency and flexibility. There are, however, many ways of defining these bijections, some of which are better than others. This paper offers a new property of enumeration combinators called fairness that identifies enumeration combinators that are better suited to property-based testing. Intuitively, the result of a fair combinator indexes into its argument enumerations equally when constructing its result. For example, extracting the nth element from our enumeration of three-tuples indexes about elements into each of its components instead of, say, indexing into one and into the other two, as you would if a three-tuple were built out of nested pairs. Similarly, extracting the nth element from our enumeration of a three-way union returns an element that is into one of the argument enumerators. The paper presents a semantics of enumeration combinators, a theory of fairness, proofs establishing fairness of our new combinators and that some combinations of fair combinators are not fair. We also report on an evaluation of fairness for the purpose of finding bugs in programming-language models. We show that fair enumeration combinators have complementary strengths to an existing, well-tuned ad hoc random generator (better on short time scales and worse on long time scales) and that using unfair combinators is worse across the board.
Cite
Cites 21 works (1 here)
With notes (1)
Finding and Understanding Bugs in C Compilers yangFindingUnderstandingBugs
Compilers should be correct. To improve the quality of C compilers, we created Csmith, a randomized test-case generation tool, and spent three years using it to find compiler bugs. During this period we reported more than 325 previously unknown bugs to compiler developers. Every compiler we tested was found to crash and also to silently generate wrong code when presented with valid input. In this paper we present our compiler-testing tool and the results of our bug-hunting study. Our first contribution is to advance the state of the art in compiler testing. Unlike previous tools, Csmith generates programs that cover a large subset of C while avoiding the undefined and unspecified behaviors that would destroy its ability to automatically find wrong-code bugs. Our second contribution is a collection of qualitative and quantitative results about the bugs we have found in open-source C compilers.
External (20)
- Programming with Enumerable Sets of Structures (2015)
- SciFe: Scala Framework for Efficient Enumeration of Data Structures with Invariants (2014)
- The Racket virtual machine and randomized testing (2013)
- Constraining Delimited Control with Contracts (2013)
- Compact Serialization of Prolog Terms (with Catalan Skeletons, Cantor Tupling and Gödel Numberings) (2013)
- A list-machine benchmark for mechanized metatheory (2012)
- The New Quickcheck for Isabelle: Random, Exhaustive and Symbolic Testing Under One Roof (2012)
- Feat: Functional Enumeration of Algebraic Types (2012)
- Run Your Research: On the Effectiveness of Lightweight Mechanization (2012)
- Testing an Optimising Compiler by Generating Random Lambda Terms (2012)
- Deriving a Fast Inverse of the Generalized Cantor N-tupling Bijection (2012)
- Testing an Optimising Compiler by Generating Random Lambda Terms (2011)
- Every Bit Counts: The binary representation of typed data and programs (2010)
- Semantics Engineering with PLT Redex (2009)
- Randomized Testing in PLT Redex (2009)
- SmallCheck and Lazy SmallCheck: Automatic Exhaustive Testing for Small Values (2008)
- An Elegant Pairing Function (2006)
- A Visual Environment for Developing Context-Sensitive Term Rewriting Systems (2004)
- QuickCheck: a lightweight tool for random testing of Haskell programs (2000)
- An Empirical Study of the Reliability of UNIX Utilities (1990)