Reference. A typed, algebraic approach to parsing
In this paper, we recall the definition of the context-free expressions (or µ-regular expressions), an algebraic presentation of the context-free languages. Then, we define a core type system for the context-free expressions which gives a compositional criterion for identifying those context-free expressions which can be parsed unambiguously by predictive algorithms in the style of recursive descent or LL(1). Next, we show how these typed grammar expressions can be used to derive a parser combinator library which both guarantees linear-time parsing with no backtracking and single-token lookahead, and which respects the natural denotational semantics of context-free expressions. Finally, we show how to exploit the type information to write a staged version of this library, which produces dramatic increases in performance, even outperforming code generated by the standard parser generator tool ocamlyacc.
Cite
Cited by (3)
Definition. LL(1) Condition ll1-condition
A context-free grammar satisfies the LL(1) condition if it satisfies the following three conditions:
- All of its productions have pairwise disjoint first sets
- If a concatenation of nonterminals appears in a production, then has a disjoint followlast set from the first set of
- At most one production is nullable
This is essentially the type system of [1], which characterizes the LL(1) condition for context-free expressions.
Intutively, an LL(1) grammar can be parsed unambiguously, and without backtracking, by a predictive parser that only needs one token of lookahead.
flap: A Deterministic Parser with Fused Lexing yallop-2023-flap
Lexers and parsers are typically defined separately and connected by a token stream. This separate definition is important for modularity and reduces the potential for parsing ambiguity. However, materializing tokens as data structures and case-switching on tokens comes with a cost. We show how to fuse separately-defined lexers and parsers, drastically improving performance without compromising modularity or increasing ambiguity. We propose a deterministic variant of Greibach Normal Form that ensures deterministic parsing with a single token of lookahead and makes fusion strikingly simple, and prove that normalizing context free expressions into the deterministic normal form is semantics-preserving. Our staged parser combinator library, flap, provides a standard interface, but generates specialized token-free code that runs two to six times faster than ocamlyacc on a range of benchmarks.
Zippy LL(1) parsing with derivatives EdelmannZippy2020
In this paper, we present an efficient, functional, and formally verified parsing algorithm for LL(1) context-free expressions based on the concept of derivatives of formal languages. Parsing with derivatives is an elegant parsing technique, which, in the general case, suffers from cubic worst-case time complexity and slow performance in practice. We specialise the parsing with derivatives algorithm to LL(1) context-free expressions, where alternatives can be chosen given a single token of lookahead. We formalise the notion of LL(1) expressions and show how to efficiently check the LL(1) property. Next, we present a novel linear-time parsing with derivatives algorithm for LL(1) expressions operating on a zipper-inspired data structure. We prove the algorithm correct in Coq and present an implementation as a part of Scallion, a parser combinators framework in Scala with enumeration and pretty printing capabilities.
Cites 34 works (9 here)
With notes (9)
Productive coprogramming with guarded recursion atkey-2013-productive
Infinitary Axiomatization of the Equational Theory of Context-Free Languages grathwohl_infinitary_2013
We give a natural complete infinitary axiomatization of the equational theory of the context-free languages, answering a question of Lei\\textbackslashss\ (1992).
Context-Free Languages, Coalgebraically winterCFL
We give a coalgebraic account of context-free languages using the functor D(X) = 2 × XA for deterministic automata over an alphabet A, in three different but equivalent ways: (i) by viewing context-free grammars as D-coalgebras; (ii) by defining a format for behavioural differential equations (w.r.t. D) for which the unique solutions are precisely the context-free languages; and (iii) as the D-coalgebra of generalized regular expressions in which the Kleene star is replaced by a unique fixed point operator. In all cases, semantics is defined by the unique homomorphism into the final coalgebra of all languages, paving the way for coinductive proofs of context-free language equivalence. Furthermore, the three characterizations can serve as the basis for the definition of a general coalgebraic notion of context-freeness, which we see as the ultimate long-term goal of the present study.
Applicative programming with effects mcbride-2008-applicative
In this article, we introduce Applicative functors – an abstract characterisation of an applicative style of effectful programming, weaker than Monads and hence more widespread. Indeed, it is the ubiquity of this programming pattern that drew us to the abstraction. We retrace our steps in this article, introducing the applicative pattern by diverse examples, then abstracting it to define the Applicative type class and introducing a bracket notation that interprets the normal application syntax in the idiom of an Applicative functor. Furthermore, we develop the properties of applicative functors and the generic operations they support. We close by identifying the categorical structure of applicative functors and examining their relationship both with Monads and with Arrow.
Deterministic regular languages bruggemannkleinwood
The ISO standard for Standard Generalized Markup Language (SGML) provides a syntactic meta-language for the definition of textual markup systems. In the standard the right hand sides of productions are called content models and they are based on regular expressions. The allowable regular expressions are those that are “unambiguous” as defined by the standard. Unfortunately, the standard’s use of the term “unambiguous” does not correspond to the two well known notions, since not all regular languages are denoted by “unambiguous” expressions. Furthermore, the standard’s definition of “unambiguous” is somewhat vague. Therefore, we provide a precise definition of “unambiguous expressions” and rename them deterministic regular expressions to avoid any confusion. A regular expression E is deterministic if the canonical -free finite automaton recognizing L(E) is deterministic. A regular language is deterministic if there is a deterministic expression that denotes it. We give a Kleene-like theorem for deterministic regular languages and we characterize them in terms of the structural properties of the minimal deterministic automata recognizing them. The latter result enables us to decide if a given regular expression denotes a deterministic regular language and, if so, to construct an equivalent deterministic expression.
Towards Kleene Algebra with recursion leis_towards_1992
We extend Kozen’s theory KA of Kleene Algebra to axiomatize parts of the equational theory of context-free languages, using a least fixed-point operator μ instead of Kleene’s iteration operator*.
Programming Techniques: Regular expression search algorithm thompsonProgrammingTechniquesRegular1968
A method for locating specific character strings embedded in character text is described and an implementation of this method in the form of a compiler is discussed. The compiler accepts a regular expression as source language and produces an IBM 7094 program as object language. The object program then accepts the text to be searched as input and produces a signal every time an embedded string in the text matches the given regular expression. Examples, problems, and solutions are also presented.
On the translation of languages from left to right KNUTH1965607
There has been much recent interest in languages whose grammar is sufficiently simple that an efficient left-to-right parsing algorithm can be mechanically produced from the grammar. In this paper, we define LR(k) grammars, which are perhaps the most general ones of this type, and they provide the basis for understanding all of the special tricks which have been used in the construction of parsing algorithms for languages with simple structure, e.g. algebraic languages. We give algorithms for deciding if a given grammar satisfies the LR(k) condition, for given k, and also give methods for generating recognizes for LR(k) grammars. It is shown that the problem of whether or not a grammar is LR(k) for some k is undecidable, and the paper concludes by establishing various connections between LR(k) grammars and deterministic languages. In particular, the LR(k) condition is a natural analogue, for grammars, of the deterministic condition, for languages.
Derivatives of Regular Expressions brzozowskiDerivativesRegularExpressions1964
Kleene’s regular expressions, which can be used for describing sequential circuits, were defined using three operators (union, concatenation and iterate) on sets of sequences. Word descriptions of problems can be more easily put in the regular expression language if the language is enriched by the inclusion of other logical operations. However, in the problem of converting the regular expression description to a state diagram, the existing methods either cannot handle expressions with additional operators, or are made quite complicated by the presence of such operators.In this paper the notion of a derivative of a regular expression is introduced and the properties of derivatives are discussed. This leads, in a very natural way, to the construction of a state diagram from a regular expression containing any number of logical operators.
External (25)
- Generating mutually recursive definitions (2018)
- Menhir Reference Manual (2017)
- Supercompiling with Staging (2014)
- Staged parser combinators for efficient data processing (2014)
- The Design and Implementation of BER MetaOCaml (2014)
- Higher-order functional reactive programming without spacetime leaks (2013)
- Optimizing data structures in high-level programs (2013)
- Core_bench: Micro-Benchmarking for OCaml (2013)
- Unembedding domain-specific languages (2009)
- Parsing Techniques: A Practical Guide (2nd ed.) (2007)
- Algebraically Complete Semirings and Greibach Normal Form (2005)
- Generating LR syntax error messages from examples (2003)
- From Interpreter to Compiler and Virtual Machine: a Functional Derivation (2003)
- Generation of LR parsers by partial evaluation (2000)
- Multistage Programming: Its Theory and Applications (1999)
- Generalised Recursive Descent parsing and Follow-Determinism (1998)
- Eta-expansion does The Trick (1996)
- A modal analysis of staged computation (1996)
- Deterministic, Error-Correcting Combinator Parsers (1996)
- Call-by-name CPS-translation as a binding-time improvement (1995)
- Improving binding times without explicit CPS-conversion (1992)
- Higher-order functions for parsing (1992)
- On Kleene Algebras and Closed Semirings (1990)
- Programming Languages and Their Definition: Selected Papers of H. Bekic (1984)
- Syntax-Directed Transduction (1968)