Reference. A typed, algebraic approach to parsing

In this paper, we recall the definition of the context-free expressions (or µ-regular expressions), an algebraic presentation of the context-free languages. Then, we define a core type system for the context-free expressions which gives a compositional criterion for identifying those context-free expressions which can be parsed unambiguously by predictive algorithms in the style of recursive descent or LL(1). Next, we show how these typed grammar expressions can be used to derive a parser combinator library which both guarantees linear-time parsing with no backtracking and single-token lookahead, and which respects the natural denotational semantics of context-free expressions. Finally, we show how to exploit the type information to write a staged version of this library, which produces dramatic increases in performance, even outperforming code generated by the standard parser generator tool ocamlyacc.

Cite

Cite as @krishnaswami_typed_2019 (helia, typst) · \cite{krishnaswami_typed_2019} (LaTeX)
BibTeX
bibtex · 15 lines
@inproceedings{krishnaswami_typed_2019,
 title = {A typed, algebraic approach to parsing},
 author = {Krishnaswami, Neelakantan R. and Yallop, Jeremy},
 year = {2019},
 isbn = {978-1-4503-6712-7},
 doi = {10.1145/3314221.3314625},
 url = {https://dl.acm.org/doi/10.1145/3314221.3314625},
 urldate = {2024-04-22},
 booktitle = {Proceedings of the 40th {ACM} {SIGPLAN} {Conference} on {Programming} {Language} {Design} and {Implementation}},
 pages = {379--393},
 publisher = {ACM},
 address = {Phoenix AZ USA},
 month = {June},
 language = {en}
}
hayagriva YAML (typst)
yaml · 20 lines
krishnaswami_typed_2019:
  type: article
  title: A typed, algebraic approach to parsing
  author:
  - Krishnaswami, Neelakantan R.
  - Yallop, Jeremy
  date: 2019-06
  page-range: 379-393
  url:
    value: https://dl.acm.org/doi/10.1145/3314221.3314625
    date: 2024-04-22
  serial-number:
    doi: 10.1145/3314221.3314625
    isbn: 978-1-4503-6712-7
  parent:
    type: proceedings
    title: Proceedings of the 40th {ACM} {SIGPLAN} {Conference} on {Programming} {Language} {Design} and {Implementation}
    publisher:
      name: ACM
      location: Phoenix AZ USA
Cited by (3)

Definition. LL(1) Condition ll1-condition

A context-free grammar satisfies the LL(1) condition if it satisfies the following three conditions:

  • All of its productions have pairwise disjoint first sets
  • If a concatenation of nonterminals 𝐴⊗𝐵 appears in a production, then 𝐴 has a disjoint followlast set from the first set of 𝐵
  • At most one production is nullable

This is essentially the type system of [1], which characterizes the LL(1) condition for context-free expressions.

Intutively, an LL(1) grammar can be parsed unambiguously, and without backtracking, by a predictive parser that only needs one token of lookahead.

flap: A Deterministic Parser with Fused Lexing yallop-2023-flap

Lexers and parsers are typically defined separately and connected by a token stream. This separate definition is important for modularity and reduces the potential for parsing ambiguity. However, materializing tokens as data structures and case-switching on tokens comes with a cost. We show how to fuse separately-defined lexers and parsers, drastically improving performance without compromising modularity or increasing ambiguity. We propose a deterministic variant of Greibach Normal Form that ensures deterministic parsing with a single token of lookahead and makes fusion strikingly simple, and prove that normalizing context free expressions into the deterministic normal form is semantics-preserving. Our staged parser combinator library, flap, provides a standard interface, but generates specialized token-free code that runs two to six times faster than ocamlyacc on a range of benchmarks.
PDF · DOI · arXiv · pldb

Zippy LL(1) parsing with derivatives EdelmannZippy2020

In this paper, we present an efficient, functional, and formally verified parsing algorithm for LL(1) context-free expressions based on the concept of derivatives of formal languages. Parsing with derivatives is an elegant parsing technique, which, in the general case, suffers from cubic worst-case time complexity and slow performance in practice. We specialise the parsing with derivatives algorithm to LL(1) context-free expressions, where alternatives can be chosen given a single token of lookahead. We formalise the notion of LL(1) expressions and show how to efficiently check the LL(1) property. Next, we present a novel linear-time parsing with derivatives algorithm for LL(1) expressions operating on a zipper-inspired data structure. We prove the algorithm correct in Coq and present an implementation as a part of Scallion, a parser combinators framework in Scala with enumeration and pretty printing capabilities.
DOI · pldb
Cites 34 works (9 here)
With notes (9)

Productive coprogramming with guarded recursion atkey-2013-productive

PDF · DOI · pldb

Infinitary Axiomatization of the Equational Theory of Context-Free Languages grathwohl_infinitary_2013

We give a natural complete infinitary axiomatization of the equational theory of the context-free languages, answering a question of Lei\\textbackslashss\ (1992).
DOI

Context-Free Languages, Coalgebraically winterCFL

We give a coalgebraic account of context-free languages using the functor D(X) = 2 × XA for deterministic automata over an alphabet A, in three different but equivalent ways: (i) by viewing context-free grammars as D-coalgebras; (ii) by defining a format for behavioural differential equations (w.r.t. D) for which the unique solutions are precisely the context-free languages; and (iii) as the D-coalgebra of generalized regular expressions in which the Kleene star is replaced by a unique fixed point operator. In all cases, semantics is defined by the unique homomorphism into the final coalgebra of all languages, paving the way for coinductive proofs of context-free language equivalence. Furthermore, the three characterizations can serve as the basis for the definition of a general coalgebraic notion of context-freeness, which we see as the ultimate long-term goal of the present study.
DOI

Applicative programming with effects mcbride-2008-applicative

In this article, we introduce Applicative functors – an abstract characterisation of an applicative style of effectful programming, weaker than Monads and hence more widespread. Indeed, it is the ubiquity of this programming pattern that drew us to the abstraction. We retrace our steps in this article, introducing the applicative pattern by diverse examples, then abstracting it to define the Applicative type class and introducing a bracket notation that interprets the normal application syntax in the idiom of an Applicative functor. Furthermore, we develop the properties of applicative functors and the generic operations they support. We close by identifying the categorical structure of applicative functors and examining their relationship both with Monads and with Arrow.
PDF · DOI · pldb

Deterministic regular languages bruggemannkleinwood

The ISO standard for Standard Generalized Markup Language (SGML) provides a syntactic meta-language for the definition of textual markup systems. In the standard the right hand sides of productions are called content models and they are based on regular expressions. The allowable regular expressions are those that are “unambiguous” as defined by the standard. Unfortunately, the standard’s use of the term “unambiguous” does not correspond to the two well known notions, since not all regular languages are denoted by “unambiguous” expressions. Furthermore, the standard’s definition of “unambiguous” is somewhat vague. Therefore, we provide a precise definition of “unambiguous expressions” and rename them deterministic regular expressions to avoid any confusion. A regular expression E is deterministic if the canonical 𝜀-free finite automaton 𝑀𝐸 recognizing L(E) is deterministic. A regular language is deterministic if there is a deterministic expression that denotes it. We give a Kleene-like theorem for deterministic regular languages and we characterize them in terms of the structural properties of the minimal deterministic automata recognizing them. The latter result enables us to decide if a given regular expression denotes a deterministic regular language and, if so, to construct an equivalent deterministic expression.
DOI

Towards Kleene Algebra with recursion leis_towards_1992

We extend Kozen’s theory KA of Kleene Algebra to axiomatize parts of the equational theory of context-free languages, using a least fixed-point operator μ instead of Kleene’s iteration operator*.
DOI

Programming Techniques: Regular expression search algorithm thompsonProgrammingTechniquesRegular1968

A method for locating specific character strings embedded in character text is described and an implementation of this method in the form of a compiler is discussed. The compiler accepts a regular expression as source language and produces an IBM 7094 program as object language. The object program then accepts the text to be searched as input and produces a signal every time an embedded string in the text matches the given regular expression. Examples, problems, and solutions are also presented.
DOI

On the translation of languages from left to right KNUTH1965607

There has been much recent interest in languages whose grammar is sufficiently simple that an efficient left-to-right parsing algorithm can be mechanically produced from the grammar. In this paper, we define LR(k) grammars, which are perhaps the most general ones of this type, and they provide the basis for understanding all of the special tricks which have been used in the construction of parsing algorithms for languages with simple structure, e.g. algebraic languages. We give algorithms for deciding if a given grammar satisfies the LR(k) condition, for given k, and also give methods for generating recognizes for LR(k) grammars. It is shown that the problem of whether or not a grammar is LR(k) for some k is undecidable, and the paper concludes by establishing various connections between LR(k) grammars and deterministic languages. In particular, the LR(k) condition is a natural analogue, for grammars, of the deterministic condition, for languages.
DOI

Derivatives of Regular Expressions brzozowskiDerivativesRegularExpressions1964

Kleene’s regular expressions, which can be used for describing sequential circuits, were defined using three operators (union, concatenation and iterate) on sets of sequences. Word descriptions of problems can be more easily put in the regular expression language if the language is enriched by the inclusion of other logical operations. However, in the problem of converting the regular expression description to a state diagram, the existing methods either cannot handle expressions with additional operators, or are made quite complicated by the presence of such operators.In this paper the notion of a derivative of a regular expression is introduced and the properties of derivatives are discussed. This leads, in a very natural way, to the construction of a state diagram from a regular expression containing any number of logical operators.
DOI
krishnaswami_typed_2019 reference entries/refs/krishnaswami_typed_2019/krishnaswami_typed_2019.hel