Reference. Distribution Theoretic Semantics for Non-Smooth Differentiable Programming

Pedro H. Azevedo de Amorim, Christopher Lam · arXiv · 2022 · differentiable-programming denotational-semantics preprint · Web · arXiv

With the wide spread of deep learning and gradient descent inspired optimization algorithms, differentiable programming has gained traction. Nowadays it has found applications in many different areas as well, such as scientific computing, robotics, computer graphics and others. One of its notoriously difficult problems consists in interpreting programs that are not differentiable everywhere.

In this work we define 𝜆𝛿, a core calculus for non-smooth differentiable programs and define its semantics using concepts from distribution theory, a well-established area of functional analysis. We also show how 𝜆𝛿 presents better equational properties than other existing semantics and use our semantics to reason about a simplified ray tracing algorithm. Further, we relate our semantics to existing differentiable languages by providing translations to and from other existing differentiable semantic models. Finally, we provide a proof-of-concept implementation in PyTorch of the novel constructions in this paper.

Cite
Cite as @amorim_lam_2022 (helia, typst) · \cite{amorim_lam_2022} (LaTeX)
BibTeX
bibtex · 8 lines
@unpublished{amorim_lam_2022,
 title = {Distribution Theoretic Semantics for Non-Smooth Differentiable Programming},
 author = {Amorim, Pedro H. Azevedo de and Lam, Christopher},
 year = {2022},
 url = {https://arxiv.org/abs/2207.05946},
 eprinttype = {arXiv},
 eprint = {2207.05946}
}
hayagriva YAML (typst)
yaml · 10 lines
amorim_lam_2022:
  type: manuscript
  title: Distribution Theoretic Semantics for Non-Smooth Differentiable Programming
  author:
  - Amorim, Pedro H. Azevedo de
  - Lam, Christopher
  date: 2022
  url: https://arxiv.org/abs/2207.05946
  serial-number:
    arxiv: '2207.05946'
Download: amorim_lam_2022.bib · amorim_lam_2022.yml
Cites 29 works (2 here)
With notes (2)

A convenient category for higher-order probability theory heunen-2017-aLICS · 2017
DOI · arXiv

Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints staton-2016-semanticsLICS · 2016
DOI · arXiv
External (27)
amorim_lam_2022 reference entries/refs/amorim_lam_2022/amorim_lam_2022.hel