All Notes

Definition. Representable Grammars representable-grammar

For any string 𝑀, we can define a representable grammar βŒˆπ‘€βŒ‰ which matches exactly the string 𝑀 and nothing else. The parse trees for a representable grammar are proofs that the string is exactly equal to 𝑀:

βŒˆπ‘€βŒ‰(𝑒)=(𝑒≑𝑀)

Predecessors simplify later predecessors-simplify-later

A predecessor of π‘₯ is a top element of its strict downset: a strict morphism 𝜌:𝑝→π‘₯ through which every strict morphism into π‘₯ factors uniquely. Equivalently, π–²π—π—‹π—‚π–Όπ—π–£π—ˆπ—π—‡(π‘₯)≅𝗒𝑝 β€” the downset is representable.

The Yoneda lemma then collapses later to evaluation:

βŠ³π‘ƒ(π‘₯)=π–―π—Œπ—π’žοΈ€(π–²π—π—‹π—‚π–Όπ—π–£π—ˆπ—π—‡(π‘₯),𝑃)β‰…π–―π—Œπ—π’žοΈ€(𝗒𝑝,𝑃)≅𝑃(𝑝),

with 𝗇𝖾𝗑𝗍 becoming restriction along 𝜌. The name is from the naturals: every strict map into 𝑛+1 factors through 𝑛→𝑛+1, so on πœ” β€” in the topos of trees β€” later is just the shift

βŠ³π‘ƒ(0)β‰…βŠ€,βŠ³π‘ƒ(𝑛+1)≅𝑃(𝑛),

and a LΓΆb step is a base value together with a rule producing the value at 𝑛+1 from the value at 𝑛.

When this predecessor exists, we can give a simpler description of later, as in the topos of trees, but this may not be possible in all direct categories.

Algebras as a displayed category algebras-displayed

Fix an endofunctor 𝐹:π’žοΈ€β†’π’žοΈ€. The 𝐹-algebras form a displayed category AlgStr(𝐹) over π’žοΈ€.

Over an object π‘₯, a displayed object of AlgStr(𝐹) is a structure map

π›Όβˆˆπ’žοΈ€(𝐹π‘₯,π‘₯).

Over 𝑓:π‘₯→𝑦, a displayed morphism from 𝛼 to 𝛽 is the proposition that 𝑓 is an algebra homomorphism:

𝛼⋆𝑓=𝐹𝑓⋆𝛽.

The total category Alg(𝐹)=∫AlgStr(𝐹) is the category of 𝐹-algebras.

The co-Eilenberg–Moore category as a displayed category co-eilenberg-moore-displayed

A comonad on π’žοΈ€ is a monad π‘Š on π’žοΈ€op. Everything about its coalgebras is then inherited from the Eilenberg–Moore construction, instantiated at the opposite category β€” nothing is defined twice.

Algebras of π‘Š over π’žοΈ€op are coalgebras π›Ύβˆˆπ’žοΈ€(π‘₯,π‘Šπ‘₯) of the underlying endofunctor, and the monad algebra laws, read in π’žοΈ€op, are the comonad coalgebra laws β€” the unit and multiplication of π‘Š, viewed in π’žοΈ€, are the counit πœ€ and comultiplication 𝛿:

π›Ύβ‹†πœ€π‘₯=𝗂𝖽π‘₯𝛾⋆𝛿π‘₯=π›Ύβ‹†π‘Šπ›Ύ.

The co-Eilenberg–Moore category is the opposite of the total category:

coEM(π‘Š)=(EM(π‘Š))op.

Coalgebras as a displayed category coalgebras-displayed

Fix an endofunctor 𝐹:π’žοΈ€β†’π’žοΈ€. Coalgebras require no new construction: a coalgebra is an algebra in the opposite category. Define

CoalgStr(𝐹)=AlgStr(𝐹op),

a displayed category over π’žοΈ€op, where 𝐹op:π’žοΈ€opβ†’π’žοΈ€op is 𝐹 acting on the opposite category.

Concretely, over an object π‘₯ a displayed object is a structure map

π›Ύβˆˆπ’žοΈ€(π‘₯,𝐹π‘₯).

The category of coalgebras is the opposite of the total category:

Coalg(𝐹)=(∫CoalgStr(𝐹))op.

The outer opposite returns morphisms to the direction of π’žοΈ€: a morphism (π‘₯,𝛾)β†’(𝑦,𝛿) is a map 𝑓:π‘₯→𝑦 with

𝑓⋆𝛿=𝛾⋆𝐹𝑓.

Definition. The comparison functor of an adjunction comparison-functor

An adjunction πΉβŠ£π‘ˆ with 𝐹:π’žοΈ€β†’π’ŸοΈ€ and π‘ˆ:π’ŸοΈ€β†’π’žοΈ€ induces a monad 𝑇=π‘ˆβˆ˜πΉ on π’žοΈ€. Write πœ€:πΉβˆ˜π‘ˆβ‡’π–¨π–½ for the counit of the adjunction. Every object 𝑑 of π’ŸοΈ€ then induces a 𝑇-algebra carried by the object π‘ˆπ‘‘, witnessed by the map

π‘ˆπœ€π‘‘:π‘ˆπΉπ‘ˆπ‘‘β†’π‘ˆπ‘‘.

This assignment extends to a functor into the Eilenberg–Moore category,

𝐾:π’ŸοΈ€β†’EM(𝑇),

the comparison functor of the adjunction.

Dually, an adjunction induces a comonad on the other side and a comparison into the co-Eilenberg–Moore category. When these comparisons are equivalences we say that the adjunction πΉβŠ£π‘ˆ is (co)monadic.

Definition. Direct categories direct-category

A well-founded order is a set 𝐷 with a proposition-valued transitive relation < admitting no infinite descent: every element is accessible. Write π‘Žβ‰€π‘ for (π‘Ž<𝑏)∨(π‘Ž=𝑏).

A direct structure on a category π’žοΈ€ over (𝐷,<) is a functor

deg:π’žοΈ€β†’(𝐷,≀)

into the well-founded order viewed as a poset category. The functor organizes two pieces of data at once: an ordering on the objects, and the invariant that morphisms respect it β€” 𝑓:π‘₯→𝑦 forces degπ‘₯≀deg𝑦.

A direct structure equips the objects with a well-founded strict relation

π‘₯β‰Ίπ‘¦βŸΊdegπ‘₯<deg𝑦.

Intuitively, direct categories are the right generalization of well-foundedness to the categorical setting: a direct category is essentially one whose underlying graph is a directed acyclic graph, layered by degree, so that data at an object may be defined by recursion from data at all objects strictly below it.

The degrees order the objects, while the morphisms of π’žοΈ€ say how an object sits over its predecessors.

Definition. Earlier on presheaves earlier-presheaf

Later takes a limit over smaller indices. Dually, the earlier modality takes a colimit over larger indices: an element of βŠ²π‘ƒ at π‘₯ is a 𝑃-element sitting at some object 𝑦 strictly above π‘₯, carried down along a chosen morphism π‘₯→𝑦.

Earlier is left adjoint to later:

⊲⊣⊳

Under this adjunction, 𝗇𝖾𝗑𝗍:π‘ƒβ†’βŠ³π‘ƒ corresponds to

𝗉𝗋𝖾𝗏:βŠ²π‘ƒβ†’π‘ƒ,

.

The Eilenberg–Moore category as a displayed category eilenberg-moore-displayed

Fix a monad (𝑇,πœ‚,πœ‡) on π’žοΈ€. Its Eilenberg–Moore category arises in two displayed layers. The first layer is the displayed category of algebras AlgStr(𝑇) of the underlying endofunctor.

The second layer, EMStr(𝑇), is displayed over the total category Alg(𝑇). Over an algebra (π‘₯,𝛼) the displayed objects are the propositions that 𝛼 satisfies the monad algebra laws:

πœ‚π‘₯⋆𝛼=𝗂𝖽π‘₯πœ‡π‘₯⋆𝛼=𝑇𝛼⋆𝛼.

The Eilenberg–Moore category is the total category of the tower:

EM(𝑇)=∫EMStr(𝑇).

Definition. Initial algebra initial-algebra

Fix an endofunctor 𝐹:π’žοΈ€β†’π’žοΈ€. An initial 𝐹-algebra, written πœ‡πΉ, is an initial object of the category of algebras Alg(𝐹).

Unfolding the universal property: an initial algebra is an algebra (πœ‡πΉ,in) such that every algebra (π‘₯,𝛼) admits a unique morphism fold𝛼:πœ‡πΉβ†’π‘₯ satisfying

in⋆(fold𝛼)=𝐹(fold𝛼)⋆𝛼.

Definition. Later on families later-family

Conjugation with the adjunction between presheaves and families π‘ˆβŠ£Cofree lets us induce a later construction on families from the one on presheaves,

⊳Fam=π‘ˆβˆ˜βŠ³βˆ˜Cofree:Fam(π’žοΈ€)β†’Fam(π’žοΈ€).

Concretely, later on families evaluates to

⊳Fam𝐴(π‘₯)β‰…βˆπ‘¦β‰Ίπ‘₯Β βˆπ‘“:𝑦→π‘₯𝐴(𝑦)

Definition. Later on presheaves later-presheaf

The later modality for presheaves on a direct category is given by the presheaf of natural transformations

out of the strict downset.

(βŠ³π‘ƒ)(π‘₯)=π–―π—Œπ—π’žοΈ€(π–²π—π—‹π—‚π–Όπ—π–£π—ˆπ—π—‡(π‘₯),𝑃).

An element of βŠ³π‘ƒ at π‘₯ is a coherent choice of 𝑃-elements at all objects strictly smaller than π‘₯.

Restriction in βŠ³π‘ƒ along 𝑓:𝑦→π‘₯ precomposes with the induced map π–²π—π—‹π—‚π–Όπ—π–£π—ˆπ—π—‡(𝑦)β†’π–²π—π—‹π—‚π–Όπ—π–£π—ˆπ—π—‡(π‘₯). At an object of minimal degree the strict downset is empty, so βŠ³π‘ƒ is trivial there.

Via functoriality, every presheaf restricts to smaller indices. Thus we may define the map

𝗇𝖾𝗑𝗍:π‘ƒβ†’βŠ³π‘ƒ

that sends an element 𝑝 over π‘₯ to the family of all its restrictions along morphisms from strictly lower objects.

Theorem. LΓΆb induction on families lob-family

Like later on families, the recursion principle for families is inherited from that on presheaves. Given a family 𝐴 and a step

πœ‘π‘₯:⊳Fam𝐴(π‘₯)→𝐴(π‘₯)for each π‘₯,

the construction is a chain of transpositions with LΓΆb for presheaves used in the middle:

Just as for presheaves, the fixed point constructed above is unique: the two transpositions are bijections, and the presheaf-level fixed point is already unique.

Theorem. LΓΆb induction for presheaves on a direct category lob-presheaf

Let π’žοΈ€ be a direct category and 𝑃 a presheaf on it. Every map

πœ‘:βŠ³π‘ƒβ†’π‘ƒ

has a fixed point: a global element π—…ΓΆπ–»πœ‘:βŠ€β†’π‘ƒ with

π—…ΓΆπ–»πœ‘=π—…ΓΆπ–»πœ‘β‹†π—‡π–Ύπ—‘π—β‹†πœ‘,

and this fixed point is unique.

The hypothesis says: the value of 𝑃 at any object is determined by its values over the strict past β€” πœ‘ turns a coherent family over the strict downset of π‘₯ into a value at π‘₯. The proof is recursion along the well-founded β‰Ί: at each π‘₯, the section already constructed over the past assembles into an element of (βŠ³π‘ƒ)(π‘₯), and πœ‘ extends it to π‘₯.

Definition. Locally contractive endofunctors locally-contractive-functor

Write π‘‹β‡’π‘Œ for the presheaf of morphisms π‘‹β†’π‘Œ. An endofunctor 𝐹 on presheaves is locally contractive when its action on morphisms factors through later: there is a map

𝐹𝛿:⊳(π‘‹β‡’π‘Œ)β†’(πΉπ‘‹β‡’πΉπ‘Œ)

Definition. Monadicity and comonadicity monadicity-comonadicity

An adjunction πΉβŠ£π‘ˆ with 𝐹:π’žοΈ€β†’π’ŸοΈ€ and π‘ˆ:π’ŸοΈ€β†’π’žοΈ€ induces a monad 𝑇=π‘ˆβˆ˜πΉ on π’žοΈ€, and a comparison functor

𝐾:π’ŸοΈ€β†’EM(𝑇)

sending each object of π’ŸοΈ€ to the 𝑇-algebra it carries.

The functor π‘ˆ is monadic when 𝐾 is an equivalence: the adjunction exhibits π’ŸοΈ€ as objects of π’žοΈ€ equipped with algebraic structure for 𝑇, the Eilenberg–Moore category.

Comonadicity is monadicity in the opposite category: a left adjoint 𝐿:π’ŸοΈ€β†’π’žοΈ€ with right adjoint 𝑅 induces a comonad π‘Š=πΏβˆ˜π‘… on π’žοΈ€, a comparison π’ŸοΈ€β†’coEM(π‘Š) into the co-Eilenberg–Moore category, and 𝐿 is comonadic when this comparison is an equivalence.

The adjoint triple between presheaves and families presheaf-family-adjoint-triple

A family over π’žοΈ€ is a set 𝐴(π‘₯) for each object π‘₯, with no action of morphisms. Families form a category Fam(π’žοΈ€): a morphism 𝐴→𝐡 is a function 𝐴(π‘₯)→𝐡(π‘₯) for each π‘₯.

Forgetting the restriction maps of a presheaf gives a functor

π‘ˆ:π–―π—Œπ—π’žοΈ€β†’Fam(π’žοΈ€).

It has both a left and a right adjoint,

FreeβŠ£π‘ˆβŠ£Cofree.

The two adjoints demonstrate different means of forcing a family to be functorial. The right adjoint universally quantifies over morphisms in,

Cofree(𝐴)(π‘₯)=βˆπ‘¦π’žοΈ€(𝑦,π‘₯)→𝐴(𝑦),

with restriction along 𝑓 given by precomposition. The left adjoint instead existentially quantifiers over morphisms out:

Free(𝐴)(π‘₯)=βˆ‘π‘¦π’žοΈ€(π‘₯,𝑦)×𝐴(𝑦),

with restriction acting on the first component. (For Free we ask that π’žοΈ€ have a set of objects, so that this sum is a set and thus Free defines a presheaf.)

Theorem. Presheaves are monadic and comonadic over families presheaves-monadic-comonadic-over-families

The adjoint triple FreeβŠ£π‘ˆβŠ£Cofree induces a monad 𝑇=π‘ˆβˆ˜Free and a comonad π‘Š=π‘ˆβˆ˜Cofree on Fam(π’žοΈ€).

Both comparison functors are equivalences: presheaves are the Eilenberg–Moore algebras of 𝑇 and the co-Eilenberg–Moore coalgebras of π‘Š,

π–―π—Œπ—π’žοΈ€β‰ƒEM(𝑇)π–―π—Œπ—π’žοΈ€β‰ƒcoEM(π‘Š).

So presheaves are both monadic and comonadic over families.

Reading the algebra structure concretely: a 𝑇-algebra on a family 𝐴 is a map βˆ‘π‘¦π’žοΈ€(π‘₯,𝑦)×𝐴(𝑦)→𝐴(π‘₯) for each π‘₯, subject to the monad algebra laws β€” that is, exactly a functorial action of restriction.

The comonadic reading is the same structure seen from the element’s side: a π‘Š-coalgebra is a map 𝐴(π‘₯)β†’βˆπ‘¦π’žοΈ€(𝑦,π‘₯)→𝐴(𝑦), giving each value its restriction along every morphism into π‘₯. Where the monad says restriction acts on values, the comonad says a value already carries all of its restrictions β€” and the coalgebra laws say it does so coherently.

Definition. Proper and maximal sieves proper-maximal-sieve

The representable 𝗒π‘₯ is itself a sieve on π‘₯. A sieve on π‘₯ is proper when it is not equal to the representable.

Say that a proper sieve is maximal when it contains all other proper sieves as a sub-sieve.

Definition. Sieves sieve

A sieve on an object π‘₯ of π’žοΈ€ is a subobject of the representable presheaf 𝗒π‘₯: a presheaf 𝑆 with a monic morphism 𝑆↣𝗒π‘₯. Sieves are a generalization from the notion of ideal found in ring theory to category theory.

A morphism 𝑓:𝑦→π‘₯ belongs to 𝑆, written π‘†βˆ‹π‘“, when 𝑓 lies in the image of the inclusion at 𝑦. Because 𝑆 is a presheaf and the inclusion is natural, membership is closed under precomposition:

π‘†βˆ‹π‘“β‡’π‘†βˆ‹π‘”β‹†π‘“for every 𝑔:𝑧→𝑦.

A sieve is thus a β€œdownward closed” collection of morphisms into π‘₯.

Sieves on π‘₯ are ordered by refinement: π‘†βŠ†π‘‡ when every morphism belonging to 𝑆 belongs to 𝑇.

Theorem. Maximality of the strict downset among proper sieves strict-downset-maximal

Call a direct structure reflecting when every morphism between objects of equal degree is a split epimorphism. In a reflecting direct category, every non-invertible-in-degree morphism strictly raises degree, and the strict downset is as large as a proper sieve can be:

If the direct structure is reflecting, then every proper sieve 𝑆 on π‘₯ refines into the strict downset:

π‘†βŠ†π–²π—π—‹π—‚π–Όπ—π–£π—ˆπ—π—‡(π‘₯).

Suppose π‘“βˆˆπ‘† with 𝑓:𝑦→π‘₯ of equal degree. By reflection 𝑓 has a section 𝑠, and closure under precomposition gives 𝑠⋆𝑓=𝗂𝖽π‘₯βˆˆπ‘†, contradicting properness.

So every morphism in 𝑆 strictly raises degree. That is, every morphism in 𝑆 is also a member of π–²π—π—‹π—‚π–Όπ—π–£π—ˆπ—π—‡(π‘₯).

Definition. The strict downset sieve of a direct category strict-downset-sieve

Let π’žοΈ€ carry a direct structure. The strict downset π–²π—π—‹π—‚π–Όπ—π–£π—ˆπ—π—‡(π‘₯) of an object π‘₯ is the presheaf of morphisms into π‘₯ from strictly lower objects:

π–²π—π—‹π—‚π–Όπ—π–£π—ˆπ—π—‡(π‘₯)(𝑦)={𝑓:𝑦→π‘₯βˆ£π‘¦β‰Ίπ‘₯},

with restriction by precomposition β€” well defined since degrees are non-decreasing, so precomposing can only stay strictly below.

The evident inclusion π–²π—π—‹π—‚π–Όπ—π–£π—ˆπ—π—‡(π‘₯)β†£γ‚ˆπ‘₯ makes π–²π—π—‹π—‚π–Όπ—π–£π—ˆπ—π—‡(π‘₯) a sieve on π‘₯. It is moreover a proper sieve, as it exlcudes the identity.

Definition. Terminal coalgebra terminal-coalgebra

Fix an endofunctor 𝐹:π’žοΈ€β†’π’žοΈ€. A terminal 𝐹-coalgebra, written 𝜈𝐹, is a terminal object of the category of coalgebras Coalg(𝐹) β€” equivalently, an initial algebra for 𝐹op.

Unfolding the universal property: a terminal coalgebra is a coalgebra (𝜈𝐹,out) such that every coalgebra (π‘₯,𝛾) admits a unique morphism unfold𝛾:π‘₯β†’πœˆπΉ satisfying

(unfold𝛾)⋆out=𝛾⋆𝐹(unfold𝛾).

Definition. Total category total-category

The total category of a displayed category over π’žοΈ€ collects the displayed data into a single category. Its objects are pairs of an object π‘₯ of π’žοΈ€ with an object over it, and its morphisms are pairs of a morphism 𝑓:π‘₯→𝑦 with a displayed morphism over 𝑓.

Projecting out the first components is a functor . Constructions presented displayed β€” algebras, Eilenberg–Moore categories β€” get their forgetful functor for free as this projection.

Well-founded posets are thin direct categories well-founded-poset-as-thin

Every poset forms a thin category. Similarly, if the poset is well-founded then it induces a thin direct category.

Definition. Category category

A category π’žοΈ€ consists of

  1. A type of objects π’žοΈ€0
  2. For each pair of objects π‘₯,𝑦:π’žοΈ€0 a set of morphisms π’žοΈ€(π‘₯,𝑦). We may simply write a morphism with an arrow, denote 𝑓:π’žοΈ€(π‘₯,𝑦) as 𝑓:π‘₯→𝑦 or π‘₯→𝑓𝑦 or similar
  3. A composition operation on morphisms. For 𝑓:π‘₯→𝑦 and 𝑔:𝑦→𝑧, there is a morphism 𝑓⋆𝑔:π‘₯→𝑧
  4. For each π‘₯:π’žοΈ€0, an identity morphism 𝗂𝖽π‘₯:π‘₯β†’π‘₯
  5. Left-unitality of composition: for all 𝑓:π‘₯→𝑦, an equality

    𝗂𝖽𝖫𝑓:𝗂𝖽π‘₯⋆𝑓=𝑓
  6. Right-unitality of composition: for all 𝑓:π‘₯→𝑦, an equality

    𝗂𝖽𝖱𝑓:𝑓⋆𝗂𝖽𝑦=𝑓
  7. Associativity of composition: for all 𝑓:π‘₯→𝑦, 𝑔:𝑦→𝑧, β„Ž:𝑧→𝑀, an equality

    π–Ίπ—Œπ—Œπ—ˆπ–Όπ‘“,𝑔,β„Ž:(𝑓⋆𝑔)β‹†β„Ž=𝑓⋆(π‘”β‹†β„Ž)

Concretely, the definition above is meant to model the one used in the Cubical standard library [1].

However, the notion of category is flexible. Depending on the context, we may be talking of small, locally small, wild, or any other kind of category that may augment which things we require to be (homotopy) sets, which things we require to be small types, etc. For the most part, the same idea of a category will apply across all of these settings.

Definition. Displayed Category displayed-category

A displayed category over a base category π’žοΈ€ packages the data of a category that β€œlies over” π’žοΈ€: each object and each morphism of π’žοΈ€ is equipped with a fiber of objects and morphisms displayed atop it. It consists of

  1. For each object π‘₯:π’žοΈ€0, a type of displayed objects lying over π‘₯. We write for a displayed object over π‘₯.
  2. For each morphism 𝑓:π’žοΈ€(π‘₯,𝑦) and displayed objects and , a set of displayed morphisms lying over 𝑓. We write such a displayed morphism as or , subscripting the arrow with the base morphism 𝑓 it lies over.
  3. A displayed composition operation. For 𝑓:π‘₯→𝑦 and 𝑔:𝑦→𝑧 with displayed morphisms and , there is a displayed morphism lying over the composite 𝑓⋆𝑔.
  4. For each , a displayed identity lying over 𝗂𝖽π‘₯.
  5. Left-unitality, displayed: for all , a heterogeneous equality

  6. Right-unitality, displayed: for all , a heterogeneous equality

  7. Associativity, displayed: for all , , over 𝑓:π‘₯→𝑦, 𝑔:𝑦→𝑧, β„Ž:𝑧→𝑀, a heterogeneous equality

Because the type of displayed morphisms depends on the base morphism 𝑓, the two sides of each displayed law inhabit different displayed hom-sets β€” those indexed by the two sides of the corresponding base-category equation. Each displayed law is therefore a heterogeneous equality π‘Ž=𝑝𝑏: a path from π‘Ž to 𝑏 lying over the base path 𝑝, rather than an equation within a single fixed set.

Concretely, this definition models the one used in the Cubical standard library [1].

A displayed category is to a category as a dependent type is to a context. In this way, displayed categories are effectively dependent categories, as we present the structure of as a category parametrized by the structure of π’žοΈ€.

Displayed categories were introduced by Ahrens and Lumsdaine [2]. The point is that a displayed category over π’žοΈ€ is equivalent to the data of a category π’ŸοΈ€ together with a functor 𝐹:π’ŸοΈ€β†’π’žοΈ€, but presented as families indexed by the objects and morphisms of π’žοΈ€ β€” so that constructions like Grothendieck fibrations can be defined without ever invoking equality of objects.

Definition. Bicategory bicategory

A bicategory is a notion of weak 2-category that arises as a category weakly enriched in categories. That is, instead of having hom sets, between any two objects a bicategory has hom categories such that the enriched category laws hold up to invertible 2-cell rather than strictly.

A bicategory 𝒦︀ consists of

  1. A type of objects 𝒦︀0, or 0-cells
  2. For all π‘₯,𝑦:𝒦︀0, a category 𝒦︀1(π‘₯,𝑦). We may elide the subscript and simply write this as 𝒦︀(π‘₯,𝑦). Refer to the objects of 𝒦︀(π‘₯,𝑦) as 1-cells between π‘₯ and 𝑦, and we may write 𝑓:𝒦︀(π‘₯,𝑦) as 𝑓:π‘₯→𝑦 or π‘₯→𝑓𝑦. For 𝑓,𝑔:𝒦︀(π‘₯,𝑦), refer to the morphisms in 𝒦︀(π‘₯,𝑦) between 𝑓 and 𝑔 as 2-cells and write the morphism 𝛼:(𝒦︀(π‘₯,𝑦))(𝑓,𝑔) as 𝛼:𝑓⇒𝑔 or 𝑓⇒𝛼𝑔
  3. For each π‘₯:𝒦︀0, an identity 1-cell 1π‘₯:𝒦︀(π‘₯,π‘₯)
  4. For all π‘₯,𝑦,𝑧:𝒦︀0, a composition functor 𝒦︀⋆π‘₯,𝑦,𝑧:𝒦︀(π‘₯,𝑦)×𝒦︀(𝑦,𝑧)→𝒦︀(π‘₯,𝑧). For 1-cells 𝑓:π‘₯→𝑦 and 𝑔:𝑦→𝑧, write their composite as 𝑓⋆𝑔:π‘₯→𝑧
  5. For all 𝑀,π‘₯,𝑦,𝑧:𝒦︀0, a natural isomorphism, the associator 𝛼 between the two composite functors 𝒦︀(𝑀,π‘₯)×𝒦︀(π‘₯,𝑦)×𝒦︀(𝑦,𝑧)→𝒦︀(𝑀,𝑧) that compose the leftmost, respectively rightmost, pair first:

    𝒦︀⋆𝑀,𝑦,π‘§βˆ˜(𝒦︀⋆𝑀,π‘₯,𝑦×id)⇒𝒦︀⋆𝑀,π‘₯,π‘§βˆ˜(id×𝒦︀⋆π‘₯,𝑦,𝑧)

    Its component at 1-cells 𝑓,𝑔,β„Ž is the invertible 2-cell

    𝛼𝑓,𝑔,β„Ž:(𝑓⋆𝑔)β‹†β„Žβ‡’π‘“β‹†(π‘”β‹†β„Ž)
  6. For all π‘₯,𝑦:𝒦︀0, natural isomorphisms, the left unitor πœ† and right unitor 𝜌, each between an endofunctor of 𝒦︀(π‘₯,𝑦) and the identity functor:

    𝒦︀⋆π‘₯,π‘₯,π‘¦βˆ˜βŸ¨1π‘₯,idβŸ©β‡’id𝒦︀⋆π‘₯,𝑦,π‘¦βˆ˜βŸ¨id,1π‘¦βŸ©β‡’id

    where 1π‘₯ and 1𝑦 in the pairings βŸ¨βˆ’,βˆ’βŸ© denote the constant functors at the identity 1-cells. The components at a 1-cell 𝑓:𝒦︀(π‘₯,𝑦) are the invertible 2-cells

    πœ†π‘“:1π‘₯β‹†π‘“β‡’π‘“πœŒπ‘“:𝑓⋆1𝑦⇒𝑓
  7. such that for all 𝑓:𝒦︀(π‘₯,𝑦) and 𝑔:𝒦︀(𝑦,𝑧) the triangle below commutes in 𝒦︀(π‘₯,𝑧):

  8. and such that for all composable 1-cells 𝑓,𝑔,β„Ž,π‘˜ the pentagon below commutes:

Definition. Monad in a bicategory monad-in-a-bicategory

Fix a bicategory 𝒦︀, with composition ⋆, identity 1-cells 1π‘₯, associator 𝛼, and unitors πœ†,𝜌. A monad in 𝒦︀ internalises the usual notion of monad: it is an endo-1-cell carrying a multiplication and a unit that satisfy the monoid laws up to the coherence cells of the bicategory.

A monad in 𝒦︀ consists of

  1. a 0-cell π‘₯, the object the monad acts on;
  2. an endo-1-cell 𝑑:𝒦︀(π‘₯,π‘₯);
  3. a multiplication 2-cell πœ‡:𝑑⋆𝑑⇒𝑑;
  4. a unit 2-cell πœ‚:1π‘₯⇒𝑑;
  5. such that πœ‡ is associative: the following diagram of 2-cells commutes in 𝒦︀(π‘₯,π‘₯), where the top map is the associator that rebrackets the threefold composite:

  6. and such that πœ‡ and πœ‚ satisfy the unit laws: the following two diagrams commute in 𝒦︀(π‘₯,π‘₯), where the hypotenuses are the left and right unitors:

Taking 𝒦︀ to be the bicategory of categories, functors, and natural transformations recovers an ordinary monad on a category: 𝑑 is the endofunctor, πœ‡ the multiplication, and πœ‚ the unit, with the coherence cells all identities.

Freely transported terms in dependent type theory freely-transported-terms

Given 𝐴 and 𝐡:π΄β†’π“π²π©πž we can make sense of transported terms along equalities between indices in 𝐴. Say, with

π—Œπ—Žπ–»π—Œπ—:(𝑝:π‘Ž=π‘Žβ€²)β†’π΅π‘Žβ†’π΅π‘Žβ€²

for π‘Ž,π‘Žβ€²:𝐴.

For instance, if 𝑝:π‘Ž=π‘Žβ€² and 𝑏:π΅π‘Ž then π—Œπ—Žπ–»π—Œπ—π‘π‘:π΅π‘Žβ€²

To avoid landing in transport hell, I suspect that it may be preferable to work inside of a description of freely transported terms instead of taking semantic transports. The hypothesis is that by using descriptions of formal transport rather than actually computing a transport, we may defer the computation of an actual transport until the end of a construction. So instead of working with π΅π‘Ž directly, perhaps we may work with

π–₯π—‹π–Ύπ–Ύπ–²π—Žπ–»π—Œπ—π΅π‘Žβ‰”βˆ‘(π‘Žβ€²:𝐴)βˆ‘(𝑝:π‘Ž=π‘Žβ€²)π΅π‘Žβ€²

I think that this is very closely related to the Fording trick, as a map out of π–₯π—‹π–Ύπ–Ύπ–²π—Žπ–»π—Œπ—π΅π‘Ž,

𝑓:βˆ‘(π‘Žβ€²:𝐴)βˆ‘(𝑝:π‘Ž=π‘Žβ€²)π΅π‘Žβ€²β†’πΆ

can instead be described as a map,

𝑔:(π‘Žβ€²:𝐴)β†’(𝑝:π‘Ž=π‘Žβ€²)β†’π΅π‘Žβ€²β†’πΆ

Both this and the fording trick use the Coyoneda lemma to represent an dependent type family.

Definition. Displayed Total Category displayed-total-category

Given a displayed category over a category π’žοΈ€ and another displayed category over , the total category of , we can define , the displayed total category of , as a displayed category over π’žοΈ€.

Definition. Free Monoidal Category over a Set free-monoidal-category

Fix a set 𝑋. The objects of the free monoidal category over 𝑋, π–₯π—‹π–Ύπ–Ύπ–¬π—ˆπ—‡(𝑋), are generated inductively by the elements of 𝑋 and a unit element 𝐼 over a binary operation βŠ—. The morphisms are given by a quotient-inductive type. They are generated by associators, unitors, and identity over composition and parallel action over βŠ— then quotiented by associativity and composition equation to satisfy the category laws, equations constraining the associators/unitors to be natural isomorphisms, and pentagon/triangle equations to satiate the axioms of a monoidal category.

Definition 0.1. Global Elimination Principle for the Free Monoidal Category free-monoidal-category-elimination

Given any displayed monoidal category 𝑀𝙳 over π–₯π—‹π–Ύπ–Ύπ–¬π—ˆπ—‡(𝑋) with an interpretation πœ„:𝑋⇝𝑀𝙳, we may construct a global section π–₯π—‹π–Ύπ–Ύπ–¬π—ˆπ—‡(𝑋)→𝑀𝙳. We refer to this as the global elimination principle of π–₯π—‹π–Ύπ–Ύπ–¬π—ˆπ—‡(𝑋).

Definition. Global Elimination Principle for the Free Monoidal Category free-monoidal-category-elimination

Given any displayed monoidal category 𝑀𝙳 over π–₯π—‹π–Ύπ–Ύπ–¬π—ˆπ—‡(𝑋) with an interpretation πœ„:𝑋⇝𝑀𝙳, we may construct a global section π–₯π—‹π–Ύπ–Ύπ–¬π—ˆπ—‡(𝑋)→𝑀𝙳. We refer to this as the global elimination principle of π–₯π—‹π–Ύπ–Ύπ–¬π—ˆπ—‡(𝑋).

Products of Categories as Total Categories product-as-total-category

Given categories π’žοΈ€ and π’ŸοΈ€, π’žοΈ€Γ—π’ŸοΈ€ is equivalent to the total category of weakening.

Definition. Quiver quiver

A quiver is just a directed graph presented via a type of objects, a type of edges, and two projection functions that pick out source and target of an edge.

Definition. Reindexing a Displayed Category reindexing

Given a displayed category over a category π’žοΈ€ and a functor 𝐹:π’ŸοΈ€β†’π’žοΈ€, the reindexing of along 𝐹 is a displayed category over π’ŸοΈ€. It is simply the portion of over the image of 𝐹.

Definition. Thin Category thin-category

A category 𝐢 is thin if there is at most one morphism between any two objects.

Definition. Weakening a Category weakening-category

For categories π’žοΈ€ and π’ŸοΈ€, we can define the weakening of π’ŸοΈ€ over π’žοΈ€ as a displayed category over π’žοΈ€ which trivially displays a copy of π’ŸοΈ€ over each object of π’žοΈ€.

(𝗐𝖾𝖺𝗄𝖾𝗇(π’žοΈ€,π’ŸοΈ€))π‘β‰”π’ŸοΈ€

Definition. Element of a Presheaf element-of-presheaf

An element of a presheaf 𝑃 at an object 𝑐 is an element π‘₯ of the set 𝑃𝑐.

Definition. Universal Element of a Presheaf universal-element

A universal element of a presheaf 𝑃 on a category 𝐢 is an element π‘₯βˆˆπ‘ƒπ‘, where 𝑐 is some object of 𝐢, demonstrating that 𝑃 is representable by 𝑐.

π‘ƒβ‰…γ‚ˆπ‘

(Slightly) more concretely, a universal element is captured by the following three pieces of data

  • An object 𝑐 of 𝐢
  • An element π‘₯βˆˆπ‘ƒπ‘
  • A proof that the map sending a morphism 𝑓:𝑏→𝑐 to (𝑃𝑓)(π‘₯) is an equivalence

This third point states that morphisms from 𝑏 into 𝑐 are uniquely determined by an element of 𝑃 at the domain 𝑏.

Or equivalently, universal elements are terminal in the category of elements.

What is a universal property, really? universal-property

Universal properties are a convenient method for defining an object in a category up to isomorphism. Rather than giving a concrete, bottom-up construction of an object, we can instead uniquely specify its behavior.

Consider the example of products in a category 𝐢. We say that the product of 𝑐 and 𝑑 is any object 𝑝 of 𝐢 such that the following diagram commutes.

We say that 𝑝 satisfies the universal property of the product of 𝑐 and 𝑑.

Surely this matches our set-based intuition of what a product should behave like. Similarly, we can sketch out constructions of other universal properties like initial objects, terminal objects, exponentials, etc. However, what is precisely meant by the term universal property?

The notion of a universal property is made precise by the notion of a universal element of a presheaf. That is, an object satisfies a universal property if we can build a universal element of the appropriate presheaf at that object.

Let’s look at the universal element characterization of the products example. Note that a map into a product is determined by a map into each component. To map into 𝑐×𝑑, we need both a map into 𝑐 and a map into 𝑑, as in the above diagram. That is, to build a map 𝑏→𝑐×𝑑, we must simultaneously provide elements of γ‚ˆπ‘ and γ‚ˆπ‘‘ at 𝑏.

Using the product of presheaves, this means we are providing a single element of the presheaf (γ‚ˆπ‘)Γ—(γ‚ˆπ‘‘). Quite nicely, the universal element of this presheaf provides the object of 𝐢 that is the product of 𝑐 and 𝑑. The universal element, provided that it exists, contains the following data:

  • An object 𝑝
  • An element π‘₯∈(γ‚ˆπ‘Γ—γ‚ˆπ‘‘)𝑝
  • A proof that the map sending 𝑓:𝑏→𝑝 to a pair of maps 𝑓1:𝑏→𝑐, 𝑓2:𝑏→𝑑 is an equivalence. Therefore, any element of γ‚ˆπ‘Γ—γ‚ˆπ‘‘ factors through π‘₯

Recall that the product of presheaves is computed pointwise in the category of sets, so if we expand the type of the element π‘₯ above we find that π‘₯ is a pair of maps 𝑝→𝑐 and 𝑝→𝑑.

(γ‚ˆπ‘Γ—γ‚ˆπ‘‘)𝑝≅(γ‚ˆπ‘π‘)Γ—(γ‚ˆπ‘‘π‘)

The first part of this pair is precisely πœ‹1. Correspondingly, the second part of this pair is πœ‹2. Finally, the universality of the element π‘₯ (i.e. the proof that any other element factors through π‘₯) captures our commutative diagram from above.

This is a very rough sketch of what a universal property is, and has elided for now an important application of the Yoneda lemma. In any case, all a universal property is really saying is that a particular presheaf is representable; and, rather elegantly, a universal element of a presheaf is convenient packaging of that representability proof.

In summary, universal properties are not as ad-hoc as they may initially seem, and the language of presheaves provides a reusable and precise definition that can be instantiated to describe a very large class of properties.

Cloudflare Parsing Error cloudflare-parsing-error

An error in the Cloudflare HTML parser would allow uninitialized memory to be dumped when there were imbalanced HTML tags.

This is indeed a case where a verified parser would have alleviated the issue.

Definition. GrΓΆbner Basis groebner-basis

Let 𝐾 be a ring and 𝐾[π‘₯0,…,π‘₯𝑛] a polynomial ring over it. Suppose πΌβŠ‚πΎ[π‘₯0,…,π‘₯𝑛] is an ideal. A GrΓΆbner basis for 𝐼 is a generating set of polynomials for the ideal that is minimal with respect to a given ordering on the monomials π‘₯0,…,π‘₯𝑛.

Given an ideal 𝐼, a GrΓΆbner basis for 𝐼 may be found via Buchberger’s algorithm. Intuitively, Buchberger’s algorithm attempts to solve a system of polynomial equations by iterated polynomial division to eliminate variables. At any given point, there is a degree of freedom in which what variable will be eliminated by the next division. The algorithm attempts to eliminate variables with respect to the monomial ordering.

Buchberger’s algorithm may be viewed simultaneously as a generalization of the Quine-McCluskey Boolean minimization algorithm and as a special case of the Knuth-Bendix algorithm.

The complexity of Buchberger’s algorithm is a little unwieldy to estimate in general. However, just like SAT solvers there are enough optimizations to execute Buchberger reasonably fast in practice. For instance, it is fast enough to handle several hundreds of polynomials, each having hundreds of terms with very large coefficients.

There are some very fun applications of this approach, such as Solving Sudoku with Algebra. This idea has also been applied to inferring polynomial loop invariants.

GrΓΆbner Bases for Inferring Polynomial Loop Invariants groebner-loop-invariants

When the tools in I4: Incremental inference of inductive invariants for verification of distributed protocols and On Symmetry and Quantification: A New Approach to Verify Distributed Protocols search for an inductive invariant of a distributed system, the search procedure instantiates a series of small finite models and tries to infer from their truth tables a series of logical formulae that hold over those finite models. These formulae are found by running the Quine-McCluskey algorithm for minimization of Boolean functions. The prime implicants found by Quine-McCluskey have a latent symmetry that can be abstracted into quantified formulae. There are only so many small numbers, and so small finite models may propose formulae that do not hold at larger sizes. However, if you find a formula that holds at size 𝑛 as well as size 𝑛+1, then it is likely a good candidate to hold at all sizes. You need to be a little careful if your protocol is indexed by several variables, but mostly this general idea holds when abstracting to a protocol of unbounded size. Further discussion of this idea can be found in SAT-based quantified symmetric minimization of the reachable states of distributed protocols: An update.

My observation was that Quine-McCluskey is just a special instance of Buchberger’s algorithm for computing GrΓΆbner Bases. That is, you can describe Boolean formulae as polynomials over the field with two elements, and in this translation Quine-McCluskey and Buchberger each compute the same data. This observation isn’t new in and of itself, but it does open up an opportunity to generalize the invariant search procedure that is used above.

The place I went looking to apply this idea was in the search of polynomial loop invariants. If a loop had an invariant that is expressible as a polynomial relation between the program variables, then you could apply the same idea as above to infer the loop invariant.

Suppose π‘₯1,…,π‘₯𝑛 are the variables in scope of program 𝑆 and 𝑆 contains a loop that we want to infer an invariant for. Denote the value of π‘₯𝑖 at the 𝑗-th loop iteration by π‘₯𝑖,𝑗. The invariant search procedure proceeds intuitively as the following: we will keep track of the minimal set of polynomials that could interpolate between all of the variable assignments that we have witnessed thus far. Formally this is kept track of by the ideal of polynomials. At the 𝑗-th loop iteration we add a new generator to the ideal which corresponds to the assignments π‘₯1,𝑗,…,π‘₯𝑛,𝑗. The GrΓΆbner basis for this ideal provides the minimal data needed to generate all the assignments witnessed thus far, so if this process saturates then the GrΓΆbner basis encodes a polynomial loop invariant. The nice thing about polynomials is that they have finite degree which guarantees that this process does indeed saturate (provided that the degree of the invariant is smaller than the number of loop iterations).

I was so excited to find this idea. I’d felt like it was my first good idea in grad school. Then I read Automatic Generation of Polynomial Loop Invariants: Algebraic Foundations and found out someone had done this 20 years ago. I still wonder from time to time if there is room to further refine this idea or perhaps further generalize it. For instance, there is further generalization beyond Quine-McCluskey or Buchberger to the Knuth-Bendix algorithm, which seems to be a more general instance of both of these algorithms. So perhaps this search procedure can be weakened to an even more general class? Although, I’m not yet familiar much with the Knuth-Bendix algorithm.

A Method for Verifying Translational Invariance of Image Processing Neural Networks translation-invariance-verification

My understanding for how one may prove a safety property 𝑃 for a neural network 𝑁 is as follows. First, express 𝑃 as an input-output property. That is, choose some region 𝐷 in the domain of 𝑁 to represent the inputs of interest. Further choose some region 𝐴 in the codomain of 𝑁 to describe safe outputs. Then describe 𝑃 as the property:

βˆ€π‘₯∈𝐷.𝑁(π‘₯)∈𝐴

For instance, this is more or less how Reluplex, Marabou, and 𝛼𝛽-CROWN each work. Squinting my eyes, this is the only such method for verifying a safety property for 𝑁. That is, this is the only way to get a 100% guarantee that 𝑃 holds rather than some high measure of confidence.

I believe in this formalism I have an idea for how to express the property β€œπ‘ is translationally invariant” where 𝑁 is an image classifying net.

To this end, we need a continuous artifact that captures what it means to translate an image. We can express an image as a matrix of pixels 𝐼 (or perhaps several parallel matrices if we care about color channels, but stick to a single grayscale matrix for now). To shift 𝐼 over by a single pixel, we may left-multiply 𝐼 by the shift matrix 𝑆, where 𝑆 is the matrix filled with zeros and has 1β€²s on the subdiagonal. Note that all the directions of shifting 𝐼 are captured by the combinations of left/right multiplication by 𝑆, 𝑆𝑇.

Shifting by multiple pixels is now expressed by the matrix 𝑆𝑛𝐼, however this is still a discrete dynamical system. We don’t have a continuous object by which we can test our safety property. My initial thought to continuousify this system was to express some sort of exponential flow by 𝑆. That is, consider the matrix

π‘†π‘‘πΌβ‰”π‘’π‘‘π—…π—ˆπ—€(𝑆)

𝑆𝑑𝐼 then captures what it means to translate the image 𝐼 over by 𝑑, a real-valued β€œamount of shifting”. This choice of continuous artifact could then be used for a verification effort, however there are some issues related to numerical stability because 𝑆 isn’t invertible. Because it is not invertible, π—…π—ˆπ—€(𝑆) doesn’t actually exist. So to make the above construction work, we need to mildly perturb 𝑆 and take a pseudoinverse. This sort of works and makes it so 𝑆𝑑𝐼 does capture some real valued shift, but it is only accurate when 𝑑 is small. So this is maybe problematic for our verification effort. This may be resolvable by chopping up the problem into subproblems, each of which is thin enough for the current iteration is accurate enough on that subproblem. However, there are two big things to consider.

  1. The exponential approach above is probably too complicated. It seems

likely that you may be able to take some (sequence of) linear interpolation(s) between the 𝑆𝑛𝐼’s. This will still be some continuous object that captures a real-valued shift and it will be much more stable than the approach above.

  1. Even if we sort out which continuous object represents translation

of an image, I cannot for the life of me train any neural network that is translationally invariant. So the proof method is useless if there is nothing that it would ever apply to.

Precisely in this last point, I have mostly focused on trying to train a small CNN for MNIST handwritten digit classification that preserves the output class for small, reasonable translations of the digit. I’ve used data augmentation to predispose the network to being translationally invariant, and even though I can get a high degree of invariance, I cannot get a network that is invariant for all of the examples even in the training set or a reserved testing set.

It may be the case that the method I propose for measuring this invariance could be used to adversarially train a network to have better invariance. It may also be the case that all CNNs are bound to suffer from small degrees of translational sensitivity. I cannot find the citation at the moment, but there was a paper that suggested that CNNs suffer from weird issues of translational sensitivity that relate to the size of the convolutional window. So maybe this approach is doomed to fail anyway.

On the whole, I will say that machine learning verification almost sounds like an oxymoron. That is, if you have the expressivity to properly state a sophisticated safety property, then you likely understand the problem enough to not need to resort to machine learning in the first place. So almost tautologically, it seems that there cannot be satisfying verification of neural nets, as the tasks of machine learning and verification live on very different epistemic foundations.

The related works I could liberate from Zotero may be found below.

15 entries

0.2 Adequate Losses via Quantitative Linear Logic capucci-2026-adequate

As neural components are increasingly embedded in existing symbolic software – including safety-critical systems – the question arises of how to specify and enforce the safety of the newly introduced neural parts. Unlike traditional logical specifications, these must be amenable not only to the standard Boolean interpretation, but also to training and optimisation. The latter calls for a quantitative interpretation of the logical syntax, subject to further requirements such as smoothness and differentiability. Moreover, the qualitative and quantitative sides of the logic must share a unifying proof-theoretic and categorical semantics. Finally, the new logic should link cleanly to the substructural and program logics that underpin the verification of existing symbolic programs. In this paper, we present a logic that ticks all of these boxes. We introduce a family of calculi, pQLL, indexed by a hardness degree 𝑝, prove a cut-elimination theorem for them, and establish completeness with respect to enriched residuated β€˜soft’ lattices. At 𝑝=∞, pQLL reduces to multiplicative additive linear logic (MALL), and provability in pQLL converges to provability in MALL as π‘β†’βˆž. We express optimisation objectives in the syntax of this logic and prove the quantitative adequacy of neuro-symbolic loss functions – a result that has eluded the neuro-symbolic machine learning community for nearly a decade.
DOI Β· arXiv

0.3 Quantitative Linear Logic for Neuro-Symbolic Learning and Verification flinkow-2026-quantitative

Differentiable Logics are deployed in neuro-symbolic learning tasks as a way of embedding logical constraints in the training objective of neural networks. A differentiable logic consists of a syntax to write logical properties and a semantics to interpret them as real-valued functions to be folded in the loss function. A defining trade-off of the field is that between logical properties of the connectives, and analytic concerns for the semantics, with both aspects being relevant in applications. At one extreme we find fuzzy logics, that have well-established algebraic and proof-theoretic foundations, and at the other ad-hoc differentiable logics like Fischer’s DL2, conceived for deep learning applications. However, no satisfactory foundation has emerged yet. We propose a resolution to this long-standing tension via a novel logic, Quantitative Linear Logic (QLL), with foundational ambitions. Our design is driven by naturality – the idea that, since logical constraints are translated to losses, the semantics of the connectives should be pertinent operations used in ML practice (that is, sum and log-sum-exp) on additive quantities (like logits). We then judge the result on two aspects: logical adequacy – that they satisfy most of the standard logical laws of Linear Logic; and empirical effectiveness – test-time performance (as measured by adversarial attacks) is well-correlated to the actual verification of the logical constraints (as measured by off-the-shelf neural network verifiers), which makes QLL stand out among SoTA techniques.
DOI Β· arXiv

0.4 Quantifiers for Differentiable Logics in Rocq (Extended Abstract) marulandagiraldo-2025-quantifiers

DOI

0.5 Fundamental Components of Deep Learning: A category-theoretic approach gavranovicFundamentalComponentsDeep

Deep learning, despite its remarkable achievements, is still a young field. Like the early stages of many scientific disciplines, it is marked by the discovery of new phenomena, ad-hoc design decisions, and the lack of a uniform and compositional mathematical foundation. From the intricacies of the implementation of backpropagation, through a growing zoo of neural network architectures, to the new and poorly understood phenomena such as double descent, scaling laws or in-context learning, there are few unifying principles in deep learning. This thesis develops a novel mathematical foundation for deep learning based on the language of category theory. We develop a new framework that is a) end-to-end, b) unform, and c) not merely descriptive, but prescriptive, meaning it is amenable to direct implementation in programming languages with sufficient features. We also systematise many existing approaches, placing many existing constructions and concepts from the literature under the same umbrella. In Part I we identify and model two main properties of deep learning systems parametricity and bidirectionality by we expand on the previously defined construction of actegories and Para to study the former, and define weighted optics to study the latter. Combining them yields parametric weighted optics, a categorical model of artificial neural networks, and more. Part II justifies the abstractions from Part I, applying them to model backpropagation, architectures, and supervised learning. We provide a lens-theoretic axiomatisation of differentiation, covering not just smooth spaces, but discrete settings of boolean circuits as well. We survey existing, and develop new categorical models of neural network architectures. We formalise the notion of optimisers and lastly, combine all the existing concepts together, providing a uniform and compositional framework for supervised learning.
DOI

0.6 Architecture-Preserving Provable Repair of Deep Neural Networks taoArchitecturePreservingProvableRepair2023

Deep neural networks (DNNs) are becoming increasingly important components of software, and are considered the state-of-the-art solution for a number of problems, such as image recognition. However, DNNs are far from infallible, and incorrect behavior of DNNs can have disastrous real-world consequences. This paper addresses the problem of architecture-preserving V-polytope provable repair of DNNs. A V-polytope defines a convex bounded polytope using its vertex representation. V-polytope provable repair guarantees that the repaired DNN satisfies the given specification on the infinite set of points in the given V-polytope. An architecture-preserving repair only modifies the parameters of the DNN, without modifying its architecture. The repair has the flexibility to modify multiple layers of the DNN, and runs in polynomial time. It supports DNNs with activation functions that have some linear pieces, as well as fully-connected, convolutional, pooling and residual layers. To the best our knowledge, this is the first provable repair approach that has all of these features. We implement our approach in a tool called APRNN. Using MNIST, ImageNet, and ACAS Xu DNNs, we show that it has better efficiency, scalability, and generalization compared to PRDNN and REASSURE, prior provable repair methods that are not architecture preserving. CCS Concepts: β€’ Computing methodologies β†’ Neural networks; β€’ Theory of computation β†’ Linear programming; β€’ Software and its engineering β†’ Software post-development issues.
PDF Β· DOI Β· pldb

0.7 Compiling Higher-Order Specifications to SMT Solvers: How to Deal with Rejection Constructively daggitt-2023-compiling

PDF Β· DOI Β· pldb

0.8 Vehicle: Interfacing Neural Network Verifiers with Interactive Theorem Provers daggitt-2022-vehicle

Verification of neural networks is currently a hot topic in automated theorem proving. Progress has been rapid and there are now a wide range of tools available that can verify properties of networks with hundreds of thousands of nodes. In theory this opens the door to the verification of larger control systems that make use of neural network components. However, although work has managed to incorporate the results of these verifiers to prove larger properties of individual systems, there is currently no general methodology for bridging the gap between verifiers and interactive theorem provers (ITPs). In this paper we present Vehicle, our solution to this problem. Vehicle is equipped with an expressive domain specific language for stating neural network specifications which can be compiled to both verifiers and ITPs. It overcomes previous issues with maintainability and scalability in similar ITP formalisations by using a standard ONNX file as the single canonical representation of the network. We demonstrate its utility by using it to connect the neural network verifier Marabou to Agda and then formally verifying that a car steered by a neural network never leaves the road, even in the face of an unpredictable cross wind and imperfect sensors. The network has over 20,000 nodes, and therefore this proof represents an improvement of 3 orders of magnitude over prior proofs about neural network enhanced systems in ITPs.
arXiv

0.9 PRIMA: General and Precise Neural Network Certification via Scalable Convex Hull Approximations mullerPRIMAGeneralPrecise2022

Formal verification of neural networks is critical for their safe adoption in real-world applications. However, designing a precise and scalable verifier which can handle different activation functions, realistic network architectures and relevant specifications remains an open and difficult challenge. In this paper, we take a major step forward in addressing this challenge and present a new verification framework, called PRIMA. PRIMA is both (i) general: it handles any non-linear activation function, and (ii) precise: it computes precise convex abstractions involving multiple neurons via novel convex hull approximation algorithms that leverage concepts from computational geometry. The algorithms have polynomial complexity, yield fewer constraints, and minimize precision loss. We evaluate the effectiveness of PRIMA on a variety of challenging tasks from prior work. Our results show that PRIMA is significantly more precise than the state-of-the-art, verifying robustness to input perturbations for up to 20%, 30%, and 34% more images than existing work on ReLU-, Sigmoid-, and Tanh-based networks, respectively. Further, PRIMA enables, for the first time, the precise verification of a realistic neural network for autonomous driving within a few minutes.
PDF Β· DOI Β· arXiv Β· pldb

0.10 Provable repair of deep neural networks sotoudehProvableRepairDeep2021

Deep Neural Networks (DNNs) have grown in popularity over the past decade and are now being used in safety-critical domains such as aircraft collision avoidance. This has motivated a large number of techniques for finding unsafe behavior in DNNs. In contrast, this paper tackles the problem of correcting a DNN once unsafe behavior is found. We introduce the provable repair problem, which is the problem of repairing a network N to construct a new network Nβ€² that satisfies a given specification. If the safety specification is over a finite set of points, our Provable Point Repair algorithm can find a provably minimal repair satisfying the specification, regardless of the activation functions used. For safety specifications addressing convex polytopes containing infinitely many points, our Provable Polytope Repair algorithm can find a provably minimal repair satisfying the specification for DNNs using piecewise-linear activation functions. The key insight behind both of these algorithms is the introduction of a Decoupled DNN architecture, which allows us to reduce provable repair to a linear programming problem. Our experimental results demonstrate the efficiency and effectiveness of our Provable Repair algorithms on a variety of challenging tasks.
PDF Β· DOI Β· pldb

0.11 Tracking translation invariance in CNNs myburghTrackingTranslationInvariance2021

Although Convolutional Neural Networks (CNNs) are widely used, their translation invariance (ability to deal with translated inputs) is still subject to some controversy. We explore this question using translation-sensitivity maps to quantify how sensitive a standard CNN is to a translated input. We propose the use of Cosine Similarity as sensitivity metric over Euclidean Distance, and discuss the importance of restricting the dimensionality of either of these metrics when comparing architectures. Our main focus is to investigate the effect of different architectural components of a standard CNN on that network’s sensitivity to translation. By varying convolutional kernel sizes and amounts of zero padding, we control the size of the feature maps produced, allowing us to quantify the extent to which these elements influence translation invariance. We also measure translation invariance at different locations within the CNN to determine the extent to which convolutional and fully connected layers, respectively, contribute to the translation invariance of a CNN as a whole. Our analysis indicates that both convolutional kernel size and feature map size have a systematic influence on translation invariance. We also see that convolutional layers contribute less than expected to translation invariance, when not specifically forced to do so.
Web Β· arXiv

0.12 Towards Verified Artificial Intelligence seshiaVerifiedArtificialIntelligence2020

Verified artificial intelligence (AI) is the goal of designing AI-based systems that have strong, ideally provable, assurances of correctness with respect to mathematically-specified requirements. This paper considers Verified AI from a formal methods perspective. We describe five challenges for achieving Verified AI, and five corresponding principles for addressing these challenges.
DOI Β· arXiv

0.13 Verification of Deep Convolutional Neural Networks Using ImageStars tranVerificationDeepConvolutional2020

Convolutional Neural Networks (CNN) have redefined stateof-the-art in many real-world applications, such as facial recognition, image classification, human pose estimation, and semantic segmentation. Despite their success, CNNs are vulnerable to adversarial attacks, where slight changes to their inputs may lead to sharp changes in their output in even well-trained networks. Set-based analysis methods can detect or prove the absence of bounded adversarial attacks, which can then be used to evaluate the effectiveness of neural network training methodology. Unfortunately, existing verification approaches have limited scalability in terms of the size of networks that can be analyzed.
PDF Β· DOI Β· arXiv Β· pldb

0.14 Stride and Translation Invariance in CNNs moutonStrideTranslationInvariance2020

Convolutional Neural Networks have become the standard for image classification tasks, however, these architectures are not invariant to translations of the input image. This lack of invariance is attributed to the use of stride which ignores the sampling theorem, and fully connected layers which lack spatial reasoning. We show that stride can greatly benefit translation invariance given that it is combined with sufficient similarity between neighbouring pixels, a characteristic which we refer to as local homogeneity. We also observe that this characteristic is dataset-specific and dictates the relationship between pooling kernel size and stride required for translation invariance. Furthermore we find that a trade-off exists between generalization and translation invariance in the case of pooling kernel size, as larger kernel sizes lead to better invariance but poorer generalization. Finally we explore the efficacy of other solutions proposed, namely global average pooling, anti-aliasing, and data augmentation, both empirically and through the lens of local homogeneity.
DOI Β· arXiv

0.15 Why do deep convolutional networks generalize so poorly to small image transformations? azulayWhyDeepConvolutional

Convolutional Neural Networks (CNNs) are commonly assumed to be invariant to small image transformations: either because of the convolutional architecture or because they were trained using data augmentation. Recently, several authors have shown that this is not the case: small translations or rescalings of the input image can drastically change the network’s prediction. In this paper, we quantify this phenomena and ask why neither the convolutional architecture nor data augmentation are sufficient to achieve the desired invariance. Specifically, we show that the convolutional architecture does not give invariance since architectures ignore the classical sampling theorem, and data augmentation does not give invariance because the CNNs learn to be invariant to transformations only for images that are very similar to typical images from the training set. We discuss two possible solutions to this problem: (1) antialiasing the intermediate representations and (2) increasing data augmentation and show that they provide only a partial solution at best. Taken together, our results indicate that the problem of insuring invariance to small image transformations in neural networks while preserving high accuracy remains unsolved.
DOI

0.16 Formal Verification of CNN-based Perception Systems kouvarosFormalVerificationCNNbased2018

We address the problem of verifying neural-based perception systems implemented by convolutional neural networks. We define a notion of local robustness based on affine and photometric transformations. We show the notion cannot be captured by previously employed notions of robustness. The method proposed is based on reachability analysis for feed-forward neural networks and relies on MILP encodings of both the CNNs and transformations under question. We present an implementation and discuss the experimental results obtained for a CNN trained from the MNIST data set.
DOI Β· arXiv

Definition. Constant Presheaf constant-presheaf

The constant presheaf of a set 𝑋 on a category 𝐢 is a functor Ξ”(𝑋):πΆπ‘œπ‘β†’π’πžπ­ such that

Ξ”(𝑋)𝑐≔𝑋

I often call this the discrete presheaf for 𝑋, but I don’t know if that’s standard.

Finite Cardinal Arithmetic in a Topos finite-cardinal-arithmetic-topos

In a topos with a natural numbers object (𝑁,𝑧,𝑠), you can define finite cardinals as objects that arise as the pullback along a morphism 𝑝:βŠ€β†’π‘ of the generic finite cardinal. Write [𝑝] for the cardinal corresponding to 𝑝.

The above characterization is a little obtuse and does warrant some more explanation. One way to make it more concrete is that [𝑧]=βŠ₯ and [π‘ βˆ˜π‘›]=βŠ€βŠ•[𝑛], and that finite cardinals warrant a nice induction principle. If 𝑃 is a property expressible in the internal language such that βŠ₯ satisfies 𝑃, and that whenever 𝐴 satisfies 𝑃 then βŠ€βŠ•π΄ satisfies 𝑃; then every finite cardinal satisfies 𝑃. That is, 𝑃 forms a (𝑧,𝑠)-closed subobject of 𝑁 and thus 𝑃 is all of 𝑁.

My working mental model is in a presheaf topos, where the natural numbers object can be defined explicitly as Ξ”(β„•).

Definition 0.17. Constant Presheaf constant-presheaf

The constant presheaf of a set 𝑋 on a category 𝐢 is a functor Ξ”(𝑋):πΆπ‘œπ‘β†’π’πžπ­ such that

Ξ”(𝑋)𝑐≔𝑋

I often call this the discrete presheaf for 𝑋, but I don’t know if that’s standard.

The finite cardinals with respect to Ξ”(β„•) can be characterized then as,

[0]≔βŠ₯
[π—Œπ—Žπ–Ό(𝑛)]β‰”βŠ€βŠ•[𝑛]

These obey nice algebraic properties.

[π‘›π‘š]β‰…[𝑛]Γ—[π‘š]
[𝑛+π‘š]β‰…[𝑛]βŠ•[π‘š]
[π‘›π‘š]β‰…[π‘š]β†’[𝑛]

Definition. A Grammar of Finite Cardinals finite-cardinal-grammar

Define a grammar for each finite cardinal 𝑛:β„•.

[0]≔βŠ₯
[π—Œπ—Žπ–Ό(𝑛)]β‰”βŠ€βŠ•[𝑛]

Then define the grammar of all finite cardinals as

π–₯𝗂𝗇≔⨁𝑛:β„•[𝑛]

Definition. Finite Unambiguity finite-unambiguity

Define a grammar 𝐴 to be finitely unambiguous if 𝐴&𝐴≅𝐴.

There is likely a better name for this.

Finite Unambiguity is Not Equivalent to Unambiguity finite-unambiguity-not-unambiguity

For a while I believed finite unambiguity to be equivalent to the other definitions of unambiguity.

Definition 0.18. Finite Unambiguity finite-unambiguity

Define a grammar 𝐴 to be finitely unambiguous if 𝐴&𝐴≅𝐴.

There is likely a better name for this.

Definition 0.19. Unambiguity as Subterminality unambiguity-as-subterminality

A grammar 𝐴 is unambiguous if the unique map into the terminal object is a monomorphism. That is, 𝐴 is a subobject of ⊀.

Definition 0.20. Unambiguity as Unique Map into Codomain unambiguity-as-unique-map

A grammar 𝐴 is unambiguous if for all grammars 𝐡 and maps 𝑒,𝑒′:𝐴⊒𝐡 we have 𝑒=𝑒′.

In a category with terminal objects, this is equivalent to unambiguity defined via subterminality.

Definition 0.20.1. Unambiguity as Subterminality unambiguity-as-subterminality

A grammar 𝐴 is unambiguous if the unique map into the terminal object is a monomorphism. That is, 𝐴 is a subobject of ⊀.

We may use an analogy from the category of sets, however I had missed the infinite case when translating this idea to grammars.

It is true that if a grammar is unambiguous, then it is finitely unambiguous. However, the converse does not hold unless the grammar has finitely many parse trees for each string. This finiteness condition is a semantic one. If this proof of unambiguity were to be internalized, then it could maybe be captured through the lens of some grammar of finite cardinals.

Let 𝐴≔⨁𝑛:β„•βŠ€. 𝐴 is finitely unambiguous but not unambiguous. The isomorphism between 𝐴 and 𝐴&𝐴 amounts to building a bijection between β„• and β„•Γ—β„•, which is straightforward.

What Connectives Preserve Finiteness? finiteness-preserving-connectives

In order to bridge the gap between finite unambiguity and unambiguity, we can try to restrict to grammars that have finitely many parse trees. We’d expect this to address the concern semantically, as that fixes the problem when the parses are interpreted in π’πžπ­.

We can define when a grammar is finite, and then try to prove that finiteness is preserved on some sane operations on grammars. For instance, &, βŠ•, and β†’ each preserve finiteness. I haven’t proven this myself (which would be a useful exercise), but the following is substantiated in any topos with a natural numbers object [johnstone-2002].

0.21 Finite Cardinal Arithmetic in a Topos finite-cardinal-arithmetic-topos

In a topos with a natural numbers object (𝑁,𝑧,𝑠), you can define finite cardinals as objects that arise as the pullback along a morphism 𝑝:βŠ€β†’π‘ of the generic finite cardinal. Write [𝑝] for the cardinal corresponding to 𝑝.

The above characterization is a little obtuse and does warrant some more explanation. One way to make it more concrete is that [𝑧]=βŠ₯ and [π‘ βˆ˜π‘›]=βŠ€βŠ•[𝑛], and that finite cardinals warrant a nice induction principle. If 𝑃 is a property expressible in the internal language such that βŠ₯ satisfies 𝑃, and that whenever 𝐴 satisfies 𝑃 then βŠ€βŠ•π΄ satisfies 𝑃; then every finite cardinal satisfies 𝑃. That is, 𝑃 forms a (𝑧,𝑠)-closed subobject of 𝑁 and thus 𝑃 is all of 𝑁.

My working mental model is in a presheaf topos, where the natural numbers object can be defined explicitly as Ξ”(β„•).

Definition 0.21.1. Constant Presheaf constant-presheaf

The constant presheaf of a set 𝑋 on a category 𝐢 is a functor Ξ”(𝑋):πΆπ‘œπ‘β†’π’πžπ­ such that

Ξ”(𝑋)𝑐≔𝑋

I often call this the discrete presheaf for 𝑋, but I don’t know if that’s standard.

The finite cardinals with respect to Ξ”(β„•) can be characterized then as,

[0]≔βŠ₯
[π—Œπ—Žπ–Ό(𝑛)]β‰”βŠ€βŠ•[𝑛]

These obey nice algebraic properties.

[π‘›π‘š]β‰…[𝑛]Γ—[π‘š]
[𝑛+π‘š]β‰…[𝑛]βŠ•[π‘š]
[π‘›π‘š]β‰…[π‘š]β†’[𝑛]

The case of βŠ— is more problematic. Semantically, for all 𝑀:String we may bound the size of the set of parse trees

|(π΄βŠ—π΅)𝑀|≀Σ𝑀1++𝑀2=𝑀|𝐴𝑀1||𝐡𝑀2|

Precisely knowing this bound isn’t too important, but certainly it exists. We could capture this behavior by just adding an axiom that if 𝐴 and 𝐡 are each finite, then π΄βŠ—π΅ is finite. Although, I’d rather not add an axiom.

We could directly try to prove that βŠ— preserves finiteness. The statement would follow from showing that βŠ— preserves monomorphisms. That is, if 𝑓:𝐴β†ͺ𝐢 and 𝑔:𝐡β†ͺ𝐷, it suffices to show that π‘“βŠ—π‘”:π΄βŠ—π΅β†’πΆβŠ—π· is a monomorphism. This would imply finiteness, because you may then apply this for 𝐢 and 𝐷 as π–₯𝗂𝗇. However, you would also need to show that π–₯π—‚π—‡βŠ—π–₯𝗂𝗇 is finite, which isn’t immediately clear.

Internal Finiteness internally-finite-grammar

Define a grammar 𝐴 to be finite (or perhaps subfinite) if it is a subobject of grammar of finite cardinals.

𝐴β†ͺπ–₯𝗂𝗇

Where π–₯𝗂𝗇 is defined as follows.

Definition 0.22. A Grammar of Finite Cardinals finite-cardinal-grammar

Define a grammar for each finite cardinal 𝑛:β„•.

[0]≔βŠ₯
[π—Œπ—Žπ–Ό(𝑛)]β‰”βŠ€βŠ•[𝑛]

Then define the grammar of all finite cardinals as

π–₯𝗂𝗇≔⨁𝑛:β„•[𝑛]

Star Continuity is a Semantic Property star-continuity-semantic-property

Star continuity in Dependent Lambek Calculus can often be a convenient proof technique, but it’s important to remember that this shouldn’t be the first line of defense.

Star continuity holds in the Agda model, but does it hold in the syntactic model? I believe that it does because of the presence of the indexed coproducts. So perhaps it isn’t so sinister after all. It is worth noting that much of the reasoning performed by inducting on the length of a Kleene star isn’t very elegant. If a proof necessitates star continuity, then it doesn’t seem to be aided greatly by the type system.

Definition. Unambiguity as Subterminality unambiguity-as-subterminality

A grammar 𝐴 is unambiguous if the unique map into the terminal object is a monomorphism. That is, 𝐴 is a subobject of ⊀.

Definition. Unambiguity as Unique Map into Codomain unambiguity-as-unique-map

A grammar 𝐴 is unambiguous if for all grammars 𝐡 and maps 𝑒,𝑒′:𝐴⊒𝐡 we have 𝑒=𝑒′.

In a category with terminal objects, this is equivalent to unambiguity defined via subterminality.

Definition 0.23. Unambiguity as Subterminality unambiguity-as-subterminality

A grammar 𝐴 is unambiguous if the unique map into the terminal object is a monomorphism. That is, 𝐴 is a subobject of ⊀.

Definition. Unambiguity via the Diagonal Being an Isomorphism unambiguity-via-diagonal

Finite unambiguity does not serve as an adequate definition of unambiguity that is equivalent to unambiguity as subterminality and unambiguity as a unique map. However, the definition attempted via finite unambiguity can be refined to something that is equivalent to these.

A grammar 𝐴 is unambiguous if Ξ”:𝐴⊒𝐴&𝐴 is an isomorphism. Equivalently, 𝐴 is unambiguous if πœ‹1:𝐴&𝐴⊒𝐴 and πœ‹2:𝐴&𝐴⊒𝐴 are equal.

Definition. Subobject subobject

In a category π’žοΈ€, a subobject of 𝑐 is an isomorphism class of monomorphisms into 𝑐.

Definition. Terminal Object terminal-object

An object 𝑐 in a category π’žοΈ€ is terminal if there is a unique morphism from any other object into it.

Because terminal objects are unique up to unique isomorphism (as are all universal objects) we often just write ⊀ to refer to the terminal object. Likewise, !:π‘‘β†’βŠ€ refers to the unique morphism into ⊀.

Definition. Equalizer equalizer

Let 𝑐 and 𝑑 be objects in a category π’žοΈ€ with two parallel morphisms 𝑓,𝑔:𝑐→𝑑. The equalizer of 𝑓 and 𝑔, if it exists, is the universal object π–Ύπ—Š with the following property:

  • There is a morphism πœ‹:π–Ύπ—Šβ†’π‘
  • π‘“βˆ˜πœ‹=π‘”βˆ˜πœ‹

Constructing Equalizers in Type Theory equalizers-in-type-theory

In the presence of Ξ£-types, one may construct all equalizers. Given types 𝐴 and 𝐡 with functions 𝑓,𝑔:𝐴→𝐡, the equalizer may be constructed as

π–Ύπ—Šπ‘“,π‘”β‰”βˆ‘π‘Ž:𝐴(𝑓(π‘Ž)=𝑔(π‘Ž))

Definition. Subobject Classifier subobject-classifier

Subsets 𝐴 of a set 𝑋 may classically be identified with a characteristic map πœ’π΄:π‘‹β†’π–»π—ˆπ—ˆπ—…. Intuitively, for every π‘₯:𝑋, πœ’π΄ gives a truth value to the statement β€œπ‘₯ is in the subset 𝐴”. In this manner, the domain of the characteristic map, π–»π—ˆπ—ˆπ—…, classifies the subsets of 𝑋.

Generalizing over this principle, in a category π’žοΈ€ an object Ξ© is a subobject classifier if maps into it from some object 𝑐 likewise uniquely identify a subobject of 𝑐.

We can understand Ξ© to behave like an object of truth values that are not necessarily boolean valued. A morphism 𝑝:𝑐→Ω can be thought of like a predicate on 𝑐. If 𝑐 were a set, this would precisely be the characteristic function on it. However, this idea can generalize beyond sets. For instance in the category of graphs, Ξ© is a cleverly constructed graph such that any graph homomorphism 𝑔→Ω into it picks out a unique subgraph of 𝑔.

There are always two (suggestively named) disjoint β€œpoints” of Ξ©, thought of as morphisms out of the terminal object π—π—‹π—Žπ–Ύ,π–Ώπ–Ίπ—…π—Œπ–Ύ:βŠ€β†’Ξ©. I think if we’re being careful, π—π—‹π—Žπ–Ύ may properly be the β€œsubobject classifier” but I always use the term to refer to Ξ© itself.

A subobject πœ™:𝑑β†ͺ𝑐 induces a unique characteristic morphism π‘πœ™:𝑐→Ω such that

π‘πœ™βˆ˜πœ™=π—π—‹π—Žπ–Ύβˆ˜!𝑑

Moreover, the appropriate square must be a pullback.

Definition. Kleene Star in Dependent Lambek Calculus kleene-star

For a grammar 𝐴, the Kleene star π΄βˆ— is defined as a least-fixed point,

πœ‡π‘₯.πœ–βŠ•(π΄βŠ—π‘₯)

Definition. Star Continuity star-continuity

A Kleene Algebra is star continuous if for all π‘₯,𝑦,𝑧

sup𝑛β‰₯0π‘₯𝑦𝑛𝑧=π‘₯π‘¦βˆ—π‘§

Definition. Star Continuity in Dependent Lambek Calculus star-continuity-in-dependent-lambek

For a grammar 𝐴, the Kleene star π΄βˆ— is isomorphic to an indexed coproduct.

π΄βˆ—β‰…β¨π‘›:β„•π΄βŠ—π‘›

That is, we may view the parses of π΄βˆ— like a linear list comprising parses of 𝐴 concatenated together. Further, for each of these lists we may know the precise length.

When viewing Dependent Lambek Calculus as a model of Kleene algebra, this is precisely the statement that star continuity holds.

Definition. First Set first-set

The first set (π–₯π—‚π—‹π—Œπ—(β‹…)) of a grammar 𝐴 are all the characters that may appear at the beginning of a word in the language of 𝐴.

π–₯π—‚π—‹π—Œπ—(𝐴)={𝑐|βˆƒπ‘€.π‘π‘€βˆˆπΏ(𝐴)}

Definition. First Sets in Dependent Lambek Calculus first-set-in-dependent-lambek

The first set of a grammar 𝐴 may be captured in Lambek𝙳 via the following proposition:

π‘βˆ‰π–₯π—‚π—‹π—Œπ—(𝐴)≔𝐴&('𝑐'βŠ—βŠ€)⊒βŠ₯

Or perhaps with ones of the grammars

𝐴⇒¬('𝑐'βŠ—βŠ€),'𝑐'βŠ—βŠ€β‡’Β¬π΄

Definition. FollowLast Set followlast-set

The followlast set (π–₯π–«π–Ίπ—Œπ—(β‹…)) of a grammar 𝐴 are all the characters that may follow a word in the language of 𝐴 in a string that is in the language of 𝐴.

π–₯π–«π–Ίπ—Œπ—(𝐴)={𝑐|βˆƒπ‘€,𝑀𝐴.π‘€π΄βˆˆπΏ(𝐴)βˆ§π‘€π΄π‘π‘€βˆˆπΏ(𝐴)}

Definition. FollowLast Sets in Dependent Lambek Calculus followlast-set-in-dependent-lambek

The followlast set of a grammar 𝐴 may be captured in Lambek𝙳 via the following proposition:

π‘βˆ‰π–₯π–«π–Ίπ—Œπ—(𝐴)≔𝐴&(π΄βŠ—'𝑐'βŠ—βŠ€)⊒βŠ₯

Or perhaps with ones of the grammars

𝐴⇒¬(π΄βŠ—'𝑐'βŠ—βŠ€),π΄βŠ—'𝑐'βŠ—βŠ€β‡’Β¬π΄

Definition. LL(1) Condition ll1-condition

A context-free grammar satisfies the LL(1) condition if it satisfies the following three conditions:

  • All of its productions have pairwise disjoint first sets
  • If a concatenation of nonterminals π΄βŠ—π΅ appears in a production, then 𝐴 has a disjoint followlast set from the first set of 𝐡
  • At most one production is nullable

This is essentially the type system of [1], which characterizes the LL(1) condition for context-free expressions.

Intutively, an LL(1) grammar can be parsed unambiguously, and without backtracking, by a predictive parser that only needs one token of lookahead.

Definition. Nullability in Dependent Lambek Calculus nullability-in-dependent-lambek

The nullability (π–­π—Žπ—…π—…(β‹…)) of a grammar 𝐴 may be captured in Lambek𝙳 via the following proposition:

Β¬π–­π—Žπ—…π—…(𝐴)≔𝐴&πœ–βŠ’βŠ₯

Or perhaps with one of the grammars

π΄β‡’Β¬πœ€,πœ€β‡’Β¬π΄

Definition. Nullable Grammar nullable-grammar

A grammar 𝐴 is nullable if the empty string belongs to the language of 𝐴.

Definition. Sequential Unambiguity sequential-unambiguity

Grammars 𝐴 and 𝐡 are sequentially unambiguous if the followlast set of 𝐴 is disjoint from the first set of 𝐡.

π–₯π–«π–Ίπ—Œπ—(𝐴)∩π–₯π—‚π—‹π—Œπ—(𝐡)=βˆ…

We can understand this intuitively by characterizing the behavior of a left-to-right parser of π΄βŠ—π΅. First it searches for a parse of 𝐴, then upon finding a character that is not in π–₯π–«π–Ίπ—Œπ—(𝐴) it may begin trying search for 𝐡.

That is, there is a unique boundary between the 𝐴-parse and the 𝐡-parse.

Definition. Closed Monoidal Structure closed-monoidal-category

A monoidal category is left closed if for each π‘ŒβˆˆπΆ, the functor βˆ’βŠ—π‘Œ:𝐢→𝐢 has a right adjoint π‘ŒβŠΈβˆ’:𝐢→𝐢 forming the left internal-hom out of π‘Œ.

That is, for all 𝑋,π‘Œ,𝑍, there is a natural isomorphism 𝐢[π‘‹βŠ—π‘Œ,𝑍]≅𝐢[𝑋,[π‘Œ,𝑍]]

There is an obvious right-handed variant π‘ŒβŸœβˆ’ that is right adjoint to π‘ŒβŠ—βˆ’.

If 𝐢 is both right and left closed, the monoidal category is simply called closed (or perhaps biclosed).

Am I on to Something? am-i-onto-something

Here I am documenting the half-baked ideas I’ve had but have never taken anywhere. On these I’ve either been out of depth on the requisite knowledge, rediscovered something that isn’t as new as I’d hoped, or simply don’t have the time. I hope one day I can return to these and flesh them out.

2 entries

0.24 GrΓΆbner Bases for Inferring Polynomial Loop Invariants groebner-loop-invariants

When the tools in I4: Incremental inference of inductive invariants for verification of distributed protocols and On Symmetry and Quantification: A New Approach to Verify Distributed Protocols search for an inductive invariant of a distributed system, the search procedure instantiates a series of small finite models and tries to infer from their truth tables a series of logical formulae that hold over those finite models. These formulae are found by running the Quine-McCluskey algorithm for minimization of Boolean functions. The prime implicants found by Quine-McCluskey have a latent symmetry that can be abstracted into quantified formulae. There are only so many small numbers, and so small finite models may propose formulae that do not hold at larger sizes. However, if you find a formula that holds at size 𝑛 as well as size 𝑛+1, then it is likely a good candidate to hold at all sizes. You need to be a little careful if your protocol is indexed by several variables, but mostly this general idea holds when abstracting to a protocol of unbounded size. Further discussion of this idea can be found in SAT-based quantified symmetric minimization of the reachable states of distributed protocols: An update.

My observation was that Quine-McCluskey is just a special instance of Buchberger’s algorithm for computing GrΓΆbner Bases. That is, you can describe Boolean formulae as polynomials over the field with two elements, and in this translation Quine-McCluskey and Buchberger each compute the same data. This observation isn’t new in and of itself, but it does open up an opportunity to generalize the invariant search procedure that is used above.

The place I went looking to apply this idea was in the search of polynomial loop invariants. If a loop had an invariant that is expressible as a polynomial relation between the program variables, then you could apply the same idea as above to infer the loop invariant.

Suppose π‘₯1,…,π‘₯𝑛 are the variables in scope of program 𝑆 and 𝑆 contains a loop that we want to infer an invariant for. Denote the value of π‘₯𝑖 at the 𝑗-th loop iteration by π‘₯𝑖,𝑗. The invariant search procedure proceeds intuitively as the following: we will keep track of the minimal set of polynomials that could interpolate between all of the variable assignments that we have witnessed thus far. Formally this is kept track of by the ideal of polynomials. At the 𝑗-th loop iteration we add a new generator to the ideal which corresponds to the assignments π‘₯1,𝑗,…,π‘₯𝑛,𝑗. The GrΓΆbner basis for this ideal provides the minimal data needed to generate all the assignments witnessed thus far, so if this process saturates then the GrΓΆbner basis encodes a polynomial loop invariant. The nice thing about polynomials is that they have finite degree which guarantees that this process does indeed saturate (provided that the degree of the invariant is smaller than the number of loop iterations).

I was so excited to find this idea. I’d felt like it was my first good idea in grad school. Then I read Automatic Generation of Polynomial Loop Invariants: Algebraic Foundations and found out someone had done this 20 years ago. I still wonder from time to time if there is room to further refine this idea or perhaps further generalize it. For instance, there is further generalization beyond Quine-McCluskey or Buchberger to the Knuth-Bendix algorithm, which seems to be a more general instance of both of these algorithms. So perhaps this search procedure can be weakened to an even more general class? Although, I’m not yet familiar much with the Knuth-Bendix algorithm.

0.25 A Method for Verifying Translational Invariance of Image Processing Neural Networks translation-invariance-verification

My understanding for how one may prove a safety property 𝑃 for a neural network 𝑁 is as follows. First, express 𝑃 as an input-output property. That is, choose some region 𝐷 in the domain of 𝑁 to represent the inputs of interest. Further choose some region 𝐴 in the codomain of 𝑁 to describe safe outputs. Then describe 𝑃 as the property:

βˆ€π‘₯∈𝐷.𝑁(π‘₯)∈𝐴

For instance, this is more or less how Reluplex, Marabou, and 𝛼𝛽-CROWN each work. Squinting my eyes, this is the only such method for verifying a safety property for 𝑁. That is, this is the only way to get a 100% guarantee that 𝑃 holds rather than some high measure of confidence.

I believe in this formalism I have an idea for how to express the property β€œπ‘ is translationally invariant” where 𝑁 is an image classifying net.

To this end, we need a continuous artifact that captures what it means to translate an image. We can express an image as a matrix of pixels 𝐼 (or perhaps several parallel matrices if we care about color channels, but stick to a single grayscale matrix for now). To shift 𝐼 over by a single pixel, we may left-multiply 𝐼 by the shift matrix 𝑆, where 𝑆 is the matrix filled with zeros and has 1β€²s on the subdiagonal. Note that all the directions of shifting 𝐼 are captured by the combinations of left/right multiplication by 𝑆, 𝑆𝑇.

Shifting by multiple pixels is now expressed by the matrix 𝑆𝑛𝐼, however this is still a discrete dynamical system. We don’t have a continuous object by which we can test our safety property. My initial thought to continuousify this system was to express some sort of exponential flow by 𝑆. That is, consider the matrix

π‘†π‘‘πΌβ‰”π‘’π‘‘π—…π—ˆπ—€(𝑆)

𝑆𝑑𝐼 then captures what it means to translate the image 𝐼 over by 𝑑, a real-valued β€œamount of shifting”. This choice of continuous artifact could then be used for a verification effort, however there are some issues related to numerical stability because 𝑆 isn’t invertible. Because it is not invertible, π—…π—ˆπ—€(𝑆) doesn’t actually exist. So to make the above construction work, we need to mildly perturb 𝑆 and take a pseudoinverse. This sort of works and makes it so 𝑆𝑑𝐼 does capture some real valued shift, but it is only accurate when 𝑑 is small. So this is maybe problematic for our verification effort. This may be resolvable by chopping up the problem into subproblems, each of which is thin enough for the current iteration is accurate enough on that subproblem. However, there are two big things to consider.

  1. The exponential approach above is probably too complicated. It seems

likely that you may be able to take some (sequence of) linear interpolation(s) between the 𝑆𝑛𝐼’s. This will still be some continuous object that captures a real-valued shift and it will be much more stable than the approach above.

  1. Even if we sort out which continuous object represents translation

of an image, I cannot for the life of me train any neural network that is translationally invariant. So the proof method is useless if there is nothing that it would ever apply to.

Precisely in this last point, I have mostly focused on trying to train a small CNN for MNIST handwritten digit classification that preserves the output class for small, reasonable translations of the digit. I’ve used data augmentation to predispose the network to being translationally invariant, and even though I can get a high degree of invariance, I cannot get a network that is invariant for all of the examples even in the training set or a reserved testing set.

It may be the case that the method I propose for measuring this invariance could be used to adversarially train a network to have better invariance. It may also be the case that all CNNs are bound to suffer from small degrees of translational sensitivity. I cannot find the citation at the moment, but there was a paper that suggested that CNNs suffer from weird issues of translational sensitivity that relate to the size of the convolutional window. So maybe this approach is doomed to fail anyway.

On the whole, I will say that machine learning verification almost sounds like an oxymoron. That is, if you have the expressivity to properly state a sophisticated safety property, then you likely understand the problem enough to not need to resort to machine learning in the first place. So almost tautologically, it seems that there cannot be satisfying verification of neural nets, as the tasks of machine learning and verification live on very different epistemic foundations.

The related works I could liberate from Zotero may be found below.

15 entries

0.25.1 Adequate Losses via Quantitative Linear Logic capucci-2026-adequate

As neural components are increasingly embedded in existing symbolic software – including safety-critical systems – the question arises of how to specify and enforce the safety of the newly introduced neural parts. Unlike traditional logical specifications, these must be amenable not only to the standard Boolean interpretation, but also to training and optimisation. The latter calls for a quantitative interpretation of the logical syntax, subject to further requirements such as smoothness and differentiability. Moreover, the qualitative and quantitative sides of the logic must share a unifying proof-theoretic and categorical semantics. Finally, the new logic should link cleanly to the substructural and program logics that underpin the verification of existing symbolic programs. In this paper, we present a logic that ticks all of these boxes. We introduce a family of calculi, pQLL, indexed by a hardness degree 𝑝, prove a cut-elimination theorem for them, and establish completeness with respect to enriched residuated β€˜soft’ lattices. At 𝑝=∞, pQLL reduces to multiplicative additive linear logic (MALL), and provability in pQLL converges to provability in MALL as π‘β†’βˆž. We express optimisation objectives in the syntax of this logic and prove the quantitative adequacy of neuro-symbolic loss functions – a result that has eluded the neuro-symbolic machine learning community for nearly a decade.
DOI Β· arXiv

0.25.2 Quantitative Linear Logic for Neuro-Symbolic Learning and Verification flinkow-2026-quantitative

Differentiable Logics are deployed in neuro-symbolic learning tasks as a way of embedding logical constraints in the training objective of neural networks. A differentiable logic consists of a syntax to write logical properties and a semantics to interpret them as real-valued functions to be folded in the loss function. A defining trade-off of the field is that between logical properties of the connectives, and analytic concerns for the semantics, with both aspects being relevant in applications. At one extreme we find fuzzy logics, that have well-established algebraic and proof-theoretic foundations, and at the other ad-hoc differentiable logics like Fischer’s DL2, conceived for deep learning applications. However, no satisfactory foundation has emerged yet. We propose a resolution to this long-standing tension via a novel logic, Quantitative Linear Logic (QLL), with foundational ambitions. Our design is driven by naturality – the idea that, since logical constraints are translated to losses, the semantics of the connectives should be pertinent operations used in ML practice (that is, sum and log-sum-exp) on additive quantities (like logits). We then judge the result on two aspects: logical adequacy – that they satisfy most of the standard logical laws of Linear Logic; and empirical effectiveness – test-time performance (as measured by adversarial attacks) is well-correlated to the actual verification of the logical constraints (as measured by off-the-shelf neural network verifiers), which makes QLL stand out among SoTA techniques.
DOI Β· arXiv

0.25.3 Quantifiers for Differentiable Logics in Rocq (Extended Abstract) marulandagiraldo-2025-quantifiers

DOI

0.25.4 Fundamental Components of Deep Learning: A category-theoretic approach gavranovicFundamentalComponentsDeep

Deep learning, despite its remarkable achievements, is still a young field. Like the early stages of many scientific disciplines, it is marked by the discovery of new phenomena, ad-hoc design decisions, and the lack of a uniform and compositional mathematical foundation. From the intricacies of the implementation of backpropagation, through a growing zoo of neural network architectures, to the new and poorly understood phenomena such as double descent, scaling laws or in-context learning, there are few unifying principles in deep learning. This thesis develops a novel mathematical foundation for deep learning based on the language of category theory. We develop a new framework that is a) end-to-end, b) unform, and c) not merely descriptive, but prescriptive, meaning it is amenable to direct implementation in programming languages with sufficient features. We also systematise many existing approaches, placing many existing constructions and concepts from the literature under the same umbrella. In Part I we identify and model two main properties of deep learning systems parametricity and bidirectionality by we expand on the previously defined construction of actegories and Para to study the former, and define weighted optics to study the latter. Combining them yields parametric weighted optics, a categorical model of artificial neural networks, and more. Part II justifies the abstractions from Part I, applying them to model backpropagation, architectures, and supervised learning. We provide a lens-theoretic axiomatisation of differentiation, covering not just smooth spaces, but discrete settings of boolean circuits as well. We survey existing, and develop new categorical models of neural network architectures. We formalise the notion of optimisers and lastly, combine all the existing concepts together, providing a uniform and compositional framework for supervised learning.
DOI

0.25.5 Architecture-Preserving Provable Repair of Deep Neural Networks taoArchitecturePreservingProvableRepair2023

Deep neural networks (DNNs) are becoming increasingly important components of software, and are considered the state-of-the-art solution for a number of problems, such as image recognition. However, DNNs are far from infallible, and incorrect behavior of DNNs can have disastrous real-world consequences. This paper addresses the problem of architecture-preserving V-polytope provable repair of DNNs. A V-polytope defines a convex bounded polytope using its vertex representation. V-polytope provable repair guarantees that the repaired DNN satisfies the given specification on the infinite set of points in the given V-polytope. An architecture-preserving repair only modifies the parameters of the DNN, without modifying its architecture. The repair has the flexibility to modify multiple layers of the DNN, and runs in polynomial time. It supports DNNs with activation functions that have some linear pieces, as well as fully-connected, convolutional, pooling and residual layers. To the best our knowledge, this is the first provable repair approach that has all of these features. We implement our approach in a tool called APRNN. Using MNIST, ImageNet, and ACAS Xu DNNs, we show that it has better efficiency, scalability, and generalization compared to PRDNN and REASSURE, prior provable repair methods that are not architecture preserving. CCS Concepts: β€’ Computing methodologies β†’ Neural networks; β€’ Theory of computation β†’ Linear programming; β€’ Software and its engineering β†’ Software post-development issues.
PDF Β· DOI Β· pldb

0.25.6 Compiling Higher-Order Specifications to SMT Solvers: How to Deal with Rejection Constructively daggitt-2023-compiling

PDF Β· DOI Β· pldb

0.25.7 Vehicle: Interfacing Neural Network Verifiers with Interactive Theorem Provers daggitt-2022-vehicle

Verification of neural networks is currently a hot topic in automated theorem proving. Progress has been rapid and there are now a wide range of tools available that can verify properties of networks with hundreds of thousands of nodes. In theory this opens the door to the verification of larger control systems that make use of neural network components. However, although work has managed to incorporate the results of these verifiers to prove larger properties of individual systems, there is currently no general methodology for bridging the gap between verifiers and interactive theorem provers (ITPs). In this paper we present Vehicle, our solution to this problem. Vehicle is equipped with an expressive domain specific language for stating neural network specifications which can be compiled to both verifiers and ITPs. It overcomes previous issues with maintainability and scalability in similar ITP formalisations by using a standard ONNX file as the single canonical representation of the network. We demonstrate its utility by using it to connect the neural network verifier Marabou to Agda and then formally verifying that a car steered by a neural network never leaves the road, even in the face of an unpredictable cross wind and imperfect sensors. The network has over 20,000 nodes, and therefore this proof represents an improvement of 3 orders of magnitude over prior proofs about neural network enhanced systems in ITPs.
arXiv

0.25.8 PRIMA: General and Precise Neural Network Certification via Scalable Convex Hull Approximations mullerPRIMAGeneralPrecise2022

Formal verification of neural networks is critical for their safe adoption in real-world applications. However, designing a precise and scalable verifier which can handle different activation functions, realistic network architectures and relevant specifications remains an open and difficult challenge. In this paper, we take a major step forward in addressing this challenge and present a new verification framework, called PRIMA. PRIMA is both (i) general: it handles any non-linear activation function, and (ii) precise: it computes precise convex abstractions involving multiple neurons via novel convex hull approximation algorithms that leverage concepts from computational geometry. The algorithms have polynomial complexity, yield fewer constraints, and minimize precision loss. We evaluate the effectiveness of PRIMA on a variety of challenging tasks from prior work. Our results show that PRIMA is significantly more precise than the state-of-the-art, verifying robustness to input perturbations for up to 20%, 30%, and 34% more images than existing work on ReLU-, Sigmoid-, and Tanh-based networks, respectively. Further, PRIMA enables, for the first time, the precise verification of a realistic neural network for autonomous driving within a few minutes.
PDF Β· DOI Β· arXiv Β· pldb

0.25.9 Provable repair of deep neural networks sotoudehProvableRepairDeep2021

Deep Neural Networks (DNNs) have grown in popularity over the past decade and are now being used in safety-critical domains such as aircraft collision avoidance. This has motivated a large number of techniques for finding unsafe behavior in DNNs. In contrast, this paper tackles the problem of correcting a DNN once unsafe behavior is found. We introduce the provable repair problem, which is the problem of repairing a network N to construct a new network Nβ€² that satisfies a given specification. If the safety specification is over a finite set of points, our Provable Point Repair algorithm can find a provably minimal repair satisfying the specification, regardless of the activation functions used. For safety specifications addressing convex polytopes containing infinitely many points, our Provable Polytope Repair algorithm can find a provably minimal repair satisfying the specification for DNNs using piecewise-linear activation functions. The key insight behind both of these algorithms is the introduction of a Decoupled DNN architecture, which allows us to reduce provable repair to a linear programming problem. Our experimental results demonstrate the efficiency and effectiveness of our Provable Repair algorithms on a variety of challenging tasks.
PDF Β· DOI Β· pldb

0.25.10 Tracking translation invariance in CNNs myburghTrackingTranslationInvariance2021

Although Convolutional Neural Networks (CNNs) are widely used, their translation invariance (ability to deal with translated inputs) is still subject to some controversy. We explore this question using translation-sensitivity maps to quantify how sensitive a standard CNN is to a translated input. We propose the use of Cosine Similarity as sensitivity metric over Euclidean Distance, and discuss the importance of restricting the dimensionality of either of these metrics when comparing architectures. Our main focus is to investigate the effect of different architectural components of a standard CNN on that network’s sensitivity to translation. By varying convolutional kernel sizes and amounts of zero padding, we control the size of the feature maps produced, allowing us to quantify the extent to which these elements influence translation invariance. We also measure translation invariance at different locations within the CNN to determine the extent to which convolutional and fully connected layers, respectively, contribute to the translation invariance of a CNN as a whole. Our analysis indicates that both convolutional kernel size and feature map size have a systematic influence on translation invariance. We also see that convolutional layers contribute less than expected to translation invariance, when not specifically forced to do so.
Web Β· arXiv

0.25.11 Towards Verified Artificial Intelligence seshiaVerifiedArtificialIntelligence2020

Verified artificial intelligence (AI) is the goal of designing AI-based systems that have strong, ideally provable, assurances of correctness with respect to mathematically-specified requirements. This paper considers Verified AI from a formal methods perspective. We describe five challenges for achieving Verified AI, and five corresponding principles for addressing these challenges.
DOI Β· arXiv

0.25.12 Verification of Deep Convolutional Neural Networks Using ImageStars tranVerificationDeepConvolutional2020

Convolutional Neural Networks (CNN) have redefined stateof-the-art in many real-world applications, such as facial recognition, image classification, human pose estimation, and semantic segmentation. Despite their success, CNNs are vulnerable to adversarial attacks, where slight changes to their inputs may lead to sharp changes in their output in even well-trained networks. Set-based analysis methods can detect or prove the absence of bounded adversarial attacks, which can then be used to evaluate the effectiveness of neural network training methodology. Unfortunately, existing verification approaches have limited scalability in terms of the size of networks that can be analyzed.
PDF Β· DOI Β· arXiv Β· pldb

0.25.13 Stride and Translation Invariance in CNNs moutonStrideTranslationInvariance2020

Convolutional Neural Networks have become the standard for image classification tasks, however, these architectures are not invariant to translations of the input image. This lack of invariance is attributed to the use of stride which ignores the sampling theorem, and fully connected layers which lack spatial reasoning. We show that stride can greatly benefit translation invariance given that it is combined with sufficient similarity between neighbouring pixels, a characteristic which we refer to as local homogeneity. We also observe that this characteristic is dataset-specific and dictates the relationship between pooling kernel size and stride required for translation invariance. Furthermore we find that a trade-off exists between generalization and translation invariance in the case of pooling kernel size, as larger kernel sizes lead to better invariance but poorer generalization. Finally we explore the efficacy of other solutions proposed, namely global average pooling, anti-aliasing, and data augmentation, both empirically and through the lens of local homogeneity.
DOI Β· arXiv

0.25.14 Why do deep convolutional networks generalize so poorly to small image transformations? azulayWhyDeepConvolutional

Convolutional Neural Networks (CNNs) are commonly assumed to be invariant to small image transformations: either because of the convolutional architecture or because they were trained using data augmentation. Recently, several authors have shown that this is not the case: small translations or rescalings of the input image can drastically change the network’s prediction. In this paper, we quantify this phenomena and ask why neither the convolutional architecture nor data augmentation are sufficient to achieve the desired invariance. Specifically, we show that the convolutional architecture does not give invariance since architectures ignore the classical sampling theorem, and data augmentation does not give invariance because the CNNs learn to be invariant to transformations only for images that are very similar to typical images from the training set. We discuss two possible solutions to this problem: (1) antialiasing the intermediate representations and (2) increasing data augmentation and show that they provide only a partial solution at best. Taken together, our results indicate that the problem of insuring invariance to small image transformations in neural networks while preserving high accuracy remains unsolved.
DOI

0.25.15 Formal Verification of CNN-based Perception Systems kouvarosFormalVerificationCNNbased2018

We address the problem of verifying neural-based perception systems implemented by convolutional neural networks. We define a notion of local robustness based on affine and photometric transformations. We show the notion cannot be captured by previously employed notions of robustness. The method proposed is based on reachability analysis for feed-forward neural networks and relies on MILP encodings of both the CNNs and transformations under question. We present an implementation and discuss the experimental results obtained for a CNN trained from the MNIST data set.
DOI Β· arXiv

Definition. The Bicategory of Categories bicategory-of-categories

The bicategory of categories 𝖒𝖠𝖳 has

  • as 0-cells, categories (at a fixed pair of universe levels, for objects and for morphisms);
  • as hom-category 𝖒𝖠𝖳(π’žοΈ€,π’ŸοΈ€), the functor category [π’žοΈ€,π’ŸοΈ€], so 1-cells are functors and 2-cells are natural transformations;
  • as identity 1-cell, the identity functor;
  • as composition, 𝐹⋆𝐺=𝐺∘𝐹 on functors. On natural transformations 𝛽:𝐹⇒𝐹′ and 𝛾:𝐺⇒𝐺′ the horizontal composite is given directly by its components

    (𝛽⋆𝛾)𝑐=𝐺(𝛽𝑐)⋆𝛾𝐹′𝑐.

Every component of the left unitor, the right unitor and the associator is an identity morphism, and their inverses are identities too. So the only content of the triangle and pentagon is that composites of identities are identities.

𝖒𝖠𝖳 is still a bicategory and not a strict 2-category: Id∘𝐹 and 𝐹 agree on objects and on morphisms, but in the formalization they are not the same functor definitionally. The structure cells are there to name that agreement.

A monad in 𝖒𝖠𝖳 is an ordinary monad on a category, and a prestack is a pseudofunctor into 𝖒𝖠𝖳.

Definition. Category of Elements category-of-elements

Let 𝑃 be a presheaf on a category π’žοΈ€. The category of elements of 𝑃 is the displayed category over π’žοΈ€ whose displayed objects over 𝑐 are the elements π‘βˆˆπ‘ƒπ‘, and whose displayed morphisms over 𝑓:𝑐→𝑑 from 𝑝 to π‘ž are proofs that (𝑃𝑓)(π‘ž)=𝑝.

Since 𝑃𝑐 is a set, there is at most one displayed morphism over each 𝑓 between given elements: a morphism of elements is a morphism of π’žοΈ€ that happens to carry π‘ž back to 𝑝. Its total category is the classical category of elements βˆ«π‘ƒ, and a universal element of 𝑃 is exactly a terminal object of βˆ«π‘ƒ.

Definition. Corecursive algebra corecursive-algebra

Fix an endofunctor 𝐹:π’žοΈ€β†’π’žοΈ€. An algebra 𝛼:𝐹𝐡→𝐡 is corecursive when for every coalgebra 𝛾:𝑋→𝐹𝑋 there is exactly one hylomorphism from 𝛾 to 𝛼. Equivalently, the functor π–§π—’π—…π—ˆ(βˆ’,𝛼) of the hylomorphism profunctor is constantly a singleton. It is the dual of a recursive coalgebra: an 𝐹-algebra in π’žοΈ€ is corecursive exactly when it is recursive as an 𝐹op-coalgebra in π’žοΈ€op.

Example. If (𝜈𝐹,π—ˆπ—Žπ—) is a terminal coalgebra, then π—ˆπ—Žπ— is invertible and π—ˆπ—Žπ—βˆ’1:𝐹(𝜈𝐹)β†’πœˆπΉ is a corecursive algebra: a solution of β„Ž=π›Ύβ‹†πΉβ„Žβ‹†π—ˆπ—Žπ—βˆ’1 is the same thing as a solution of β„Žβ‹†π—ˆπ—Žπ—=π›Ύβ‹†πΉβ„Ž, that is, a coalgebra map into the terminal coalgebra, and there is exactly one, π—Žπ—‡π–Ώπ—ˆπ—…π–½π›Ύ.

Theorem. Day Convolution is Closed day-closed-structure

Let 𝒱︀ be a symmetric monoidal closed category that is complete and cocomplete. Let (π’žοΈ€,βŠ—π’žοΈ€,𝐼) be a small monoidal 𝒱︀-enriched category and 𝐴,𝐡 be 𝒱︀-enriched presheaves on π’žοΈ€. Define

(𝐴⊸𝐡)𝑐=βˆ«π‘’π΄π‘’βŠΈπ’±οΈ€π΅(π‘’βŠ—π’žοΈ€π‘).

Then, for Day convolution βŠ—Day,

π΄βŠ—Dayβˆ’βŠ£π΄βŠΈDayβˆ’.

Proof. Proof that Day Convolution is Closed day-closed-structure-proof

For 𝑋,𝐡:π’žοΈ€op→𝒱︀, the enriched hom in [π’žοΈ€op,𝒱︀] is given by the end:

βˆ«π‘((π΄βŠ—Day𝑋)π‘βŠΈπ’±οΈ€π΅π‘)β‰…βˆ«π‘((βˆ«π‘’,π‘£π’žοΈ€(𝑐,π‘’βŠ—π’žοΈ€π‘£)βŠ—π’±οΈ€π΄π‘’βŠ—π’±οΈ€π‘‹π‘£)βŠΈπ’±οΈ€π΅π‘)β‰…βˆ«π‘βˆ«π‘’,𝑣((π’žοΈ€(𝑐,π‘’βŠ—π’žοΈ€π‘£)βŠ—π’±οΈ€π΄π‘’βŠ—π’±οΈ€π‘‹π‘£)βŠΈπ’±οΈ€π΅π‘)β‰…βˆ«π‘’,𝑣((π΄π‘’βŠ—π’±οΈ€π‘‹π‘£)βŠΈπ’±οΈ€βˆ«π‘(π’žοΈ€(𝑐,π‘’βŠ—π’žοΈ€π‘£)βŠΈπ’±οΈ€π΅π‘))β‰…βˆ«π‘’,𝑣((π΄π‘’βŠ—π’±οΈ€π‘‹π‘£)βŠΈπ’±οΈ€π΅(π‘’βŠ—π’žοΈ€π‘£))β‰…βˆ«π‘£(π‘‹π‘£βŠΈπ’±οΈ€βˆ«π‘’(π΄π‘’βŠΈπ’±οΈ€π΅(π‘’βŠ—π’žοΈ€π‘£)))β‰…βˆ«π‘£(π‘‹π‘£βŠΈπ’±οΈ€(𝐴⊸𝐡)𝑣).

∎

Symmetrically, (𝐡⟜𝐴)𝑐=βˆ«π‘£π΄π‘£βŠΈπ’±οΈ€π΅(π‘βŠ—π’žοΈ€π‘£) and βˆ’βŠ—Dayπ΄βŠ£βˆ’βŸœπ΄, so the enriched presheaf category [π’žοΈ€op,𝒱︀] is biclosed [1].

Proof. Proof that Day Convolution is Closed day-closed-structure-proof

For 𝑋,𝐡:π’žοΈ€op→𝒱︀, the enriched hom in [π’žοΈ€op,𝒱︀] is given by the end:

βˆ«π‘((π΄βŠ—Day𝑋)π‘βŠΈπ’±οΈ€π΅π‘)β‰…βˆ«π‘((βˆ«π‘’,π‘£π’žοΈ€(𝑐,π‘’βŠ—π’žοΈ€π‘£)βŠ—π’±οΈ€π΄π‘’βŠ—π’±οΈ€π‘‹π‘£)βŠΈπ’±οΈ€π΅π‘)β‰…βˆ«π‘βˆ«π‘’,𝑣((π’žοΈ€(𝑐,π‘’βŠ—π’žοΈ€π‘£)βŠ—π’±οΈ€π΄π‘’βŠ—π’±οΈ€π‘‹π‘£)βŠΈπ’±οΈ€π΅π‘)β‰…βˆ«π‘’,𝑣((π΄π‘’βŠ—π’±οΈ€π‘‹π‘£)βŠΈπ’±οΈ€βˆ«π‘(π’žοΈ€(𝑐,π‘’βŠ—π’žοΈ€π‘£)βŠΈπ’±οΈ€π΅π‘))β‰…βˆ«π‘’,𝑣((π΄π‘’βŠ—π’±οΈ€π‘‹π‘£)βŠΈπ’±οΈ€π΅(π‘’βŠ—π’žοΈ€π‘£))β‰…βˆ«π‘£(π‘‹π‘£βŠΈπ’±οΈ€βˆ«π‘’(π΄π‘’βŠΈπ’±οΈ€π΅(π‘’βŠ—π’žοΈ€π‘£)))β‰…βˆ«π‘£(π‘‹π‘£βŠΈπ’±οΈ€(𝐴⊸𝐡)𝑣).

∎

Definition. Day Convolution day-convolution

Let 𝒱︀ be a symmetric monoidal closed category that is complete and cocomplete. Let (π’žοΈ€,βŠ—π’žοΈ€,𝐼) be a small monoidal 𝒱︀-enriched category. The Day convolution of 𝒱︀-enriched presheaves 𝐴,𝐡:π’žοΈ€op→𝒱︀ is the enriched presheaf

(π΄βŠ—Day𝐡)𝑐=βˆ«π‘’,π‘£π’žοΈ€(𝑐,π‘’βŠ—π’žοΈ€π‘£)βŠ—π’±οΈ€π΄π‘’βŠ—π’±οΈ€π΅π‘£,

with unit the representable π’žοΈ€(βˆ’,𝐼).

Day convolution makes the enriched presheaf category [π’žοΈ€op,𝒱︀] a monoidal category, symmetric when π’žοΈ€ is [1]. It is moreover closed.

Under the enriched Yoneda embedding the convolution of representables is representable, π’žοΈ€(βˆ’,π‘₯)βŠ—Dayπ’žοΈ€(βˆ’,𝑦)β‰…π’žοΈ€(βˆ’,π‘₯βŠ—π’žοΈ€π‘¦), so Day convolution is the cocontinuous extension of the tensor of π’žοΈ€.

As a Kan Extension

Equivalently, π΄βŠ—Day𝐡 is the left Kan extension of (𝑒,𝑣)β†¦π΄π‘’βŠ—π’±οΈ€π΅π‘£ along βŠ—π’žοΈ€op:π’žοΈ€opΓ—π’žοΈ€opβ†’π’žοΈ€op.

In π’πžπ­

When 𝒱︀=π’πžπ­, we recover the ordinary Day convolution of presheaves 𝐴,𝐡:π’žοΈ€opβ†’π’πžπ­, where the formula simplifies to:

(π΄βŠ—Day𝐡)𝑐=βˆ«π‘’,π‘£π’žοΈ€[𝑐,π‘’βŠ—π’žοΈ€π‘£]×𝐴𝑒×𝐡𝑣.

Definition. The Quotient and its Right Adjoint in Day Convolution day-quotient

Let 𝐢 be a small monoidal category. Recall that for presheaves 𝐴,𝐡:𝐢opβ†’π’πžπ­, the Day convolution provides a closed monoidal structure:

(π΄βŠ—π΅)(𝑐)=βˆ«π‘’,𝑣𝐢[𝑐,π‘’βŠ—π‘£]×𝐴(𝑒)×𝐡(𝑣)
(𝐴⊸𝐡)(𝑐)=βˆ«π‘’π΄(𝑒)→𝐡(π‘’βŠ—π‘)

which forms an adjunction π΄βŠ—βˆ’βŠ£π΄βŠΈβˆ’.

For covariant functors 𝐴:πΆβ†’π’πžπ­ and presheaves 𝐡:𝐢opβ†’π’πžπ­, we can define the quotient and its right adjoint, each of which is a presheaf on 𝐢:

(𝐷𝐴𝐡)(𝑐)=βˆ«π‘’π΄(𝑒)×𝐡(π‘’βŠ—π‘)
(𝐡𝐴)(𝑐)=βˆ«π‘’,𝑣𝐢[π‘’βŠ—π‘£,𝑐]β†’(𝐴(𝑒)→𝐡(𝑣))

These form the adjunction π·π΄βŠ£βˆ’π΄.

Definition. The Quotient and Residual Coincidence in Day Convolution day-quotient-coincidence

The Day quotient, 𝐷𝐴𝐡, takes 𝐴 to be a covariant functor and 𝐡 to be a contravariant one. While the residual 𝐴⊸𝐡 takes both to be contravariant.

This apparent variance mismatch disappears when we restrict to the groupoid core 𝐺=π–Όπ—ˆπ—‹π–Ύ(𝐢), where variance is trivialized since 𝐺≅𝐺op.

For any functor 𝐹:πΊβ†’π’πžπ­ on the core, we can freely extend it to both a presheaf and a covariant copresheaf on 𝐢 by left Kan extension along the respective inclusions 𝐺→𝐢op and 𝐺→𝐢:

π—†π—„π–―π—Œπ—(𝐹)(𝑐)=βˆ«π‘ βˆˆπΊπΆ[𝑐,𝑠]×𝐹(𝑠)
π—†π—„π–’π—ˆπ—‰π—Œπ—(𝐹)(𝑐)=βˆ«π‘ βˆˆπΊπΆ[𝑠,𝑐]×𝐹(𝑠)

By substituting these extensions into the definitions of the residual and the quotient, we obtain a general coincidence for any 𝐹:πΊβ†’π’πžπ­ and 𝐡:𝐢opβ†’π’πžπ­:

π—†π—„π–―π—Œπ—(𝐹)βŠΈπ΅β‰…π·π—†π—„π–’π—ˆπ—‰π—Œπ—(𝐹)𝐡

This equivalence states that computing the residual against the presheaf extension of 𝐹 is perfectly isomorphic to taking the quotient by its covariant extension.

The Representable Case

Instantiating the above theorem at a representable functor on the core, 𝐹=𝐺[π‘₯,βˆ’]: the co-Yoneda lemma says that the left Kan extensions compute to the representables on 𝐢:

π—†π—„π–―π—Œπ—(𝐺[π‘₯,βˆ’])≅𝐢[βˆ’,π‘₯]
π—†π—„π–’π—ˆπ—‰π—Œπ—(𝐺[π‘₯,βˆ’])≅𝐢[π‘₯,βˆ’]

Applying the general coincidence theorem, the left and right adjoints coincide precisely on (opposite-variance) representables:

(𝐢[βˆ’,π‘₯]⊸𝐡)(𝑐)≅𝐡(π‘₯βŠ—π‘)β‰…(𝐷𝐢[π‘₯,βˆ’]𝐡)(𝑐)

When 𝐢 is the discrete monoidal category of strings, this recovers the derivative of formal grammars.

In nominal sets, I suspect that this construction also describes name abstraction and the freshness quantifier, although I have not check all of the details of the proof.

Definition. Dependent Tensor of Grammars dependent-tensor

The tensor of grammars,

(π΄βŠ—π΅)(𝑀)=βˆ‘π‘€=𝑒𝑣𝐴(𝑒)×𝐡(𝑣)

is simply typed, so 𝐡 cannot see what 𝐴 matched.

Just as we have the generalization from the pair type 𝐴×𝐡 to the dependent pair Ξ£π‘Ž:π΄π΅π‘Ž, we can define a dependent tensor operation. It is a Ξ£-like generalization over the ordinary tensor, in which the second grammar depends on a parse tree of the first.

Write 𝖳𝗋𝖾𝖾(𝐴)=βˆ‘π‘€π΄(𝑀) for the type of parse trees of 𝐴, each paired with the string it parses.

For a grammar 𝐴 and a family of grammars 𝐡:𝖳𝗋𝖾𝖾(𝐴)→𝐆𝐫, the dependent tensor is:

(π΄βŠ—π–£π΅)(𝑀)=βˆ‘π‘€=π‘’π‘£βˆ‘π‘Ž:𝐴(𝑒)𝐡(𝑒,π‘Ž)(𝑣)

We may also write it as (π‘Žβˆˆπ΄)βŠ—π΅(π‘Ž).

When 𝐡 is a constant family, the dependent tensor reduces to the ordinary tensor π΄βŠ—π΅.

Dependence runs left to right here, which fits left-to-right parsing. The other handedness, in which the first grammar depends on a parse of the second, is also definable.

Relation to Day Convolution

Just as the ordinary tensor is given by Day convolution, I think there is a similar dependent Day convolution for which this operation is an instance. My best guess is for any 𝑋 presheaf on a monoidal category π’žοΈ€ and π‘Œ a functor from the category of elements of 𝑋 into presheaves on π’žοΈ€,

(π‘‹βŠ—π–£π‘Œ)𝑐=βˆ«π‘’,π‘£π’žοΈ€[𝑐,π‘’βŠ—π‘£]Γ—βˆ«π‘₯:π‘‹π‘’π‘Œ(𝑒,π‘₯)𝑣

I’m not positive on this though, and I would hope to also extend this to enrichments that aren’t π’πžπ­ but I don’t see how that could make sense given that I have quantified over the element π‘₯:𝑋𝑒 in the coend rather than using the tensor of the enriching category.

Displayed Categories as Dependent Types displayed-categories-as-dependent-types

Displayed category theory is the category-theoretic analogue of dependent type theory. A category plays the role of a context, and a displayed category over it the role of a dependent type in that context. The analogy extends to each construction:

Dependent type theoryDisplayed category theory
context Ξ“category π’žοΈ€
dependent type Ξ“βŠ’π΅displayed category over π’žοΈ€
dependent function (π‘Ž:𝐴)→𝐡(π‘Ž)section of
context extension Ξ“,π‘₯:𝐡total category and its projection
substitution 𝐡[𝑓]reindexing
Ξ£-typedisplayed total category
a type not depending on its contextweakening

Definition. Fiber Category fiber-category

Given a displayed category over π’žοΈ€ and an object 𝑐:π’žοΈ€0, the fiber of over 𝑐 is the category whose objects are the displayed objects , and whose morphisms are the vertical morphisms , those lying over the identity.

Identities are the displayed identities. The displayed composite of two vertical morphisms lies over 𝗂𝖽𝑐⋆𝗂𝖽𝑐 rather than 𝗂𝖽𝑐, so composition in the fiber transports it along 𝗂𝖽𝖫𝗂𝖽𝑐.

Definition. Formal Grammar formal-grammar

For an alphabet Ξ£, let Ξ£βˆ— be the free monoid of strings over Ξ£. A formal grammar 𝐴 is a family of types indexed by strings:

𝐴:Ξ£βˆ—β†’π–³π—’π—‰π–Ύ

This definition views formal grammars directly as indexed families of types over strings (which equivalently form presheaves on the discrete category of strings). This view forms the central notion of the Dependent Lambek Calculus. For a given string 𝑀, the type 𝐴(𝑀) represents the type of all valid parse trees for 𝑀 according to the grammar 𝐴. If the grammar cannot parse 𝑀, then 𝐴(𝑀) is the empty type.

Definition. The Derivative of a Grammar grammar-derivative

When a grammar quotient 𝐷𝐴𝐡 is taken with respect to a representable grammar 𝐴=βŒˆπ‘€βŒ‰ (matching exactly the string 𝑀), it coincides with the residual 𝐴⊸𝐡, as established by the Quotient and Residual Coincidence.

This special case is known as the Brzozowski derivative of 𝐡 by the string 𝑀, often written as πœ•π‘€π΅. We have the following coincidence:

(βŒˆπ‘€βŒ‰βŠΈπ΅)(𝑣)≅𝐡(𝑀·𝑣)β‰…(π·βŒˆπ‘€βŒ‰π΅)(𝑣)

Definition. Later on Grammars grammar-later

Write π‘’βŠπ‘£ when 𝑒 is a proper suffix of 𝑣, that is, 𝑣=π‘₯++𝑒 with π‘₯β‰ πœ€. The later of a grammar 𝐴 is

(▷𝐴)𝑣=βˆπ‘’βŠπ‘£π΄π‘’.

A parse of ▷𝐴 over 𝑣 is a parse of 𝐴 over every proper suffix of 𝑣. In particular (▷𝐴)πœ€ is a singleton.

This is later on families for strings under the proper-suffix order. That order is well-founded because it strictly decreases length, so it is a thin direct category. There is at most one map 𝑒→𝑣, so the product over maps from the strict past has one factor per proper suffix. Guarded recursion is modelled by presheaves on πœ”, the topos of trees, and more generally by sheaves over a well-founded base [1]. Here the later acts on families, which is what grammars are, and there is no clock.

Later is the right adjoint of the proper-suffix derivative. Let 𝖭𝖀𝖲=⨁π‘₯β‰ πœ€βŒˆπ‘₯βŒ‰ be the grammar of non-empty strings. Then the derivative (πœ•π–­π–€π–²π΅)π‘’β‰…βˆ‘π‘₯β‰ πœ€π΅(π‘₯++𝑒) has the right adjoint

πΆπ–­π–€π–²π‘£β‰…βˆπ‘₯++𝑒=𝑣𝖭𝖀𝖲π‘₯→𝐢𝑒≅(▷𝐢)𝑣,

so πœ•π–­π–€π–²βˆ’βŠ£β–·. Splitting 𝖭𝖀𝖲 into its summands gives the form of β–· that the calculus can define:

▷𝐴≅&π‘€β‰ πœ€π΄βŒˆπ‘€βŒ‰,π΄βŒˆπ‘€βŒ‰β‰…(βŒˆπ‘€βŒ‰βŠ—βŠ€)β‡’(βŒˆπ‘€βŒ‰βŠ—π΄).

The component at 𝑀 says: if the string begins with 𝑀, then the rest parses as 𝐴.

The restriction to non-empty 𝑀 is what makes this a later. At 𝑀=πœ€ the component is π΄βŒˆπœ€βŒ‰β‰…π΄. Including it would give a projection β–·π΄βŠ’π΄, and LΓΆb would then prove every grammar. For the same reason the later is a product over suffixes. A sum such as β¨π‘βŒˆπ‘βŒ‰βŠΈπ΄ is empty at πœ€, and at 𝐴=βŠ₯ it is βŠ₯. The identity step βŠ₯⊒βŠ₯ would then give LΓΆb a proof of ⊀⊒βŠ₯.

In the Agda this is β–· in Grammar/Later/Base.agda, which is defined as the indexed conjunction of √l-string. The mirror image β–·r, over proper prefixes, is defined in the same way. See also the bilateral later and the later along an arbitrary well-founded order.

Theorem. LΓΆb Induction for Grammars grammar-lob

For every grammar 𝐴 and every term πœ‘:β–·π΄βŠ’π΄, where β–· is the later on grammars, there is a unique global parse π—…ΓΆπ–»πœ‘:⊀⊒𝐴 with

π—…ΓΆπ–»πœ‘=πœ‘βˆ˜π—‡π–Ύπ—‘π—(π—…ΓΆπ–»πœ‘).

Here 𝗇𝖾𝗑𝗍𝑑:βŠ€βŠ’β–·π΄ restricts a global parse 𝑑 to every proper suffix.

The proof is recursion on the length of the string. At 𝑣, the parses already built at the proper suffixes of 𝑣 form an element of (▷𝐴)𝑣, and πœ‘ turns it into a parse at 𝑣. Uniqueness is LΓΆb for families over the proper-suffix order. In the Agda, lob in Grammar/Later/Base.agda is this recursion, done by well-founded induction on length.

To prove an entailment 𝐡⊒𝐢 this way, apply LΓΆb to 𝐴=𝐡⇒𝐢. The hypothesis β–·(𝐡⇒𝐢) is the induction hypothesis at every proper suffix. It becomes usable once a non-nullable grammar has been consumed. If πœ€&𝐡⊒βŠ₯, then

(π΅βŠ—βŠ€)&β–·π΄βŠ’π΅βŠ—π΄

because a parse of π΅βŠ—βŠ€ over 𝑣 splits 𝑣=π‘₯++𝑒 with π‘₯ non-empty, so π‘’βŠπ‘£. This is β–·-app-NE in Grammar/Later/Properties.agda. Induction on a Kleene star is the standard use.

Theorem. Next on Grammars Is Presheaf Structure grammar-next-presheaf

For presheaves, 𝗇𝖾𝗑𝗍:𝑃→▷𝑃 restricts along the strict past, as in later on presheaves. A grammar is only a family over strings, so a map 𝗇𝖾𝗑𝗍:π΄βŠ’β–·π΄ into the later on grammars is extra data. It sends a parse over 𝑣 to parses over every proper suffix of 𝑣.

Let ░𝐴=𝐴&▷𝐴, so that (░𝐴)π‘£β‰…βˆπ‘’βŠ‘π‘£π΄π‘’. This is the comonad π‘ˆβˆ˜Cofree for the suffix order, and presheaves are comonadic over families. The counit law forces the first component of a coalgebra π΄βŠ’β–‘π΄ to be the identity. Hence:

  • a presheaf on strings under the suffix order is the same as a grammar 𝐴 with a map 𝗇𝖾𝗑𝗍:π΄βŠ’β–·π΄ that satisfies coassociativity. Restricting to 𝑒′ and then to π‘’βŠπ‘’β€² must agree with restricting to 𝑒 directly;
  • for a proposition-valued grammar (a language), coassociativity is automatic, and 𝗇𝖾𝗑𝗍 exists exactly when the language is closed under suffixes. For example, ⊀ has one, but βŒˆπ‘ŽβŒ‰ does not, since (β–·βŒˆπ‘ŽβŒ‰)π‘Žβ‰…βŒˆπ‘ŽβŒ‰πœ€ is empty.

LΓΆb does not need 𝗇𝖾𝗑𝗍 on 𝐴. The fixed-point equation only restricts a global parse ⊀⊒𝐴, and a global parse can always be restricted. In the Agda, the grammar-level IsCoalgebra record, with only the field next, is in Grammar/Later/Coalgebra.agda on the guarded branch.

On presheaves the strict downset of 𝑐++𝑑 is represented by 𝑑, because every proper suffix of 𝑐++𝑑 is a suffix of 𝑑. So predecessors simplify later to (▷𝑃)(𝑐++𝑑)≅𝑃𝑑. On grammars there is no such simplification, since the factors at different suffixes are unrelated.

Definition. Grammar Quotients grammar-quotients

For formal grammars 𝐴 and 𝐡, the quotient of 𝐡 by 𝐴 is the grammar of what is left of 𝐡 once an 𝐴 has been read off the front:

(𝐷𝐴𝐡)(𝑣)=Σ𝑒𝐴(𝑒)×𝐡(𝑒·𝑣)

A parse of the quotient consists of an 𝐴-parses at prefix 𝑒 and a 𝐡-parse of the full string 𝑒·𝑣.

This operation has a right adjoint:

(𝐢𝐴)(𝑀)=Π𝑒,𝑣(𝑒·𝑣=𝑀)→𝐴(𝑒)→𝐢(𝑣)

Intuitively, 𝐢𝐴 parses 𝑀 if: for all splittings of 𝑀 into a prefix and suffix, if the prefix matches 𝐴 then the suffix must match 𝐢.

Together, these form the adjunction:

𝐷𝐴𝐡⊒𝐢⟺𝐡⊒𝐢𝐴

This is an instance of the Day quotient over the discrete monoidal category of strings.

Definition. Grammar Residuals grammar-residual

For formal grammars 𝐴 and 𝐡, the residual 𝐴⊸𝐡 (sometimes called the lollipop or linear implication) is the right adjoint to the grammar tensor π΄βŠ—π΅. It describes strings that, when prefixed by a string matching 𝐴, will match 𝐡.

Formally, it is defined as:

(𝐴⊸𝐡)(𝑣)=Π𝑒𝐴(𝑒)→𝐡(𝑒·𝑣)

Intuitively, a parse of 𝐴⊸𝐡 at a string 𝑣 is a function that takes any prefix string 𝑒 and a parse of 𝑒 in 𝐴, and produces a parse of the full concatenated string 𝑒·𝑣 in 𝐡.

Definition. Tensor of Grammars grammar-tensor

For formal grammars 𝐴 and 𝐡, their tensor π΄βŠ—π΅ represents the concatenation of the languages they describe. A parse for π΄βŠ—π΅ at a string 𝑀 consists of a splitting of 𝑀 into a prefix 𝑒 and suffix 𝑣, along with a parse of 𝑒 in 𝐴 and a parse of 𝑣 in 𝐡.

Formally, it is defined as:

(π΄βŠ—π΅)(𝑀)=βˆ‘π‘€=𝑒𝑣𝐴(𝑒)×𝐡(𝑣)

This operation is exactly the Day convolution of 𝐴 and 𝐡, where we view formal grammars as presheaves over the monoid of strings.

Definition. Hylomorphism hylomorphism

Fix an endofunctor 𝐹:π’žοΈ€β†’π’žοΈ€, a coalgebra 𝛾:𝑋→𝐹𝑋 and an algebra 𝛼:𝐹𝐡→𝐡. A hylomorphism (or coalgebra-to-algebra morphism) from 𝛾 to 𝛼 is a morphism β„Ž:𝑋→𝐡 satisfying

β„Ž=π›Ύβ‹†πΉβ„Žβ‹†π›Ό.

Definition. The hylomorphism profunctor hylomorphism-profunctor

Fix an endofunctor 𝐹:π’žοΈ€β†’π’žοΈ€. Hylomorphisms form a profunctor from coalgebras to algebras,

π–§π—’π—…π—ˆ:π–’π—ˆπ–Ίπ—…π—€(𝐹)op×𝖠𝗅𝗀(𝐹)β†’π’πžπ­,

where π–§π—’π—…π—ˆ((𝑋,𝛾),(𝐡,𝛼)) is the set of β„Ž:𝑋→𝐡 with β„Ž=π›Ύβ‹†πΉβ„Žβ‹†π›Ό.

The action is by composition. If 𝑔:(π‘Œ,𝛿)β†’(𝑋,𝛾) is a coalgebra morphism (𝑔⋆𝛾=𝛿⋆𝐹𝑔) and π‘˜:(𝐡,𝛼)β†’(𝐡′,𝛼′) is an algebra morphism (π›Όβ‹†π‘˜=πΉπ‘˜β‹†π›Όβ€²), then π‘”β‹†β„Žβ‹†π‘˜ is again a hylomorphism:

𝛿⋆𝐹(π‘”β‹†β„Žβ‹†π‘˜)⋆𝛼′=π›Ώβ‹†πΉπ‘”β‹†πΉβ„Žβ‹†π›Όβ‹†π‘˜=π‘”β‹†π›Ύβ‹†πΉβ„Žβ‹†π›Όβ‹†π‘˜=π‘”β‹†β„Žβ‹†π‘˜.

In these terms, a coalgebra 𝛾 is recursive when π–§π—’π—…π—ˆ(𝛾,βˆ’) is the terminal functor, and an algebra 𝛼 is corecursive when π–§π—’π—…π—ˆ(βˆ’,𝛼) is. For the inverse of an initial algebra the profunctor is representable, π–§π—’π—…π—ˆ(π—‚π—‡βˆ’1,βˆ’)≅𝖠𝗅𝗀(𝐹)(𝗂𝗇,βˆ’), and dually π–§π—’π—…π—ˆ(βˆ’,π—ˆπ—Žπ—βˆ’1)β‰…π–’π—ˆπ–Ίπ—…π—€(𝐹)(βˆ’,π—ˆπ—Žπ—) for a terminal coalgebra.

When every value of π–§π—’π—…π—ˆ is a singleton, every divide-and-conquer specification over 𝐹 has exactly one solution. This is what local contractivity guarantees.

Definition. Lax Functor lax-functor

A lax functor 𝐹:β„¬οΈ€β†’π’žοΈ€ between bicategories consists of

  1. a map on 0-cells, π‘₯↦𝐹π‘₯;
  2. for all π‘₯,𝑦, a functor 𝐹π‘₯,𝑦:ℬ︀(π‘₯,𝑦)β†’π’žοΈ€(𝐹π‘₯,𝐹𝑦), acting on 1-cells and 2-cells;
  3. a unit comparison, natural 2-cells 𝐹π‘₯0:1𝐹π‘₯⇒𝐹(1π‘₯);
  4. a composition comparison, 2-cells 𝐹𝑓,𝑔2:𝐹𝑓⋆𝐹𝑔⇒𝐹(𝑓⋆𝑔) natural in 𝑓 and 𝑔;
  5. such that three coherence laws hold, one for each structure cell of ℬ︀:

    • left unit: (𝐹0▷𝐹𝑓)⋆𝐹1,𝑓2⋆𝐹(πœ†π‘“)=πœ†πΉπ‘“;
    • right unit: (𝐹𝑓◁𝐹0)⋆𝐹𝑓,12⋆𝐹(πœŒπ‘“)=πœŒπΉπ‘“;
    • associativity: (𝐹𝑓,𝑔2β–·πΉβ„Ž)⋆𝐹𝑓⋆𝑔,β„Ž2⋆𝐹(𝛼𝑓,𝑔,β„Ž)=𝛼𝐹𝑓,𝐹𝑔,πΉβ„Žβ‹†(𝐹𝑓◁𝐹𝑔,β„Ž2)⋆𝐹𝑓,π‘”β‹†β„Ž2.

Here ⋆ between 2-cells is vertical composition, and πœƒβ–·β„Ž and β„Žβ—πœƒ are whiskerings.

Lax functors compose: (𝐺∘𝐹)0=𝐺0⋆𝐺(𝐹0) and (𝐺∘𝐹)𝑓,𝑔2=𝐺𝐹𝑓,𝐹𝑔2⋆𝐺(𝐹𝑓,𝑔2). The coherence laws of the composite follow from those of 𝐹 and 𝐺 and naturality of 𝐺2, without using the triangle or pentagon of any of the bicategories involved.

A lax functor whose comparison cells are invertible is a pseudofunctor.

Definition. Lax and Pseudonatural Transformations lax-natural-transformation

Let 𝐹,𝐺:β„¬οΈ€β†’π’žοΈ€ be lax functors. A lax natural transformation 𝜎:𝐹⇒𝐺 consists of

  1. for each 0-cell π‘₯, a 1-cell 𝜎π‘₯:𝐹π‘₯→𝐺π‘₯;
  2. for each 1-cell 𝑓:π‘₯→𝑦, a 2-cell filling the naturality square,

    πœŽπ‘“:πΉπ‘“β‹†πœŽπ‘¦β‡’πœŽπ‘₯⋆𝐺𝑓;
  3. such that πœŽπ‘“ is natural in 𝑓: for a 2-cell πœƒ:𝑓⇒𝑔, (πΉπœƒβ–·πœŽπ‘¦)β‹†πœŽπ‘”=πœŽπ‘“β‹†(𝜎π‘₯β—πΊπœƒ);
  4. and such that 𝜎 respects the comparison cells of 𝐹 and 𝐺: one law relating 𝜎1π‘₯ to 𝐹0, 𝐺0 and the unitors, and one relating πœŽπ‘“β‹†π‘” to πœŽπ‘“, πœŽπ‘”, 𝐹2, 𝐺2 and the associators.

A lax natural transformation is pseudonatural when every πœŽπ‘“ is invertible. As with pseudofunctors, this is a property, so the pseudonatural transformations 𝐹⇒𝐺 are a full subcategory of the category of lax transformations and modifications.

Between prestacks the 1-cells are taken to be pseudonatural. The reason is biuniversality: a transformation whose components 𝜎π‘₯ are all equivalences of categories is an equivalence of prestacks only if its naturality cells are invertible, and a biuniversal element should be exactly a representation of a prestack up to such an equivalence.

Definition. Locally Discrete Bicategory locally-discrete-bicategory

Every category π’žοΈ€ is a bicategory 𝖫𝖣(π’žοΈ€) with only identity 2-cells. Its 0-cells are the objects of π’žοΈ€, and the hom-category 𝖫𝖣(π’žοΈ€)(π‘₯,𝑦) is the discrete category on the set π’žοΈ€(π‘₯,𝑦): a 2-cell 𝑓⇒𝑔 is a proof that 𝑓=𝑔. Composition and identities are those of π’žοΈ€; the unitors and associator are the unit and associativity laws of π’žοΈ€. Since the homs of π’žοΈ€ are sets, any two parallel 2-cells are equal, so the triangle and pentagon hold trivially.

A functor 𝐹:π’žοΈ€β†’π’ŸοΈ€ gives a pseudofunctor 𝖫𝖣(𝐹):𝖫𝖣(π’žοΈ€)→𝖫𝖣(π’ŸοΈ€), whose comparison 2-cells are the functor laws of 𝐹.

The locally discrete bicategory is how ordinary indexed categories enter bicategorical language: a prestack on 𝖫𝖣(π’žοΈ€) is a pseudofunctor π’žοΈ€op→𝖒𝖠𝖳, and its Grothendieck construction is a displayed category over π’žοΈ€.

Definition. Monoidal Category monoidal-category

A monoidal category is a category π’žοΈ€ together with

  1. a functor βŠ—:π’žοΈ€Γ—π’žοΈ€β†’π’žοΈ€, the tensor product;
  2. an object 𝐼 of π’žοΈ€, the unit;
  3. natural isomorphisms

    𝛼π‘₯,𝑦,𝑧:(π‘₯βŠ—π‘¦)βŠ—π‘§β†’π‘₯βŠ—(π‘¦βŠ—π‘§)πœ†π‘₯:πΌβŠ—π‘₯β†’π‘₯𝜌π‘₯:π‘₯βŠ—πΌβ†’π‘₯

    the associator, left unitor and right unitor;

such that the triangle and the pentagon below commute for all objects 𝑀,π‘₯,𝑦,𝑧.

Definition. Nominal Sets nominal-set

Fix a countably infinite set of names 𝔸. A nominal set [1] is a set 𝑋 equipped with an action by the group of finite permutations 𝖯𝖾𝗋𝗆(𝔸), such that every element π‘₯βˆˆπ‘‹ has a finite support.

A finite set of names π‘†βŠ†π”Έ supports π‘₯ if any permutation fixing 𝑆 pointwise also fixes π‘₯. The intersection of all supports for π‘₯ is called the least support, denoted π—Œπ—Žπ—‰π—‰(π‘₯).

The category of nominal sets is equivalent to the Schanuel topos. Under this equivalence, a nominal set 𝑋 corresponds to a functor π•€β†’π’πžπ­, where 𝕀 is the category of finite sets and injections, given by mapping a finite set of names 𝑑 to the set of elements supported by 𝑑:

𝑋(𝑑)={π‘₯βˆˆπ‘‹|π—Œπ—Žπ—‰π—‰(π‘₯)βŠ†π‘‘}

Definition. Nominal Sets as Day Quotients nominal-sets-quotient

In the Schanuel topos, the underlying category for Day convolution is 𝐢=𝕀op, where 𝕀 is the category of finite sets and injections.

Given a nominal set 𝑋, its presheaf action describes elements supported by 𝑑:

𝑋(𝑑)={π‘₯βˆˆπ‘‹|π—Œπ—Žπ—‰π—‰(π‘₯)βŠ†π‘‘}

Instead of taking 𝑋 a priori as a presheaf, we can view it as a finitely- supported 𝖯𝖾𝗋𝗆(𝔸)-set. We can restrict our attention to the groupoid core 𝐺=π–Όπ—ˆπ—‹π–Ύ(𝕀), asking for the support to be exactly the input:

𝑋𝑠={π‘₯βˆˆπ‘‹|π—Œπ—Žπ—‰π—‰(π‘₯)=𝑠}

This family 𝑋‒:πΊβ†’π’πžπ­ is functorial on finite sets and bijections.

By extending this functor along the inclusions described in Quotient Coincidence, we can extend 𝑋‒ to both a presheaf π—†π—„π–―π—Œπ—(𝑋‒) and a copresheaf π—†π—„π–’π—ˆπ—‰π—Œπ—(𝑋‒) on 𝐢. This suggests the equivalence:

π—†π—„π–―π—Œπ—(𝑋‒)βŠΈπ‘Œβ‰…π·π—†π—„π–’π—ˆπ—‰π—Œπ—(𝑋‒)π‘Œ

I suspect this allows us to describe name abstraction [𝑋]π‘Œ [1] β€”which ordinarily looks like an operation on two presheaves of the same varianceβ€”as the quotient of π‘Œ by the induced copresheaf π—†π—„π–’π—ˆπ—‰π—Œπ—(𝑋‒).

Concretely, [𝑋]π‘Œ is usually given by the quotient of the product by an equivalence relation:

[𝑋]π‘Œβ‰…π‘‹Γ—π‘ŒβˆΌ

where (π‘₯,𝑦)∼(πœ‹π‘₯,πœ‹π‘¦) for permutations πœ‹ fixing π—Œπ—Žπ—‰π—‰(𝑦)βˆ’π—Œπ—Žπ—‰π—‰(π‘₯).

On the other hand, the Day quotient computes to a coend:

(π·π—†π—„π–’π—ˆπ—‰π—Œπ—(𝑋‒)π‘Œ)(𝑐)=βˆ«π‘ π‘‹π‘ Γ—π‘Œ(π‘ βŠŽπ‘)

A priori, a coend over 𝑠 of the product π‘‹π‘ Γ—π‘Œ(π‘ βŠŽπ‘) is expressed as the quotient of a set of triples by an equivalence relation β‰ˆ:

{(𝑠,π‘₯,𝑦)|𝑠=π—Œπ—Žπ—‰π—‰(π‘₯),π—Œπ—Žπ—‰π—‰(𝑦)βŠ†π‘ βŠŽπ‘}/β‰ˆ

However, because the comprehension formula fixes 𝑠=π—Œπ—Žπ—‰π—‰(π‘₯), we reduce to a quotient of pairs:

{(π‘₯,𝑦)|π—Œπ—Žπ—‰π—‰(𝑦)βŠ†π—Œπ—Žπ—‰π—‰(π‘₯)βŠŽπ‘}/β‰ˆ

I suspect that this quotient will equate to the one given by [𝑋]π‘Œ, thus resolving the apparent issues with variance. I further suspect that one will need the sheaf condition (pullback-preservation) of nominal sets to establish this equivalence.

Definition. Opposite Bicategory opposite-bicategory

The opposite ℬ︀op of a bicategory ℬ︀ has the same 0-cells and reverses the 1-cells but not the 2-cells:

ℬ︀op(π‘₯,𝑦)=ℬ︀(𝑦,π‘₯).

Composition swaps its arguments, 𝑓⋆op𝑔=𝑔⋆𝑓. The left unitor of ℬ︀op is the right unitor of ℬ︀ and vice versa, and the associator of ℬ︀op is the inverse of the associator of ℬ︀, with its arguments reversed.

Since the 2-cells keep their direction, a lax functor 𝐹:β„¬οΈ€β†’π’žοΈ€ induces a lax (not oplax) functor ℬ︀opβ†’π’žοΈ€op with the same action on cells. Reversing the 2-cells instead gives the bicategory ℬ︀co, whose hom-categories are the opposites (ℬ︀(π‘₯,𝑦))op.

Duality saves work: a coherence lemma about ℬ︀ can often be obtained by instantiating a companion lemma at ℬ︀op, which swaps left and right.

Definition. Pseudofunctor pseudofunctor

A pseudofunctor 𝐹:β„¬οΈ€β†’π’žοΈ€ is a lax functor whose unit and composition comparisons

𝐹π‘₯0:1𝐹π‘₯⇒𝐹(1π‘₯)𝐹𝑓,𝑔2:𝐹𝑓⋆𝐹𝑔⇒𝐹(𝑓⋆𝑔)

are invertible 2-cells. So 𝐹 preserves identities and composition up to coherent isomorphism.

Being pseudo is a property of a lax functor: invertibility of a 2-cell is a proposition, since inverses are unique. The data of a pseudofunctor is exactly the data of a lax functor, and everything proved about lax functors applies to pseudofunctors unchanged. Pseudofunctors are closed under composition and identities, as lax functors are, because invertible 2-cells are closed under composition and under the action of a functor on hom-categories.

The main examples here are prestacks, pseudofunctors ℬ︀op→𝖒𝖠𝖳. When ℬ︀ is locally discrete on a category π’žοΈ€, these are the pseudofunctors π’žοΈ€op→𝖒𝖠𝖳 of fibred category theory.

Definition. Recursive coalgebra recursive-coalgebra

Fix an endofunctor 𝐹:π’žοΈ€β†’π’žοΈ€. A coalgebra 𝛾:𝑋→𝐹𝑋 is recursive when for every algebra 𝛼:𝐹𝐡→𝐡 there is exactly one hylomorphism β„Ž:𝑋→𝐡 from 𝛾 to 𝛼, that is, exactly one solution of

β„Ž=π›Ύβ‹†πΉβ„Žβ‹†π›Ό.

Equivalently, the functor π–§π—’π—…π—ˆ(𝛾,βˆ’) of the hylomorphism profunctor is constantly a singleton.

Recursiveness is a coalgebraic form of well-foundedness: 𝛾 decomposes each input into subproblems, and recursiveness says that every divide-and-conquer program built on this decomposition has a unique meaning, without mentioning an order on inputs. [1] use recursive coalgebras on categories of indexed families to obtain algorithms that are correct by the type of the map they compute.

Example. If (πœ‡πΉ,𝗂𝗇) is an initial algebra, then 𝗂𝗇 is invertible (Lambek’s lemma) and π—‚π—‡βˆ’1:πœ‡πΉβ†’πΉ(πœ‡πΉ) is a recursive coalgebra. Precomposing with the isomorphism 𝗂𝗇, the equation β„Ž=π—‚π—‡βˆ’1β‹†πΉβ„Žβ‹†π›Ό is equivalent to π—‚π—‡β‹†β„Ž=πΉβ„Žβ‹†π›Ό, which says β„Ž is an algebra map out of the initial algebra; there is exactly one, π–Ώπ—ˆπ—…π–½π›Ό.

The dual notion is a corecursive algebra.

Definition. The Schanuel Topos schanuel-topos

Let 𝕀 be the category of finite sets and injections. The Schanuel topos is the category of pullback-preserving functors 𝕀opβ†’π’πžπ­.

Equivalently, it is the category of nominal sets, which are sets equipped with an action by the group of permutations on a countable set of names 𝔸, such that every element has finite support.

Definition. Section of a Displayed Category section-of-displayed-category

A section 𝑠 of a displayed category over π’žοΈ€ chooses

  1. for each object 𝑐:π’žοΈ€0, a displayed object , and
  2. for each morphism 𝑓:𝑐→𝑑, a displayed morphism lying over 𝑓,

such that 𝑠(𝗂𝖽𝑐)=𝗂𝖽𝑠(𝑐) and 𝑠(𝑓⋆𝑔)=𝑠(𝑓)⋆𝑠(𝑔).

A section is to a displayed category what a dependent function (π‘Ž:𝐴)→𝐡(π‘Ž) is to a dependent type: it picks a displayed datum over every base datum.

Definition. Types over an Algebraic Theory theory-type

Fix a finitary algebraic theory 𝒯︀: sorts 𝑆, a signature 𝜎, equations, and a set 𝑉 of generators. Write 𝑴 for the free 𝒯︀-model on 𝑉 and |𝑴|𝑠 for its carrier at sort 𝑠.

A 𝒯︀-type of sort 𝑠 is a family 𝐴:|𝑴|𝑠→𝖳𝗒𝗉𝖾.

For 𝒯︀-types 𝐴 and 𝐡 of sort 𝑠 we have the additives, defined pointwise,

(𝐴&𝐡)(π‘š)=𝐴(π‘š)×𝐡(π‘š),(π΄βŠ•π΅)(π‘š)=𝐴(π‘š)+𝐡(π‘š),(𝐴⇒𝐡)(π‘š)=𝐴(π‘š)→𝐡(π‘š),

along with ⊀, βŠ₯, and their indexed versions.

Each operation π‘œ:𝑠0,…,π‘ π‘›βˆ’1→𝑠 of 𝒯︀ gives a multiplicative, defined by Day convolution,

βŠ—[π‘œ](𝐴0,…,π΄π‘›βˆ’1)(π‘š)=Ξ£π‘œ(π‘š0,…,π‘šπ‘›βˆ’1)=π‘šΓ—Ξ π‘–<𝑛𝐴𝑖(π‘šπ‘–).

Each lifted operation also has closed structure: a residual in each of its arguments. Day algebras [1] give a more general convolutional definition.

When 𝒯︀ is the theory of monoids and 𝑉 is an alphabet, 𝒯︀-types are formal grammars. Multiplication gives βŠ—, the unit gives πœ€, and we recover Lambek𝙳.

Definition. Total Bicategory total-bicategory

The total bicategory of a displayed bicategory over 𝒦︀ packages the base and the displayed data together, one dimension up from the total category of a displayed category.

  • Its 0-cells are pairs of a 0-cell of 𝒦︀ and a displayed 0-cell over it.
  • Its hom-category from to is the total category of the displayed hom-category . So a 1-cell is a pair and a 2-cell is a pair .
  • Identities, composition, unitors and associator are pairs of the base structure and the displayed structure over it, and the triangle and pentagon hold because they hold in the base and, over that, in the displayed bicategory.

Projecting to first components is a pseudofunctor whose unit and composition comparisons are identity 2-cells.

all-notes note entries/home/all-notes.hel