Deixis and the Prague Correspondence: Stepping toward General Visual Programming by Demonstration with Direct Manipulation

Gavriel Loria

Presented at the 12th Workshop on Live Programming (LIVE 2026). https://doi.org/10.5281/zenodo.24409597

The design of general-purpose visual programming languages where one programs by demonstration is an ongoing challenge, particularly when limiting escape to text-based editing. The author reframes this challenge by coining the Prague Correspondence Problem: Can we draw a rigorous relationship between instructions that execute in a virtual machine, direct manipulations or gestures that communicate those instructions to a virtual machine, and visual representations of these instructions executing that the machine may communicate back?

Deixis is an in-progress prototype of a live visual coding environment. A user can directly demonstrate the execution of a graph-rewrite algorithm the way they want it to run. The nodes are symbols and include first-class operators, making Deixis visually homoiconic. The edges are spatial relations between nodes determined by matching on and transforming node arrangements. Its visual virtual instruction set architecture includes a base set of graph-rewrite instructions, such as creation, moving, and evaluation. Each is associated with an interaction and a visual that is concurrent with execution. A user can demonstrate programming factorial.

It remains unclear what models exist for conceptually and practically unifying computing with associated visuals and direct manipulation—where direct manipulation is execution, and where its corresponding visual representation is execution as well. Deixis suggests a path toward another model by using ideas from linguistics, such as imbuing spatial arrangements with syntactic structure. Deixis aims to advance the Prague Correspondence Problem by making visual structures interactive and executable.

https://www.dorianzgraggen.com/external/live2026

The headline

I discovered a different possible path toward unifying interactions, execution, and visuals in a programming-by-demonstration prototype, where you can match and manipulate spatial arrangements to record graph-rewriting algorithms.

Building on the past

Keeping a critical eye

At a low level, Deixis has the following limitations:

At a medium level, Deixis has the following limitations:

At a high level, Deixis has the following limitations: