Mathematical picture language program.

We give an overview of our philosophy of pictures in mathematics. We emphasize a bidirectional process between picture language and mathematical concepts: abstraction and simulation. This motivates a program to understand different subjects, using virtual and real mathematical concepts simulated by pictures.

Pictures appear throughout mathematical history, and we recount some of this story. We explain insights we gained through using mathematical pictures. Our picture language program has the goal to unify ideas from different subjects. We focus here on the 3D quon language (1). This language is a topological quantum field theory (TQFT) in 3D space, with lower-dimensional defects. A quon is a 2D defect on the boundary of a 3D manifold.

We believe quon and other languages will provide a framework for increased mathematical understanding, and we expect that looking further into the role of the mathematics of pictures will be productive. Hence, in , we pose a number of problems as the basis for a picture language research program.

Pictures have been central for visualizing insights and for motivating proofs in many mathematical areas, especially in geometry, topology, algebra, and combinatorics. They extend from ancient work in the schools of Euclid and Pythagoras to modern ideas in particle physics, category theory, and TQFT. See the interesting recent account in Silver (2). Nevertheless, we mention two aspects of mathematical pictures that we feel merit special study.

First is the importance we ascribe to mathematical analysis of pictures. We explain how we have begun to formulate a theory of mathematical analysis on pictures, in addition to the study of their topology and geometry. For example, the analytic aspect of pictures in TQFT is a less-developed area than its topological and algebraic aspects. Yet it has great potential for future advances.

Second is the notion of proof through pictures. In focusing on general mathematical properties of pictures, we wish to distinguish this quality from using pictures in a particular concrete mathematical theory. In other words, we aim to distinguish the notion of the properties of a picture language L, on the one hand, from its use through a simulation S to model a particular mathematical reality R. We thank one referee for pointing out that the distinction between L and R parallels the distinction in linguistics between syntax and semantics.

We propose that it is interesting to prove a result about the language L, and thereby, through simulation, ensure results in R. One can use a single picture language L to simulate several different mathematical areas. In fact, a theorem in L can ensure different theorems in different mathematical subjects R1, R2, etc., as a consequence of different simulations S1, S2, etc. This leads to the discussion of the simulation clock in Different configurations of the hands of the clock reveal the interrelation between picture proofs for seemingly unrelated mathematical results.

We also discuss the important distinction between two types of concepts in R that we simulate by a given S. These may be ‘‘real’’ concepts, or they may be ‘‘virtual.’’ This distinction is not absolute but depends on what language and simulation one considers. We give some examples, both in mathematics and in physics, in .

We hope these remarks about pictures can enable progress in understanding both mathematics and physics. Perhaps this can even help other subjects, such as the neurosciences, where “One picture is worth one thousand symbols.”

1. Some Major Themes

Euclidean Geometry.

Mathematicians have used pictures since the evolution of Euclidean geometry in ancient Greece. They proved abstract theorems based on axioms designed from pictorial intuition. A powerful feature of pictures is that one can easily visualize symmetries. Rotation and reflection symmetries appear in ancient arguments.

A good example comes from the problem of four points A,B,C,D on a plane, as illustrated inThe points lie on a circle if and only if the angles BAC and BDC are equal. This can be established pictorially or algebraically, and the picture proof∗ is elementary.

Abstraction and Simulation.

We propose two basic components to understanding that we call L and R. Here L denotes abstract concepts or language, while R stands for the concrete subjects or reality, which we desire to understand. We could also think of them as left and right. Simulation S represents a map from L to R.

Our universe provides a great reservoir for ideas about the real world. We can consider this as R. We understand these ideas through abstraction, including theories of mathematics, physics, chemistry, and biology. To deal with these real concepts, one often requires virtual concepts that have no meaning in the real world. These virtual concepts may not have an immediate real meaning, yet they may provide key insight to understanding the real structures.

Abstraction is a method to study complicated ideas and goes from R to L. For example, chemistry as L may provide a logical language to abstract certain laws in biology as R, with a simulation S: L R. One can continue this chain where we regard chemistry as a new R and physics as its abstraction as a new L with a new simulation S. Abstraction can be repeated yet again with mathematics as L and physics as R. At each step, one learns the axioms in L from the real concepts in R. In order to do computation, we often need to introduce virtual concepts in L to understand the concepts in R.

We are especially interested in the case that L is a picture language. Then one can represent the concepts in L by pictures that play the role of logical words, with axioms as their grammar. The axioms should be compatible with pictorial intuition. One can develop a picture language as an independent theory like Euclidean geometry.

A good simulation for a particular mathematical subject R should satisfy two conditions. Mathematical concepts in R should be simulated by simple pictures in L, and mathematical identities in R should arise from performing elementary operations on the pictures in L. Abstraction and simulation could be considered as inverse processes of each other.

Real and Virtual.

The idea of whether a concept is real or virtual plays an important role. This is not an absolute concept but depends on the simulation. For example, over the real-number field, the square root of a positive number is real, while the square root of a negative number is virtual. By enlarging the field to complex numbers, one understands this virtual concept in a fruitful way. One gets used to the power of complex numbers, and, in a new simulation, one can regard them as “real.”

In a picture language, it may be that a picture can represent a real concept in one theory and a virtual one in another. As an example, consider Feynman diagrams, the pictures that one uses to describe particle interactions. A contribution to the scattering of two physical electrons is described by the first electron emitting a photon (quantum) that is absorbed by the second electron. However, conservation of energy and momentum preclude the exchanged photon from being real: It must be virtual. Fig. 1 shows the original diagram in ref. 3.

Fig. 1.

The first Feynman diagram. Reprinted with permission from ref. 3, copyright (2013) American Physical Society.

Analytic continuation (Wick rotation) to imaginary time puts the real and virtual particles on equal footing. In that situation, the above example only involves virtual particles. In this way, one will obtain many virtual concepts, which could become real based on a proper simulation.

Modern Picture Mathematics.

Pictures played a central role both in the invention of topology, as well as in its modern understanding. A synergetic algebraic formalization was developed in parallel in category theory, which originated in the work of Eilenberg and MacLane in the 1940s (4).

TQFT appeared as a way to understand various topics using cobordisms. Remarkable examples of this point of view can be found in the work of Jones (5), Witten (6), Atiyah (7), Reshetikhin and Turaev, (8) Turaev and Viro (9), and Ocneanu (10). Various generalizations inspired by TQFT have been studied, and many results in this direction have emerged over the past 30 years.

Compared with topology, it is less obvious that pictures can also shed light on the study of algebra—in particular on representation theory. It turns out that this is the case. The insights from pictures are useful technically (such as with Young diagrams, braids, or quivers). These insights are also useful conceptually [e.g., to show how the topological properties of multiparticle systems are captured by properties of the centralizer algebra of representations, in the sense of Schur–Weyl duality (11, 12)].

Last, but not least, pictures also have deep connections with analysis on infinite-dimensional Hilbert space. This led to the discovery of the Jones polynomial (13). Conformal field theory (CFT) is a topic R closely related to this and from which we can learn laws for a picture language L.

2. Beyond Topology

When Atiyah gave a mathematical definition of TQFT, he wrote “it may well be that such topological understanding is a necessary prerequisite to building the analytical apparatus of the quantum theory” (7). Today, we want to move forward to reach a full quantum field theory. Our long-term goal is to construct quantum field theory using pictures. However, that may be too hard to achieve as the initial step.

We emphasize two properties of QFT that shed light on pictures. The first theme is “symmetry.” To understand continuum symmetry, it is useful to understand discrete symmetry that can approximate the continuum.

The second theme, “positivity,” is especially interesting, as positivity provides the basis for analysis. The striking fact is that the analysis of pictures is a mathematical theory with great potential, but is still in its infancy. On one hand, we expect to abstract concepts from analysis to pictures. This will enrich the theory of picture language in one additional dimension. On the other hand, we want to simulate analysis using picture language and provide pictorial tools.

Symmetry.

An elementary characteristic that transcends topology is ‘‘shape.’’ This goes in the direction of encapsulating geometry of the picture. The geometry can indicate the presence of additional symmetry represented by the pictures. With the shape of a picture as an additional tool, one can ask where it leads.

In lattice models of statistical physics, people often study square lattices and honeycomb lattices, which capture additional symmetry in two and three directions, respectively. The pictorial duality of lattices could provide interesting dualities of mathematical theories, such as the analog of Kramers–Wannier duality (14), illustrated in

If a vector space is simulated by square-like pictures on a two-dimensional plane, then gluing two pictures vertically or gluing them horizontally, defines two multiplications of . The 90° rotation, called the string Fourier transform (SFT) and denoted by , intertwines the two multiplications as illustrated below. See refs. 1, 15, and 16 for details and further references.

These 2D pictorial operations coincide with Fourier transform, multiplication, and convolution in Fourier analysis. This is a cornerstone for understanding pictorial Fourier duality.

Analysis.

How do we connect pictures with analysis? Usually, pictures without boundaries are scalars. How can we formulate a measurement pictorially? A main lesson from quantum field theory is the importance of reflection positivity. An elementary pictorial interpretation of reflection positivity is that gluing a picture to its mirror image is positive.

In constructive quantum field theory, extensive analysis is performed through estimating Feynman diagram pictures in terms of subdiagrams; this may use a pictorial Schwarz inequality, or some more sophisticated operator norm. These estimates are central to the proof of most results in the subject. In the framework of our present discussion, such analysis takes place on pictures in R. We call this using ‘‘pictures in analysis.’’

However, we are really interested in whether, and to what extent, one can do analysis on pictures. This means that we need to do computations in L, without reference to R. The discussion of rotation, Fourier analysis, multiplication, and convolution indicate that some progress can be made.

Do we obtain interesting analysis based on this minimal requirement? In fact, the surprise is that we already have interesting results inspired by Fourier theory. As mentioned, the 2D pictorial operation on square-like pictures is compatible with Fourier analysis.

Compactifying the plane to a sphere defines a measurement based on reflection positivity. Pictorial consistency on the sphere implies that the Fourier transform is a unitary. Many other results in Fourier analysis carry over to pictures.

3. A Pictorial Journey

Freeman Dyson described two types of mathematicians: birds and frogs (17). They do mathematics in different ways. Birds soar between different fields with unifying ideas, like Yuri Manin in his book Mathematics as Metaphor (18). Frogs sort out details to achieve great depth of understanding. In fact, it is good to attempt to encompass both metaphors! One does find both of these ingredients in our story of picture language. While we began in quantum information, we ended up traveling through much of the colorful landscape of mathematics.

In our context of language, the bird flies back and forth to discover new Rs, Ls, and Ss. The frog uses an S to understand some important problem. The shape of pictures provides key hints and insights for finding these connections. A popular article by Peter Reuell described picture language as Lego-like mathematics (19).

Harvard.

The authors began our collaboration 2 years ago in late July 2015 with much discussion at Harvard. Our first goal was to understand reflection positivity for parafermions (20, 21) in a pictorial way (15). If we call this mathematical problem R1, then it led us to define the picture language L1, which we call planar para algebra. This is a generalization of planar algebra, but with strings replaced by charged strings, and topological isotopy replaced by para isotopy. The map from L1 to R1 we call the simulation S1.

In the simulation S1, we found an elementary explanation that a 90° rotation of pictures represented Fourier transformation, and takes multiplication to convolution. We used this fact to give a geometric, picture proof of reflection positivity (15).We also found elementary picture representations for unitary Pauli matrices with eigenvalues , where , and .

Alex Wozniakowski pointed out that our work seemed related to quantum information. This led to fruitful collaboration among the three of us, in which we used the language L1 to simulate quantum communication in the mathematical framework that we name R2. Using this simulation S2, one picture deformed by isotopy into different shapes simulates different concepts in quantum information. In this way, we reproduced the standard teleportation protocol of Bennett et al. (22) by a topological design in L1 (23). With these concepts in place, we could also design other protocols, including multipartite, teleportation protocols (24).

The important point for our understanding of R2 was to follow pictorial intuition. This led to finding a natural candidate for a resource state that we called . Our picture Max in L1 for the entangled state is simple and natural, as well as suggesting entanglement in a pictorial fashion. Our two-string picture for (with qudits) is

The algebraic formula for the simulation S2Maxn in R2 iswhere we call the total charge in . It involves terms for a resource state with qudits, so, algebraically, the simulation of Maxn is complicated (23).

We then left Massachusetts to spend 4 months visiting the Research Institute for Mathematics (FIM) at the Eidgenössische Technische Hochschule (ETH) in Zurich, Switzerland.

ETH Zurich.

We finally had time to begin writing up these results (15) and later results (23, 24). We then learned that Greenberger et al. (25) had long before introduced another multiqubit resource state in quantum information. This state appears to be algebraically simpler, as it is a sum of only terms, independent of the number of qubits,Both and generalize the Bell states, and they have some very similar properties. This led us eventually to observe that and are related by a change of basis—in fact by Fourier transformation on ,To understand this further, we were intrigued by our generalization of “Kitaev’s map” from Majoranas to the Pauli spin matrices . We had discovered a natural generalization to represent a single qudit as a neutral parafermion/antiparafermion pair (15). Neutrality provides an elegant way to reduce the -dimensional space of states for two virtual parafermions to the correct -dimensional space for one real qudit. It involved introducing a “four-string” planar language L2 to describe a single qudit, and a corresponding simulation S3L2 = R3. The model R3 contains real and virtual one-qudit states. The representations of the qudit Pauli matrices are neutral, so they act on the neutral (real) subspace of dimensions. Another hint that neutrality holds the key is that the SFT equals the discrete Fourier transform in the neutral subspace of L2.

However, the language L2 and simulation S3 pose a problem for describing more than one qudit. Braiding charged strings from different qudits destroys neutrality of the individual excitations. This would allow transitions from the real -qudit space of dimension into the virtual -qudit space of dimension . Thus the obstruction to describing multiqudit states boiled down to the question: How can one ensure that real multiqudit states evolve into real multiqudit states? When we met Daniel Loss in Basel, we found that he was also considering this question. We did not find the answer immediately.

Bonn.

After Zurich, we had the opportunity to spend 6 weeks visiting two institutes in Bonn. We discovered the answer to the puzzle described in during June 2016, perhaps receiving inspiration from working in Fritz Hirzebruch’s former office at the Max Planck Institute for Mathematics.

During that time, we encountered two more hints about L2. The first is that the Frobenius algebra for the -interval Jones–Wassermann subfactor in CFT (26) isHere , a tensor of simple objects in a modular tensor category (MTC) (8), and is the multiplicity of 1 in . This formula coincides with for the group , but they have completely different meanings. Secondly, we learned the relations for bi-Frobenius algebras in the manuscript for ref. 27, which one of the authors had shared with Alex Wozniakowski.

The construction of Jones–Wassermann subfactors for an MTC requires an extension of pictures from 2D to 3D space. Here the coincidence between and is explained through an duality (28). Going from 2D to 3D also gives a natural explanation of the bi-Frobenius algebras relation.

The last piece in solving the puzzle came by adding three manifolds to these 3D pictures; this was inspired by TQFT. Finally, we unified all these ideas by formulating the 3D quon language L3, and a simulation S4 to quantum information. Moreover, we extended our approach from symmetry to MTCs based on work about the Jones–Wassermann subfactor.

We designed L3 using ideas from different Rs: quantum information, subfactor theory, TQFT, and CFT. Therefore, we expected to simulate those Rs using L3 as well as others.

In the quon language, the bi-Frobenius algebra relation has a topological interpretation. Both and are represented by single pictures Max and GHZ, where one picture is a 90° rotation of the other. Algebraically, one resource state is the Fourier transform of the other. Moreover, the complicated resource state can be computed by using its relation to the elementary resource state . For three qudits, the quon state pictures are just rotations of one another, as illustrated in

Furthermore, the quon language expresses elegantly in pictures the duality between orthonormal bases for the Pauli matrices and , so one can begin to imagine pictures describing quantum coordinates.

Back at Harvard.

Our first discovery by S4 was the topological nature of the Feynman, or quantum controlled NOT (CNOT), gate. Fig. 2, drawn by Lusa Zheglova, captures our representation of the quantum teleportation protocol. It illustrates classical communication in the foreground. In the background, it shows the quon representation of a Bell state, a CNOT gate, the Fourier transform, a measurement, and Pauli matrices (corresponding to Kitaev’s map) used in the recovery map. Together, they give the widely used quantum teleportation protocol of Bennett et al. (22), and illustrate its elegant 3D topological interpretation. The details can be found in ref. 1.

Fig. 2.

A 3D representation of quantum teleportation. Image courtesy of Lusa Zheglova.

This quon language L3 provided an opportunity to collect ideas and to think about their meaning. It became apparent that the quon language L3 was important on its own, as it could have applications in other areas of mathematics and physics besides quantum information. Furthermore, the algebraic identity between and given by the rotation of the diagrams could provide insight in other subjects through use of other simulations.

What is the meaning of this interesting algebraic identity? It actually is the Verlinde formula. Mathematically, one can generalize the pictorial construction to define GHZ and Max on higher-genus surfaces. Then the pictorial Fourier duality between GHZ and Max leads to the generalized Verlinde formula (29) for any MTC on the genus surface,Here, on the first line, is the number of quons, is the SFT, and, on the second line, are punctures (or marked points) on the genus surface, is the modular transformation, and dim is the dimension of the associated (moduli) space.

This pictorial Fourier duality also coincides with the duality of graphs on the sphere. The dual graph of the tetrahedron is also a tetrahedron. Applying the quon language to this graphic duality, we obtain a general algebraic identity Eq. for symbol self-duality. With denoting the dual object to in an MTC,In the special case of quantum , this was discovered by Barrett (30), based on an interesting identity of Roberts (31). The general formulation and proof of Eq. is in section 6 of ref. 32.

For each graph on a surface, the graphic duality gives a new algebraic identity for MTCs in quon language. Most of these identities have virtual meanings. Each graph can also be considered as a linear functional generalizing integration. The pictures generalize the symbol and capture additional pictorial relations, such as graph duality mentioned above. It would be interesting to understand these new identities and integrations in some new R.

What Next?

The progression

R1 L1 R2 L2 R3 L3 R4leads one to believe that much more is in store for the future. Each progression was inspired by insight from the previous step. We expect that this sequence will continue. We provide some possible Rs in a simulation clock!

4. Questions

We learned, from our picture journey, that focusing on the picture language itself is very fruitful. Here we collect a few questions for the future. The quon language provides an example, but we leave open the possibility of having many useful languages.

How far can one understand mathematical duality in terms of pictorial duality? For example, to what extent can one understand further properties of Fourier duality or mirror symmetry?

Many modern picture languages concern discrete combinatorial or topological data. A big question is: how does one construct a continuum theory from those pictures? Then one can ask how one can understand continuous symmetries, such as rotation invariance, in terms of picture language.

Many people have studied pictures from the point of view of topology and algebra. How far can one go to understand a different aspect: the analysis of pictures?

Can one construct a CFT from a unitary MTC?

Which family of mathematical concepts in R is pictorial?

Given symmetries and related identities in R which one would like to understand, can one find pictures in L that reflect these symmetries?

Can one identify identities in R in terms of elementary operations on pictures in L, such as by topological isotopy?

Can we do computations for R in L without using R? In this case, we say that the language L is complete. Complete picture relations may require virtual concepts that do not make sense in R.

Ideally, the real concepts in R can be represented by simple pictures in L. However, in case there are virtual pictures in L, can one find another R and S such that the virtual pictures become real? This situation leads to relations between different subjects.

How does one define a new picture language? The main constraint is consistency. One needs to introduce axioms compatible with pictorial intuition. Sometimes, one requires additional axioms that are motivated by requirements learned from R.

How does one construct examples of a picture language, possibly using one of the following methods? (i) Find L from R. (ii) Derive new examples from known ones. (iii) Construct examples in an abstract way.

How does one study pictures using axioms, as in Euclidean geometry? One can thus ask questions about the language, based on its own interest.

After defining a pictorial language L in an abstract way, how can one use the language to simulate interesting mathematics? For example, is there a CFT associated with the Haagerup subfactor (33)?

We have seen that the CNOT gate in quantum information (illustrated here in the quon language) has a pictorial interpretation. Another important gate in quantum information is the Toffoli gate. Is the Toffoli gate topological?

In the quon language, one illustrates a qudit by a 3D picture where the Z and X coordinate directions play a special role. Is there a representation of a qudit by a (3 + 1)D picture in which the three Frobenius algebras associated with X, Y, Z are represented by pictures in three orthogonal directions, and such that associativity becomes 3D topological isotopy?

Biamonte has posed other quantum information questions in ref. 34, including a pictorial understanding of the Gottesman–Knill theorem.

Is there a picture language with pictorial representations of differentiation, ordinary differential equations, and partial differential equations?

Feynman diagrams give pictures for Hermite polynomials for a given Gaussian. Is there a similar understanding of all Gaussians, Fourier transforms, and their Hermite polynomials?

Does a pictorial lattice model have a continuum limit as an interesting quantum field theory? If reflection positivity is involved, can one preserve positivity at each step in the approximation and the construction of a limit?

Kramers–Wannier duality allows one to compute the critical temperature for the 2D Ising model. Is there a duality that allows one to compute the critical temperature of a pictorial statistical model or its field theory limit? Is the limit a CFT at the critical temperature?