Quantum Relative Entropy of Tagging and Thermodynamics.

Thermodynamics establishes a relation between the work that can be obtained in a transformation of a physical system and its relative entropy with respect to the equilibrium state. It also describes how the bits of an informational reservoir can be traded for work using Heat Engines. Therefore, an indirect relation between the relative entropy and the informational bits is implied. From a different perspective, we define procedures to store information about the state of a physical system into a sequence of tagging qubits. Our labeling operations provide reversible ways of trading the relative entropy gained from the observation of a physical system for adequately initialized qubits, which are used to hold that information. After taking into account all the qubits involved, we reproduce the relations mentioned above between relative entropies of physical systems and the bits of information reservoirs. Some of them hold only under a restricted class of coding bases. The reason for it is that quantum states do not necessarily commute. However, we prove that it is always possible to find a basis (equivalent to the total angular momentum one) for which Thermodynamics and our labeling system yield the same relation.

1. Introduction

The role of information in Thermodynamics [1,2] already has a long history. Arguably, it is best exhibited in Information Heat Engines [3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22]. They are devices that cyclically extract energy from a thermal reservoir and deliver it as mechanical work. They do so by increasing the entropy of a set of bits from an information reservoir [1,23,24,25,26,27,28,29,30]. There are differences between classical bits and quantum qubits [31,32,33,34,35,36], but they share the same maximum efficiency [37,38]. Appendix A describes a basic model of an Information Heat Engine.

Physical systems in a state out of thermal equilibrium also allow the production of work. It turns out to be related to the relative entropy with respect to the equilibrium state , again an informational quantity (in this paper, always represents the binary logarithm of x). Appendix B contains a short derivation of this result. Some recent reviews compile a variety of properties and functional descriptions of relative entropy [39,40,41,42]. Probably, the most closely connected to this paper is its interpretation as the average extra number of bits that are employed when a code optimized for a given probabilistic distribution of words is used for some other. This paper contributes a new procedure that also reveals a direct connection between the relative entropy of physical systems and information reservoirs circumventing Thermodynamics. It focuses on the quantum case, particularly when the relative entropy is defined for non-commuting density matrices.

The generation of work in Information Heat Engines always requires the transfer of thermal energy from a heat reservoir and needs adequate steering of a Hamiltonian. In Szilard Engines [14,43,44], they occur at the same time as the piston moves within the cylinder. In the one particle case, every bit from an information reservoir enables the generation of mechanical work. Other thermal machines, such as turbines, also imply tuning Hamiltonians and heat transfer. Combining these devices, it is possible to produce work by increasing the entropy of informational qubits and use it to build up the relative entropy of a physical system with respect to its thermal equilibrium state. The net heat and work would vanish (see Figure 1). This consideration naturally motivates the question of whether it would be possible to do the same transformation only with informational manipulations, without any reference to Hamiltonians, temperatures or heat baths.

Figure 1

According to Thermodynamics, work can be reversibly obtained from a heat bath by consuming B bits of an information reservoir and also by decreasing its relative entropy with respect to the thermal state .

In order to simplify the quantification of the resources involved, we exclusively consider unitary operations. In addition, we only allow observational interactions on the physical system. This restricts the set of transformations to those defined by controlled unitary gates, in which the state of the physical system remains in the controlling part. Basically, we compute the informational cost of labeling a physical system by considering the number of pure state tagging qubits at the input minus those recovered at the output. In the following, we may use the terms “initialize” or “reset” to denote the action of driving a tagging qubit to a state.

The tagging operation implies using some coding procedure to correlate the quantum states of a physical system and its label. Conversely, we also consider the process of deleting that correlation and returning the tagging qubits to an information reservoir. We assume that the code has to be optimal in the sense that it uses the least average amount of bits to identify the state of a physical system. Averaging is defined with respect to an underlying probability distribution of states. For this reason, we choose a Shannon coding technique; it is asymptotically optimal and provides a simple relation between the lengths and the probabilities of the codewords. We consider two degrees of tagging. The tight-labeling implies a reversible assignation of a label to every physical system. It is described in Section 2. The loose-labeling implies tight-labeling a collection of physical systems followed by a random shuffling. It is studied in Section 3. The discussion is presented in Section 4 and the conclusions in Section 5. A simple example, using magnetic spins, is given in Appendix E, in order to illustrate some of the ideas presented in the paper.

2. Tight Labeling

In this section, we present a method for writing a label, consisting of a set of qubits, with the purpose of representing the quantum state of a physical system. We call it tight because each label is assigned unambiguously to the state of the physical system that it is attached to.

We consider a sufficiently large set of identical physical systems that will be referred to as atoms. For simplicity, a two-dimensional Hilbert space for them is assumed. The statistical distribution for each atom is determined by its quantum state, which is known to be . The eigenstates of are denoted by and their eigenvalues are ordered as . Atoms are grouped in clusters of a common length N. Besides the atoms, we assume unlimited availability of tagging qubits in either a pure or a maximally mixed state (see Figure 2).

Figure 2

Our labeling procedures assume the availability of tagging qubits in either a or a maximally mixed state, and physical systems, referred to as atoms. The labeling assigns a set of H tagging qubits to a cluster of N atoms. The cost is defined as the number of tagging qubits in state employed.

We consider a coding basis that diagonalizes . Its vectors are denoted by , where can be either 0 or 1.

The operations considered are:

Any unitary transformation on a system T of tagging qubits.

Unitary transformations on joint states of cluster C and a system T of K tagging qubits, that are defined, using the basis for C and the computational basis for T, by: where is any function that transforms binary strings of N bits into strings of K bits. They can be considered as just probing operations with respect to clusters. It is also easy to check that .

The cost of any operation will be tallied as the number of tagging qubits in a pure state that are input and not recovered at the output.

We further choose a binary Shannon lossless code for the sequences according to a frequency given by the eigenvalue of that corresponds to . The procedure defines a function that assigns a binary codeword to every string . Let H be the maximum length of all the codewords. We further define a label as an array of H tagging qubits. The vectors of the label computational basis are denoted by , where can be either 0 or 1.

The coding procedure determines a unitary operator that acts on every cluster–label pair . It can be defined by specifying how the elements of the basis transform; it is given by: where represents the Shannon codeword supplemented with the necessary trailing 0s to match a length H.

Labeling an N-atoms cluster C in state means applying the unitary operator to the cluster and a label of H tagging qubits in a state, as is depicted in Figure 3. When is diagonal in the basis, the operation is equivalent to the classical operation of coding according to the Shannon method and storing the codeword in the label. The resulting state is a classical statistical mixture of cluster–label pairs.

Figure 3

(a) represents the coding procedure as a unitary operation , controlled by the cluster C, on the tagging qubits of the label L. If the tagging qubits are initially in a state, they hold the coded string for the cluster; (b) represents the inverse operation, which is equal to , so that is the identity transformation.

We define the width of the codeword assigned to the ket as the number of bits in . Only the leading qubits of the label contain information. The trailing qubits of L are superfluous and should be replaced by others in a completely mixed state. For this purpose, we define a new unitary operation by which the trailing qubits are swapped with those of a new label D. In the labeling process, D contains H maximally mixed qubits. The operation of is depicted in Figure 4 and explained more elaborately in Appendix C.

Figure 4

Representation of the procedure employed for replacing the trailing qubits that need not be used in the codeword by maximally mixed ones, as explained in Appendix C. It is split into two unitary transformations. The first one, represented in (a), copies the first qubits of L into that enters with all its tagging qubits in the state. The remaining qubits are copied from the maximally mixed qubits of D. The second transformation, represented in (b), resets the L label and the trailing qubits of D. The overall function is recovering qubits in state and generate a new label with maximally mixed trailing qubits.

In our setting, the width of the codeword is a quantum variable represented by the operator , which operates on the Hilbert space of the cluster. Its eigenvectors are those of the basis, and its eigenvalues are the widths of their Shannon codewords. Its effect on the basis vectors is: and also represents the cost of the labeling procedure. We further define the atomic width as . For sufficiently large N, converges to . Accordingly, Thus, for sufficiently large N, the average atomic width for a codeword in state is given by: which, taking into account that , can be rewritten as and represents the atomic cost of labeling clusters in state . It is straightforward to reverse the process and check that an atomic yield of is obtained.

A cluster in state can be described as being in a probabilistic mixture of eigenstates of . Each of them defines a tight-label that allows for unambiguously identifying which eigenstate of it is attached to. The cost expressed by Equation (6) represents the average length of the codewords that correspond to clusters drawn according to the distribution defined by . The equivalent situation in Thermodynamics corresponds to averaging the work that is necessary to produce pure state physical systems (as spin systems in the example of Appendix E) out of equilibrium, following the distribution given by . However, there is a subtle difference from the case that we want to model in the next section. In it, we still have clusters whose states are drawn from the distribution defined by , but we ignore the particular state of each cluster. To cope with this new situation, it would be sufficient to overlook the precise label that is attached to each cluster, but keeping the distribution that corresponds to . This is implemented through a process of label shuffling for a collection of clusters in state . The procedure is described in Section 3.

3. Loose Labeling

The tagging procedure put forth in the previous section outputs maximally correlated cluster-label pairs. In this section, we describe a procedure that reduces the correlation. To this end, the label L assigned to cluster C will be the codeword of another cluster that belongs to a collection F of M clusters, all of them in state . Therefore, the state of F is . Figure 5 represents the process with unitary gates that acts on a collection of M labeled cluster pairs , a random set of P tagging qubits and an auxiliary label collection . The role played by is to generate a random shuffling of the labels. The process is analyzed in Appendix D, where it is shown that the average number of qubits in the array that exit in a state verifies that, for large M, converges to , where is the entropy associated with the probability distribution for the possible labels of the elements of the basis.

Figure 5

The figure describes the unitary process used to shuffle the labels of a collection F of M tight-labeled clusters, as is explained in Appendix D. Besides the labels, a set of P maximally mixed qubits and a set of M labels , all in state enter the first unitary gate (a). It shuffles the labels , according to the permutation indicated by the qubits and stores the result in the labels. The second gate (b) resets the qubits that signaled the permutation. The last gate resets the labels by regenerating them from the clusters.

Therefore, taking into account the cost of tight-labeling the F collection, given by Equation (5), the one of loosely tagging the clusters of F is which leads to a value per atom: Next, we will find a suitable expression for . Each label is assigned to a vector of the basis. Its frequency is given by . The probabilities for the set of codewords are the eigenvalues of the density matrix It is immediate to check that . Thus, Equation (9) can be rewritten as Notice that represents a CPTP (Completely Positive Trace Preserving) map, which is not a unitary transformation. However, all the operations of our labeling system are unitary. The CPTP map is used here just as a means to find a convenient expression for , not as a real operation on the state of the clusters and labels.

Next, we define a particular base for which Equation (11) will only retain the last term in the limit.

Let be the operators represented by the Pauli matrices in the base, which diagonalizes . In the Hilbert space of the -th atom of a cluster, they are denoted by . For the whole cluster, we define: Let be the basis which diagonalizes (the well-known momentum basis in Quantum Physics).

In the basis, any state is a mixture of states , where is a state defined within the invariant subspace of . The entropy of this mixture is the sum of the entropy associated with the mixture and the weighted entropy of the different : The same decomposition can be applied to the state. Taking into account that commute, so that using Equation (13) for , and substracting both entropies, we arrive at The maximum entropy of a state is , where is the dimension of the -th invariant subspace. Its maximum value is . Therefore, the absolute value of the right half side of Equation (15) can not be greater than . In the limit, so that, for sufficiently large N,

4. Discussion

A well-known situation in Thermodynamics is the availability of a thermal reservoir of freely available, non-interacting atoms in a Gibbs state, given by: where is the Hamiltonian of each atom, T is the temperature and represents the partition function.

A cluster of atoms in another state can be used to obtain work from the energy of the thermal reservoir. The obtainable work per atom, as given in [41,45] and derived in Appendix B, is and is usually collected by some physical means (mechanical, electromagnetic, etc.) which involves coupling to thermal baths and mechanical or electromagnetic energy-storage systems. Remarkably, it only depends on the relative entropy that is connected to the physics through the dependence of on the Hamiltonian and the temperature T. The work obtained matches the heat transferred from the thermal reservoir plus the decrease of the internal energy of the atom. After the process, the state of the atom is . The reverse operation, driving a system initially in state to an out of equilibrium state demands the same amount of work to be supplied in the process.

In this paper, we have come across the relative entropy from a very different approach. We have chosen to employ a coding system that asymptotically minimizes the labeling cost for . Shannon coding for state satisfies this requirement. In the process of labeling, we incur a cost that can be evaluated after substitution of for in Equation (17), in the case of loose-labeling. It is equivalent to the process of driving a system from state to state , with the following observation: while in the Thermodynamic operation the process of transforming requires work, in the labeling approach, it needs qubits in the state.

In a different Thermodynamic setting, we know that work can also be obtained from informational qubits at Information Heat Engines in the presence of a heat bath. Typically, they enter in a known pure state (say ) and exit in a maximally mixed one. They need to be coupled to a physical system that should be able to equilibrate with a thermal reservoir and couple to an external energy storage. The work obtained per qubit is

Accordingly, it is clear that the relative entropy of physical systems with respect to a thermal state can be traded for bits of information reservoirs by means of engines and heat baths within a Thermodynamic context.

We claim that, in this paper, we have described a way to do the same with purely informational manipulations. Furthermore, the physical system is accessed for probing operations that reduce to acting on the information qubits according to its state. Labeling is a particular kind of these processes.

However, the loose-labeling cost, given by Equation (11), depends on the labeling strategy through the choice of the coding basis . The atomic cost does not converge to the relative entropy unless converges to . This is trivial if commute, but, in a general quantum scenario, it can not be assumed. Nonetheless, even in the non-commuting case, it is accomplished by using the eigenbasis described in Section 3. For the general case, when another basis is chosen, does not converge to and the loose-labeling costs are lower than the quantum relative entropy. This can be deduced by substracting both: where we have taken into account that, because is diagonal in , then Notice that , so that Equation (21) can be written as which is always positive by the monotonicity of quantum relative entropy.

At any point, tracing out the label places the reduced state of the cluster back to . Therefore, we interact with the cluster just to obtain or delete information about it. The parallel with the situation in Thermodynamics is clear: bits from information reservoirs are traded for changing the relative entropy of state with respect to . From this point of view, the most important aspect of a physical system in a thermodynamical setting is knowing its state, so that it can be used to supply the corresponding work. It is the state that fixes the process by which work is obtained.

The two types of labeling described exhibit different costs. It is quite obvious because the loose-labeling implies shuffling. This leads to labels that are related not to a particular cluster, but to a collection of them that share some particular state. It is natural that the work given by Equation (19) is related to this cost because it assumes a process which is common to all of them. However, if we have a tight-labeled collection of clusters, we can process each one in a different way, chosen according to its label. Then, each cluster would contribute a work given by the relative entropy of the pure state identified according to the label. The average work value would be given by: which corresponds to the cost of tight-labeling, given by Equation (6), irrespective of the particular choice of the coding basis.

From another perspective, loose-labeling is essentially the process of disarranging the tight-labels of a collection of M clusters. Let us first assume that commute. Asymptotically, as , the number of possible orders for the set of labels tends to , which is precisely the difference between Equations (6) and (17). From a physical point of view, both expressions point to slightly different situations. When a thermal engine is tuned to supply work from a physical system out of equilibrium, its configuration depends on the state of the system. Each pure or mixed state requires different settings. Let be the work obtained from a system in a pure state . Next, we consider two cases:

the pure state of each physical system is known, and the setting can be adjusted according to it. Then, the average work obtained by processing a collection of physical systems is the weighted average of all the , each one contributing according to its corresponding eigenvalue in the density matrix . It is given by Equation (24).

only the collective mixed state of the collection is known. Then, the engine is tuned with a different set of parameters, and the average value of the extracted work is lower than in the previous case. It is given in Equation (19).

Situations (a) and (b) correspond to the tight and loose-labeling techniques, respectively. Work is immediately translated by information heat engines into reservoir bits. The conversion factor is given in Equation (20). The difference in the average value of work in (a) and (b) translates exactly into bits. The same results can be extended to the case when do not commute, provided that coding is defined in a suitable basis.

5. Conclusions

In this paper, we have described two ways to label physical systems grouped in clusters. In both procedures, unitary transformations operate on a Hilbert space determined by the cluster and additional informational qubits. In the tight-labeling method, the label identifies pure states of the cluster, while, in the loose-labeling case, the label is chosen at random from a collection of clusters that share the same mixed quantum state. The costs of both procedures have been deduced in the asymptotic limit of infinite equal systems. The evaluation has been made counting the number of informational qubits in the pure state in the final and the initial situations. They are related to the von Neumann and relative entropies , where is the state of the physical systems and is the state relative to which the labeling is optimized. In both processes, no manipulation of the physical system is attempted. Its intervention only exhibits an observational character.

We have shown that the atomic cost of tight-labeling converges to the sum of the von Neumann and relative entropies . Remarkably, this result does not depend on the coding basis. However, in the loose-labeling case, the atomic cost depends not only on , but also on the coding basis. It is bounded by the relative entropy , to which it can converge when a right basis is chosen, as explained in Section 3.

Assuming this choice of basis, we have shown that the costs of labeling, both in the tight and loose versions, correspond to what thermodynamical processing predicts by combining the models of (a) work extraction from physical systems out of equilibrium, and (b) information heat engines powered by pure state qubits. Through writing and erasing labels, we have presented a procedure to trade relative entropy for von Neumann entropy of the physical system just by informational manipulation.