Quantum pixel representations and compression for N-dimensional images.

We introduce a novel and uniform framework for quantum pixel representations that overarches many of the most popular representations proposed in the recent literature, such as (I)FRQI, (I)NEQR, MCRQI, and (I)NCQI. The proposed QPIXL framework results in more efficient circuit implementations and significantly reduces the gate complexity for all considered quantum pixel representations. Our method scales linearly in the number of pixels and does not use ancilla qubits. Furthermore, the circuits only consist of [Formula: see text] gates and [Formula: see text] gates making them practical in the NISQ era. Additionally, we propose a circuit and image compression algorithm that is shown to be highly effective, being able to reduce the necessary gates to prepare an FRQI state for example scientific images by up to 90% without sacrificing image quality. Our algorithms are made publicly available as part of QPIXL++, a Quantum Image Pixel Library.

Introduction

The growth in scientific data size and heterogeneity overwhelms current statistical and learning approaches for analysis and understanding. More specifically, the analysis of image-based data becomes increasingly challenging using current classical algorithms. Consequently, finding more efficient ways of handling scientific data is an important research priority.

Quantum computing holds the promise of speeding up computations in a wide variety of fields[1], including image processing. One of the research challenges to make quantum computing a viable platform in the post-Moore era is to reduce the complexity of a quantum circuit to accommodate many qubits. The current and near-term quantum computers, known as noisy intermediate-scale quantum (NISQ) devices, are characterized by low qubit counts, high gate error rates, and suffer from short qubit decoherence times[2]. Hence, optimizing quantum circuits into short-depth circuits is extremely important to successfully produce high-fidelity results on NISQ devices.

Quantum image processing (QIMP) extends the classical image processing operations to the quantum computing framework[3]. QIMP algorithms are used on images that have been represented in a quantum state. A variety of quantum image representation (QIR) methods has been developed[4]. The flexible representation of quantum images (FRQI)[5,6], the improved flexible representation of quantum images (IFRQI)[7], the novel enhanced quantum representation (NEQR)[8], the improved novel enhanced quantum representation (INEQR)[9], the multi-channel representation of quantum images (MCRQI/MCQI)[10,11], the novel quantum representation of color digital images (NCQI)[12], and the improved novel quantum representation of color digital images (INCQI)[13] are among the most powerful existing QIR methods. These QIR methods became extremely popular due to two main factors. First, their flexibility in encoding the positions and colors in a normalized quantum state. Second, image processing operations can be performed simultaneously on all pixels in the image by exploiting the superposition phenomenon of quantum mechanics.

In this paper, we introduce a uniform framework called the quantum pixel representation (QPIXL) that overarches all previously mentioned quantum image representations and probably many more. Furthermore, we propose a novel technique for preparing QPIXL representations that requires fewer quantum gates for all the different representations, compared to earlier results, and without introducing ancilla qubits. The proposed method makes use of an efficient synthesis technique for the uniformly controlled rotations[14] and uses only gates and controlled-NOT () gates, making the resulting circuits practical in the NISQ era. For example, the original FRQI state preparation method[5] for an image with grayscale pixels uses qubits in total, i.e., n qubits for encoding the position and 1 qubit for the color, and has a gate complexity. Recently, the FRQI gate complexity has been reduced to at the price of introducing several extra ancilla qubits[7]. In contrast, our QPIXL method for preparing an FRQI state has only a gate complexity of and does not require extra ancilla qubits. Additionally, we introduce a compression strategy to further reduce the gate complexity of QPIXL representations. In our experiments, the compression algorithm allows us to further reduce the gate complexity by up to 90% without significantly sacrificing image quality. An implementation of our algorithms is publicly available as part of the Quantum Image Pixel Library (QPIXL++)[15] at https://github.com/QuantumComputingLab. QPIXL++ is built based on QCLAB++[16,17], which allows for creating and representing quantum circuits.

Related work

Almost every image processing algorithm[18] developed in the classical sense can also be developed in the quantum environment. These quantum versions may be computationally faster and may handle data more effectively by taking advantage of properties such as coherence, superposition, and entanglement associated with quantum science. How an image is represented on a quantum computer dramatically influences the image processing operations that can be applied. Hence, QIR has become a vital area of study in QIMP. Early approaches are the qubit lattice representation[19] and the flexible representation of quantum images (FRQI)[5]. The latter, which is the FRQI method, forms the foundation of our work. The former is a quantum counterpart of classical image representation models without any significant performance improvement. At the same time, FRQI is based on quantum mechanical phenomena and captures both the color and geometry of an image in one quantum state. Besides its flexibility and the use of fewer qubits, FRQI can also perform both geometric and color operations on the image concurrently[20].

Since the FRQI only uses one qubit for storing the color information, the number of measurements to accurately retrieve an image can be very large. The NEQR addresses this issue by storing the color information in orthogonal states allowing for color retrieval in a single measurement. Although the NEQR allows for accurate image retrieval, it requires significantly more qubits and does not utilize the superposition principle in the color qubit sequence, i.e., qubits basis states are used for images with bit depth . On the other hand, the IFRQI combines both ideas and utilizes limited and discrete levels of superposition that are maximally distinguishable. The IFRQI, therefore, ensures accurate image retrieval with a small number of measurements; however, it requires extra ancilla qubits. Other existing quantum image representation models are the quantum image representation for log-polar images (QUALPI)[21], the n-qubit normal arbitrary superposition state (NASS)[22], and the generalized quantum image representation (GQIR)[23].

Several quantum image processing algorithms have been introduced in the literature using these QIRs. For example, Zhang et al.[24,25] introduced an image edge extraction algorithm (QSobel) based on FRQI and also a quantum feature extraction framework based on NEQR. Jiang et al.[26] recently proposed a new quantum image median filtering based on the NEQR. There are image segmentation algorithms that utilizes different QIRs along with the quantum Fourier transform[1,27]. Jiang et al.[23] developed a new quantum image scaling up algorithm based on the GQIR. Li et al.[28] developed a quantum version of the wavelet packet transforms based on the NASS. Zhou et al.[29] proposed a quantum realization of the bilinear interpolation method for NEQR. There are several other examples in major application areas including image filtering[30-33], image segmentation[34-36], and machine learning[37-41].

In order to run a quantum algorithm on a NISQ device, it first needs to be synthesized into elementary 1- and 2-qubit gates. The original implementation of the FRQI[5] required elementary gates, while the more recent implementation by Khan[7] reduced the complexity to elementary gates by introducing extra ancilla qubits. We propose a novel QPIXL synthesis approach that reduces the FRQI gate complexity to , i.e., N rotation gates and N gates, and does not require ancillary qubits. Furthermore, our QPIXL synthesis approach also reduces the original IFRQI gate complexity from to only and also gets rid of the ancilla qubits. Similar gains are obtained for preparing (I)NEQR, MCRQI, and (I)NCQI states.

QPIXL: Quantum pixel representations

Some of the most widely used representations for quantum images, such as (I)FRQI[5,7], (I)NEQR[8,9], MCRQI[11], and (I)NCQI[12,13], can all be described by the following general definition for quantum image representations. This representation is similar to the pixel representation for images on traditional computers and captures both pixel colors and positions into a single quantum state that we call a quantum pixel representation, QPIXL in short.

Definition 1

(Square QPIXL) The quantum state for the QPIXL representation of a image , where each pixel has color , is given by the normalized statewhere are the computational basis states on 2m-qubits and is an encoding of the color information in a quantum state on one or more qubits. The color values should be regarded as a vectorized version of the 2D color values , i.e., for .

We remark that the order of and in Definition 1 is reversed compared to the original definition[5,6]. Our ordering is consistent with the quantum circuit implementation for provided in “QPIXL quantum circuit implementation” and in the original work[5,6]. Observe that the QPIXL state creates an equal superposition over the computational basis states of the 2m-qubits in the first register, which encodes the pixel positions, and applies a tensor product with the state on the second register that encodes the color information. Definition 1 is general because it allows for flexibility in the type of color information and color encoding that is used. The mentioned QPIXL representations differ in their approach to map to .

Since Definition 1 can trivially be extended to rectangular, 3D, and higher dimensional images, we will use the following more general definition.

Definition 2

(General QPIXL) The quantum state for the QPIXL representation of an image of N pixels is given by the normalized quantum statewhere , is an encoding of the color information of pixel , and are the computational basis states on n-qubits.

Remark that in case the number of pixels N is not a power of 2, Definition 2 appends zero-valued pixels for . Consequently, the state (2) is fully determined by the N pixel values . Without loss of generality, we will assume that in the remainder of the paper.

QPIXL quantum circuit implementation

The preparation of a QPIXL state on a quantum computer can be considered as a state preparation procedure, i.e., is the result of a quantum circuit applied to the all-zero state , where n qubits are used to encode the pixel position and qubits are used for the color information.All QPIXL states are prepared in two steps: first creating an equal superposition over the n qubits that determine the pixel positions and afterwards adding the color information to the state by means of a unitary . In matrix notation, this procedure yieldswhere creates an equal superposition over the first n qubits:

FRQI in the QPIXL framework

The FRQI[5,6] fits Definitions 1 and 2 of the QPIXL framework and is applicable to grayscale image data. An FRQI encoding uses only 1 qubit for the pixel intensity information . The color mapping used is bijective as discussed in detail by Li et al[22,42]. We define this mapping as follows.

Definition 3

(FRQI mapping) For a grayscale image of N pixels where each pixel has a grayscale value , i.e., an integer value between 0 and the maximum intensity K, the QPIXL state with the FRQI mapping is defined by Definition 2 with the color mapping used in (2) given by[5,6,22,42]with and .

Observe that the FRQI representation of an N-pixel grayscale image requires qubits in total: n qubits for the pixel positions in and 1 qubit for encoding the corresponding pixel intensity information in . By Eq. (5), we have that and

Definition 3 is flexible because the grayscale value of each pixel can be encoded by choosing the angles accordingly. For example, consider an 8-bit grayscale image where each pixel has a grayscale value between 0 and 255, then the angles in Eq. (5) are given by[5,6,22,42]On the other hand, repeated measurement of the quantum state yields the probabilities and for the basis states and , respectively. Hence, we can retrieve the grayscale values from these measurements by

We note that the color mapping defined in Eq. (5) has disadvantages when the images are transformed as discussed by Li et al[43]. In this case the authors propose extensions of the FRQI, named FRQIM and FRQIMC, in order to overcome the inconvenience to implement non-permutation transforms on FRQI. For the purposes of our work, we assume until “Other QPIXL mappings” that all image data is in grayscale and that we use the FRQI encoding from Definition 3.

QPIXL-FRQI quantum circuit implementation

The circuit structure introduced in “QPIXL quantum circuit implementation” can be used to prepare the FRQI state on a quantum computer. In this case we have and that implements the mapping from Definition 3, we will denote this unitary as . This specification yieldswith, according to Eq. (4),We definewithSince is by definition a block diagonal matrix with N blocks andthe prepared FRQI state (9) becomeswhich is a vector of length 2N holding the cosine and sine values of the angles of all the pixels. It can be directly verified that this definition of agrees with Definition 3.

We can implement the circuit on a quantum computer by using N multi-controlled gates[5]. We use the notation for an gate with n control qubits. To illustrate this, we consider the FRQI encoding of a image. This 4 pixels image can be implemented as follows using 3 qubits and 4 gates:The angles correspond to the pixel values for according to Eq. (5). The decomposition of the block diagonal matrix into multi-controlled gates corresponds to the following matrix decompositionwhere each multi-controlled gate sets a single block on the diagonal.

In order to actually run the circuit on a quantum computer, we need to further synthesize the multi-controlled gates into elementary 1- and 2-qubit gates. For the case of gates this can be done as follows[44]:yielding the following circuit for the 4 pixels image example:By further decomposing the gates into 3 and 2 CNOT gates as follows[1],the directly implementable quantum circuit for requires 44 single-qubit and 32 CNOT gates in total. In the general case for images with pixels, every individual pixel value is encoded by a gate. Decomposing these gates into 1- and 2-qubit gates by the method of Barenco et al.[44] requires gates for every gate. This results in an overall circuit complexity for that scales quadratically in N, i.e., elementary gates are required to implement the full circuit for an N pixels image on a quantum computer[5]. Khan[7] recently improved the asymptotic complexity to by using ancilla qubits.

Optimal linear gate complexity

The complexity of implementing is determined by the complexity of the circuit for , a block diagonal matrix with blocks corresponding to the pixel values. In this section, we derive an alternative circuit implementation for that requires quadratically fewer gates compared to the method proposed by Le et al.[5], i.e., the asymptotic complexity of our novel implementation requires only quantum gates for a N-pixel image. Our new approach thus has optimal asymptotic scaling. It is also logarithmically faster compared to the method proposed by Khan[7] and requires no ancilla qubits.

We start by reviewing a special case of the method introduced by Möttönen et al.[14] to implement a block diagonal matrix in a quantum circuit. In that work, these circuits are called uniformly controlled rotations because they uniformly use all possible computational basis states in the control register. Let us define the nomenclature and diagrammatic notation for uniformly controlled rotations.

Definition 4

(Uniformly controlled rotations) Given , a vector of rotation angles, the uniformly controlled rotation is defined asand represented diagrammatically asThe dashed line indicates the qubits required for controlling the different diagonal positions in . The diagram on the right hand side uses a square control node to indicate that it is uniformly controlled by the first n qubits.

We know from the previous section that we can implement by using N gates. Here, we show that we can do this more efficiently by using a circuit that only consists of and gates. As an illustrative example, let us consider the following circuit for 4 arbitrary angles :The following two properties of rotations are immediate:where X is a NOT gate that appears in the CNOT gates above. We can analyze the circuit above using these two simple properties and show that the circuit does create a block diagonal matrix with blocks on the diagonal: the rotations on the 3rd qubit are all block diagonal matrices and the CNOT gates permute some of the blocks depending on the index of the first two control qubits. If we list the four diagonal blocks in binary order, or equivalently the state of the 1st and 2nd qubit, we see that the circuit has the following effect on each block:To implement a block diagonal matrix with this circuit, where the angles of the blocks correspond to , we get that the angles have to satisfyThis is a linear system with a specific structure, that we can rewrite aswhere is a scaled version of the Hadamard gate and is the permutation matrix that transforms binary ordering to Gray code ordering.

It follows that, if we solve the linear system (17) for , we can implement for any image with only 8 elementary gates: 4 rotations and 4 CNOT gates. The circuit for the example in the previous section required 74 gates: 42 1-qubit and 32 CNOT gates. Indeed, we have a quadratic improvement in gate complexity.

This strategy generalizes to block diagonal matrices that have blocks on their diagonal[14]. The circuit structure consists of a sequence of length alternating between gates and CNOT gates. The gates act on the st qubit, and thus correspond to block diagonal matrices with blocks. The target qubit of the CNOT gates is set to the st qubit and the control qubit for the th CNOT gate is set to the bit where the th and st Gray code differ. If is determined by the angles , the angles of the circuit can be computed through the linear system:As can be observed from the small-scale example (16), each angle in the transformed domain contributes to every angle in in the original spatial domain. This means that there no longer exists a correspondence between an individual angle and an individual pixel intensity . As we will illustrate in “Experiments”, this can be considered an advantage as it allows one to approximate nonlocal correlations between pixels with fewer coefficients. In QPIXL++, Eq. (18) is solved with a matrix-free approach: the Gray permutation is performed in place and requires operations, the scaled Walsh–Hadamard transform is implemented through a variant of the fast Walsh–Hadamard transform which requires operations[45]. Pseudocode for both algorithms are provided in Algorithm 1 and Algorithm 2. Algorithm 2 lists a implementation for the Gray code permutation that requires a copy, while the QPIXL++ implementation achieves the same complexity without requiring a copy. Our implementation uses double precision arithmetic which suffices as the problem is well-conditioned, i.e., .

To show that our approach scales to large-scale images, we present benchmark data for solving the linear system (18) with the matrix-free methods that are implemented in QPIXL++. The results are shown in Figure 1 for randomly generated image data ranging from pixels up to pixels. The latter corresponds to the equivalent of an image with a resolution of more than 17 gigapixels, a 4K video fragment with 2070 frames, or a 1080p video fragment with 8285 frames. These timing results are obtained on a single core of an AMD Ryzen Threadripper 3990X 64-Core Processor @ 2.9 GHz with 256 GB RAM. Computing the coefficients for the data with pixels requires just over 5 min. This shows that our method easily scales to high resolution image and video data. The only current drawback is that this computation is memory bound due to the memory required to store the image data.

Figure 1

Scaling for scaled fast Walsh–Hadamard transform (sFWHT) and in-place Gray permutation with QPIXL++.

Our new circuit requires only N rotation and N CNOT gates for an image with N pixels. As this scales linearly in the number of pixels, the asymptotic complexity of our approach is optimal. This is a quadratic improvement compared to the approach proposed by Le et al.[5] that we described in “FRQI in the QPIXL framework”. The asymptotic complexities of both approaches are summarized in Table 1. We remark that as we require just 2 gates for every pixel, our constant prefactor is also considerably smaller compared to the works by Le et al.[5] and Khan[7].

Table 1

Summary of gate complexities and qubit count for preparing the FRQI state for an image with pixels with the approaches of Le et al.[5] and Khan[7] compared to our method.

FRQI	Gate complexity	Ancilla qubits	Total qubits
Le et al.[5]	O(N^2)	0	n+1
Khan[7]	O(Nlog_2N)	n-2	2n-1
QPIXL	O(N)	0	n+1

Compression

The proposed implementation of as presented in “Optimal linear gate complexity” lends itself to an efficient circuit and thus image compression technique. As an example, we describe this idea for an FRQI image with 8 pixels.

Assume that the FRQI angle representation of an image is given by the vector and that we have computed the transformed vector according to Eq. (18). The coefficients of are then used in the following circuit for :For conciseness, we omit the labels and only state the rotation angle for the gates. Now assume that the image after the permuted Walsh-Hadamard transform is of the form , where are angles that can be considered negligible according to some compression criterion. A good approximation of the image is then given by . This corresponds to the circuit below on the left where all rotations that have 0 angle after compression have been removed. This corresponds to a reduction in gates or compression level. This step results in a sequence of consecutive gates all with the same target qubit and different control qubits. All these gates commute with each other, so we can place them in arbitrary order. Furthermore, two consecutive gates that have the same control qubit cancel each other since their product is the identity. The circuit below on the left has in the middle 1 CNOT with the first qubit as control, 2 s with the second qubit as control that cancel out, and 3 s with the third qubit as control of which two cancel with each other. It follows that the circuit on the left is equivalent to the circuit on the right with the redundant gates removed.Figure 2 illustrates the compression algorithm for an actual image of 8 pixels where all transformed angles below the tolerance are set to zero. Note that, although the compression can influence all the angles , the changes of the grayscale values are only in the range of . The reason for this is that Eq. (18) is well-conditioned so that small changes in only lead to small changes in and its corresponding grayscale values.

Figure 2

Compressing image data with 8 pixels arranged in a grid.

As we describe next, this procedure easily generalizes to images of arbitrary size. After having computed , apply a compression criterion to set the negligible coefficients to 0. Next, remove the corresponding rotations with 0 angle from the circuit. Finally, perform a parity check on the control qubits of consecutive s in the circuit: no is required for control qubits with even parity, one is required for control qubits with odd parity.

This algorithm is implemented in QPIXL++[15]. The compression criterion that we adopted selects a fixed percentage of the coefficients with largest magnitude and thus of most importance. For example, a compression setting of retains all nonzero coefficients in , while a compression of sets the smallest coefficients to zero. As we show in “Experiments”, this method can achieve high compression ratios while maintaining many features of the uncompressed image. The advantage of our approach is that we can discard coefficients after the Walsh-Hadamard transformation has been applied. In this way nonlocal correlations can be approximated with fewer coefficients compared to the untransformed data which can allow for improved compressibility. Furthermore, removing negligible angles in is guaranteed to lead to small perturbations of the original angles as Eq. (18) is well-conditioned.

Other QPIXL mappings

In this section, we extend our novel circuit implementation for for grayscale data to different image representations that fit in Definitions 1 and 2. The key difference between all representations is the definition of the color encoding in the quantum state from Definition 2. As long as we express this color mapping in terms of a combination of rotations, we can use our compressed implementation for the uniformly controlled rotations.

IFRQI

The improved FRQI method introduced by Khan [7] combines ideas from the FRQI and NEQR representations. It improves upon the measurement problem for FRQI by allowing for only 4 discrete superpositions that are maximally distinguishable upon projective measurement in the computational basis. The IFRQI color mapping for a grayscale image with bit depth 2p is defined as follows.

Definition 5

(IFRQI mapping) For a grayscale image of N pixels where each pixel has a grayscale value with binary representation , the IFRQI state is defined by Definition 2 with the color mapping used in (2) given bywhere, for

We observe that the IFRQI mapping combines two bits of color information into one rotation. It follows that for an image with bit-depth 2p, we can prepare using the circuit presented in Fig. 3a with p uniformly controlled rotations. The rotation angles correspond to bits 2i and of all N pixels according to the values defined in Definition 5. These uniformly controlled rotations can be compressed independently with our compression algorithm. The gate and qubit complexites for IFRQI with our method compared to Khan [7] are listed in Table 2.

Figure 3

Circuits for the preparation of the IFRQI, NEQR, MCRQI, and INCQI states, where the uniformly controlled rotations can be compressed with our method.

Table 2

Summary of gate complexities and qubit count for preparing the different QIR states covered in this paper and QPIXL for an image with pixels.

Method	Literature				QPIXL
	Reference	Gate complexity	Ancilla qubits	Total qubits	Gate complexity	Total qubits
FRQI	Le et al.[5]	O(N^2)	0	n+1	O(N)	n+1
	Khan[7]	O(Nlog_2N)	n-2	2n-1
IFRQI	Khan[7]	O(pNlog_2N)	n-2	2n+p-2	O(pN)	n+p
NEQR	Zhang et al.[8]	O(ℓNlog_2N)	n-2	2n+ℓ-2	O(ℓN)	n+ℓ
INEQR	Jiang et al.[9]
MCRQI	Sun et al.[10]	O(3N^2)	0	n+3	O(3N)	n+3
NCQI	Sang et al.[12]	O(3ℓNlog_2N)	n-2	2n+3ℓ-2	O(3ℓN)	n+3ℓ
INCQI	Su et al.[13]	O(4ℓNlog_2N)	n-2	2n+4ℓ-2	O(4ℓN)	n+4ℓ

For the IFRQI state, the bit depth is given by 2p and for the (I)NEQR, MCRQI, and (I)NCQI states the bit depth is given by .

NEQR

The idea for NEQR is to use a color mapping that directly encodes the length bitstring for the grayscale information in the computational basis states on qubits. The NEQR states for different colors are thus orthogonal and can be distinguished with a single projective measurement in the computational basis. In our QPIXL framework, the NEQR mapping can be defined as follows.

Definition 6

(NEQR mapping) For a grayscale image of N pixels where each pixel has a value with binary representation , the NEQR state is defined by Definition 2 with the color mapping used in (2) given bywhere

By choosing the rotation angles orthogonal, we ensure that the color information in can be retrieved through a single projective measurement. The NEQR state can be prepared through the circuit shown in Fig. 3b, where the uniformly controlled rotations can again be compressed with our method. The gate complexities for the uncompressed circuits are listed in Table 2.

MCRQI

If we want to extend the applicability of the FRQI from grayscale to color image data, we have to allow for different color channels. This approach was dubbed multi-channel representation of quantum images (MCRQI)[11]. We adapt their definition for RGB image data to our formalism and make some minor modifications.

Definition 7

(MCRQI mapping) For a color image of N RGB pixels, where the color of each pixel is given by an RGB triplet , the MCRQI state is defined by Definition 2 with the color mapping used in (2) given bywhere

We see that to encode the color information for an RGB image, we only require 2 additional qubits compared to grayscale data, which is a significant improvement over the classical case. Furthermore, we encode the color mapping as a tensor product of three qubit states, while Sun et al.[11] encodes the information in the coefficients of the color qubits, which entangles their state. Our implementation has the advantage that the different color channels are easily treated separately, while the color information can still be retrieved thanks to the normalization constraint.

The circuit implementation of for the RGB mapping defined in Definition 7 then simply combines three uniformly controlled rotation circuits with different target qubits and coefficient vectors determined by the respective color intensities as shown in Fig. 3c. As the RGB color channels are independent of each other and the uniformly controlled gates have different target qubits, each of them can be compressed separately. The asymptotic gate complexity of our method compared to the work by Sun et al.[11] is listed in Table 2. As that work essentially uses the construction of Le et al.[5], we obtain a quadratic improvement before compression.

INCQI

Similarly to the NEQR, the (I)NCQI uses a color mapping directly encoding the length bitstring for each color value in a RGB image in the computational basis stated on qbits. Consequently, this QIR can also be easily represented by our QPIXL framework through the mapping defined as follows.

Definition 8

(INCQI mapping) For a color image of N RGB pixels, where the color of each pixel is given by a tuple and each channel value in the range has a binary representation, the INCQI state is defined by Definition 2 with the color mapping used in (2) given bywhere

The definition above applies very similarly to the NCQI[12], only removing channel from the equation. The INCQI state can be prepared through the circuit shown in Fig. 3d. This circuit is built using an NEQR circuit for each channel of the ICNQI. Similarly to previous QIRs, the uniformly controlled rotations used here can also be compressed with our method. The gate complexities for the uncompressed circuits are listed in Table 2.

Further extensions

We remark that multiple extensions and combinations of the ideas presented in this section are possible. For example, where MCRQI is a color version of FRQI and (I)NCQI is a color version of NEQR, we can similarly define a color version of IFRQI. We can also adapt IFRQI to group an arbitrary number of bits instead of the two bit pairing from Definition 5. This reduces the required number of qubits and gates at the cost of quantum states that are less distinguishable and thus require more measurements. It is even possible to use different QPIXL mappings for different RGB color channels. For example, we can use an FRQI mapping for the red channel, an IFRQI mapping for the green channel, and an NEQR mapping for the blue channel. Also, a generalized version of NEQR (GNEQR) was proposed by Li et al[46], which is based on NEQR, INEQR, and NCQI. GNEQR uses qubits to represent an image with pixels and bit depth of for 4 color channels. Using similar ideas described in this section, a QPIXL-based GNEQR would need total number of qubits.

Finally, although we have presented this discussion for image data in an RGB() space, as in the work by Sun et al.[11], our approach can be readily adapted to different color spaces and even multi-spectral or hyper-spectral data. In fact, different scientific applications frequently use images in different color spaces depending on the type of analysis needed. For example, the Y’CbCr space is known for its applicability to image compression. The I1I2I3 was created targeting specifically image segmentation. The HED space is advantageous in the medical field for the analysis of specific tissues. Similarly, multi-spectral and hyper-spectral data are used in areas such as geosciences and biology, for example, where experts acquire different satellite images and mass spectrometry images respectively. In all these cases, our general definition of quantum pixel representations can be directly applied.

Experiments

This section describes a series of experiments that illustrate our proposed tools implemented in QPIXL++[15]. The current version of QPIXL++ supports the FRQI mapping from Definition 3 for grayscale image data of arbitrary dimensions.

Our first experiment replicates a result from Le et al.[6] with our circuit and compares the gate complexities. In this test, we consider 10 images with an resolution containing representations of the digits 0–9 as shown in Fig. 4. These binary images only contain black and white pixels. We require 5 qubits to encode the pixel location as we have 32 pixels in total.

Figure 4

image data containing digits 0–9, experiment replicated from Le et al.[6]. Gate complexities for the 6-qubit circuits that prepare an exact representation of the image data. The last two rows provide the reduction in gate count for our method compared to Le et al.[5,6]. All circuits contain 5 Hadamard gates to create an equal superposition over the first register.

The method of Le et al.[5] requires one gate for every pixel, bringing the total up to 32 gates. Every gate is further decomposed into 93 and 92 gates. The experiment described by Le et al.[6] reduces the number of gates through a compression algorithm that groups pixels with the same grayscale value. This method is effective for the binary data in Fig. 4 as they report lossless compression ratios between and . Figure 4 compares the number of 1-qubit and gates for our method with the results from Le et al.[6]. We ran our compression algorithm with a compression level of 0% to the circuit. Thus only coefficients in that are exactly 0 are removed, which means that our circuits are exact. Figure 4 shows that our method always provides more than 95% reduction in gate count compared to the method from Le et al.[5,6] for this example. The advantage of our method becomes even more outspoken for larger images due to the quadratic improvement.

The next example we present concerns an image taken from the MNIST database[47,48] of handwritten digits. The image of the digit “3” has a resolution of pixels that is zero padded to an image with 1024 pixels in QPIXL++ which means that roughly 75% of the coefficients are used for the actual image data. Figure 5 shows the images that are simulated with QPIXL++ at 5 different compression levels. There are no visual artifacts at 30% compression and also the image at 60% compression is close to the original quality. The image with a 75% compression ratio has more visual artifacts but is still clearly recognizable, while at 90% compression the quality begins to drop significantly. The corresponding gate complexities for the circuits are also listed in Fig. 5, all circuits contain 10 Hadamard gates to create the superposition in the first register. We observe that the reduction in gates is in perfect agreement with the compression ratio, but that there is generally a smaller reduction in gates. This is in line with the expectations for our proposed compression algorithm described in “Compression”: not all gates along a sequence of removable gates will cancel out. This experiment in particular clearly identifies a potential application of our QIR with compression to classification algorithms based on machine learning in quantum computers.

Figure 5

image data from of a ceramic matrix composite sample[49] acquired using microCT simulated with QPIXL++ at various compression levels and corresponding gate counts of the 17-qubit circuit. The final two rows list the reduction in and gates compared to the uncompressed circuits.

Our final example image stems from scientific data. This is a pixels region from a cross-section of a ceramic matrix composite (fiber reinforced polymer)[49] imaged with X-ray micro computed tomography (microCT) at the LBNL ALS beamline 8.3.2. This type of image is frequently acquired by material scientists to study the development of material deformation under stress. Consequently, image analysis algorithms to detect the circular patterns present in the image for example (cross-sections of fibers) become extremely important. As the dimensions of this grayscale image are already a power of 2, it does not need to be zero-padded. It contains both large scale structure and fine scale details. We require 16 qubits to encode the pixel locations and 1 for the grayscale intensities such that the circuit has a total of 17 qubits. The uncompressed circuit contains or 65,536 and gates. We ran our compression algorithm on the data and the results are summarized in Fig. 6.

Figure 6

image data from the MNIST[47,48] database simulated with QPIXL++ at various compression levels and corresponding gate counts of the 11-qubit circuit. The final two rows list the reduction in and gates compared to the uncompressed circuits.

As can be observed, the compression algorithm is very effective for this image. Up to 75% compression can be achieved while still maintaining both the large scale structure and the finer details. The large scale structure is still preserved at 95% compression, but the acuteness in the finer details is lost at this compression level. It is only at 99% compression that the image becomes completely dominated by compression artifacts. It becomes clear from this last example that our compression approach becomes extremely interesting when analyzing scientific data: (1) the amount of data to be processed is reduced, and (2) the approach maintains details in the image necessary for further analysis, such as feature extraction for example.

Conclusion

We have introduced an overarching framework for quantum pixel representations and showed how previously introduced image representations can be incorporated in the QPIXL framework. Among these methods are (I)FRQI, (I)NEQR, MCRQI, and (I)NCQI. We have proposed a novel circuit synthesis technique for preparing the quantum pixel representations on a quantum computer. This technique makes use of uniformly controlled rotations and significantly reduces the gate complexity for all aforementioned methods. Hence, the obtained circuits only require and gates which makes them feasible for the NISQ era. Our method requires the solution of a particular linear system which can be solved classically in time with a matrix-free approach. Furthermore, it allows for an efficient image compression algorithm that works on the transformed image data. Our experiments show that this compression approach is very effective for the FRQI mapping and can further reduce the number of gates by as much as 90% while still retaining the most prominent features of the image in the FRQI state. We repeatedly show how our method can have great impact on the analysis of scientific data and for quantum machine learning applications in the future. We have implemented and tested our algorithms in a publicly available software package QPIXL++[15] which supports QASM output. Benchmark timings show that QPIXL++ has excellent scaling properties and can handle high resolution image and video data.