Analytical and mathematical methods for revealing hidden details in ancient manuscripts and paintings: A review.

In this work, a critical review of the current nondestructive probing and image analysis approaches is presented, to revealing otherwise invisible or hardly discernible details in manuscripts and paintings relevant to cultural heritage and archaeology. Multispectral imaging, X-ray fluorescence, Laser-Induced Breakdown Spectroscopy, Raman spectroscopy and Thermography are considered, as techniques for acquiring images and spectral image sets; statistical methods for the analysis of these images are then discussed, including blind separation and false colour techniques. Several case studies are presented, with particular attention dedicated to the approaches that appear most promising for future applications. Some of the techniques described herein are likely to replace, in the near future, classical digital photography in the study of ancient manuscripts and paintings.

Introduction

This review is focused on the analytical techniques and methods that have been used to date and are likely to be used extensively in the near future to reveal hidden details in cultural heritage artefacts. Technically, all techniques used in archaeometry (the discipline that applies scientific methods to the study of cultural heritage and archaeology) are aimed at revealing what is not evident and cannot be determined without the use, as a matter of fact, of specific analytical techniques and methods. To further define the scope of this paper, the discussion will be focused on the techniques that may help improve the interpretation and understanding of the manuscripts and paintings, not considering techniques such as radiography or X-ray tomography, which, although extremely interesting for their applications, are typically used for acquiring bulk information, well below the visible surface of the objects under study. These techniques would require, because of their complexity and importance, a full separate review. In the following, probing methods, instrumentation and digital processing techniques for the analysis of the artefact surface are described and discussed. Particular attention is devoted to spectrally resolved imaging methods (reflectometry, fluorescence), although methods based on thermal or elemental analysis of artefacts are also considered when functional to the recovery of surface information. Among the processing techniques, only those that operate on sets of images (separation techniques, false colour imaging, etc.), rather than those operating on single greyscale images (image enhancement technique, segmentation, etc.) are discussed. In the conclusion, a brief discussion of the most promising approaches in the field is presented.

Keywords, including image analysis, cultural heritage, archaeology, multispectral imaging, ancient manuscripts, and blind separation techniques were searched through database “Scopus®” from 1968 to 2018.

Probing techniques

Multispectral imaging – MSI

Multispectral imaging is one of the most popular techniques for the study of cultural heritage and archaeological findings. One main advantage of MSI is that it is a non-invasive technique and therefore can be applied to any artwork, despite its possible fragility. Although the spectral resolution of this type of analysis is, in general, limited (typical bandwidths are of the order of 50 nm or even larger in multispectral imaging and of the order of 10–20 nm in so-called hyperspectral imaging), the amount of information that can be obtained is extremely high, considering the high spatial resolution of the images that can be obtained through very simple experimental setups.

MSI, originally developed for remote sensing applications, began to be applied extensively in art conservation and art history in the early 1990s [1], [2], [3], [4], [5], [6], [7], as it can reveal information in an artwork that cannot be seen by the human eye.

A multispectral image can be described as a set, or cube, of images of the same scene taken over different spectral ranges, i.e., at different wavelengths in the electromagnetic spectrum, including light outside of the visible range, such as infrared (IR) and ultraviolet (UV) light. Reflectance and fluorescence images can be independently acquired but treated simultaneously [8].

From an experimental point of view, an image in a multispectral cube (a channel) can either be isolated by specific filters [9] or using appropriate narrow-band illumination systems [10]. Scanning systems can also be used [11].

In the method’s simplest realization, four images of the subject under study are acquired in the blue, green, red and infrared spectral bands. In most cases, the infrared image is the one carrying the most information because of the unique ability of infrared radiation to penetrate the object surface, allowing for the visualization of otherwise invisible details such as underdrawings and pentimenti in canvas and panel paintings [11], [12], [13], [14]. Infrared imaging is also important for other applications because of the possible enhancement of features deriving from the different infrared reflectivity of the subject’s constituent materials. The improvement of readability of degraded manuscripts in the infrared image was demonstrated, for example, in the recovery of the burnt Herculaneum scrolls [15] and in revealing several hidden characters obscured by exposure to moisture in the Dead Sea Scrolls [16]. In other cases, e.g., in palimpsests or archaeological wall paintings, imaging in the UV spectral range often succeeds in providing additional information [17], [18].

In addition to highlighting hidden patterns, multispectral images and their further elaborations can also provide information on the materials used for the realization of a painting [19], [20], [21], [22], on illumination conditions and pigment identification [23], [24], and for monitoring the conservation of cultural heritage objects [25], [26], [27]. Grifoni et al. [28], [29] recently proposed the use of spectrally resolved images as photogrammetric sources for building 3D models of paintings that would carry information about the painted surface in depth structure (see Fig. 1).

Fig. 1

3D multispectral reconstruction of the surface of a painting. (a) RGB; (b) UV–Vis fluorescence; (c) infrared [29].

Multispectral and hyperspectral imaging, along with techniques for the digital processing of the acquired images, has been the focus of several national and international projects devoted to the study of precious artworks of great historical value. In most cases, dedicated imaging equipment has been devised and implemented. The study of ancient manuscripts and, among them, of palimpsests in particular, is one of the fields where multispectral imaging has demonstrated to give excellent results. The Archimedes palimpsest project [30] has been one of the most important efforts in this field, aimed to the recovery from a XIII century prayer book of the erased and overwritten text of a earlier copy of two lost treatises of Archimedes. In the framework of this project, Easton et al. [31], [32], [33] introduced an MSI acquisition system that makes use of narrow-band LEDs.

Other projects have been carried out regarding palimpsests in Europe, among which one of the most important and comprehensive has been the European Project “Digitale Palimpsestforschung“ (2001–2004) [34]. The project was led by the University of Hamburg and gathered the efforts of more than 50 partner institutions from 26 countries to study a large part of existing Greek palimpsests, with the help of newly developed digital technologies. From a technological perspective, new multispectral capture systems were among the results of this project, along with a set of basic image enhancement techniques and computer tools for document archiving and cataloguing.

In the project “Critical Edition of the New Sinaitic Glagolitic Euchology (Sacramentary) Fragments with the Aid of Modern Technologies” [35], [36], a portable MSI system has been used to image the Sinaitic Glagolitic manuscripts. This system consists of two multispectral LED panels and two different cameras, a greyscale camera with sensitivity from the UV to the near-infrared (NIR), and a traditional RGB camera utilized for UV fluorescence and visible-light imaging.

Also in the field of manuscript analysis, Bianco et al. [37] described an MSI apparatus that uses a filter wheel consisting of eight different optical filters and a monochromatic camera for simultaneous 3D acquisition. Lettner et al. [38] introduced a similar MSI system with an extra single-lens reflex camera. Rapantzikos and Balas [39] used a system with optical filters for imaging over 34 narrow spectral bands.

The efficacy of MSI for the analysis of texts was evaluated in [40].

X-ray fluorescence (XRF)

X-ray fluorescence (XRF) can be used to support MSI for the non-destructive elemental analysis of those parts of the artwork in which MSI is ineffective. This technique consists of acquiring the spatial distribution of the chemical elements [41], [42] of large samples.

When used to probe ancient manuscripts, XRF can distinguish among different types of iron-gall inks due to its high sensitivity to iron concentration and the impurities (typically of copper and zinc) that characterize different batches of ink or inks of different periods [43].

Experiments on the use of XRF for reading palimpsests have been conducted within a project carried out by the Centre for the Study of Manuscript Cultures at the University of Hamburg and the University library of Leipzig, in cooperation with the Hamburg synchrotron radiation laboratory (HASYLAB) and the German electron synchrotron (DESY). Within that project, monochromatized, high-flux X-ray fluorescence techniques were employed [34].

Knox et al. [44] analysed the capabilities of MSI and XRF in revealing hidden characters in various types of damaged parchment manuscripts. The conclusions of this analysis were that the nature of the inks and the condition of the parchment would influence what regions of the optical spectrum would reveal characters. As general rules, it was concluded that infrared illumination is good for revealing carbon-based ink on blackened parchment, ultraviolet fluorescence (and sometimes reflectance) can enhance erased characters, and finally, X-ray fluorescence can detect iron gall ink that is completely covered by optically opaque materials.

Thermography

Infrared thermography [45] can also be used effectively to reveal the presence of hidden patterns or structures in a large variety of objects. Multispectral imaging normally detects the near-IR radiation emerging from the objects under test (0.75–1.4 μm wavelength range); the typical wavelengths used for thermography belong to the thermal IR range (3–15 μm). Techniques based on infrared thermography are capable of detecting subsurface features in the investigated object by mapping the temperature distribution at its surface and can be implemented in different experimental arrangements [46]. A first distinction can be made on the possible presence of an artificial illumination system: passive techniques evaluate temperature differences naturally occurring at the investigated surface, whereas active techniques rely on the temporal evolution of surface temperature induced by suitably timed and filtered artificial heating systems (usually flash lamps). Both of these approaches have already been used to investigate many classes of objects relevant to cultural heritage, such as historical stone and masonry artefacts [47], [48], [49], archaeological findings and ancient documents [46], [50], [51], [52]. In particular, active pulsed thermography has been successfully applied to non-invasively highlight the presence of ancient texts in parchment book bindings, to characterize the status of conservation of painted decorations and to reveal the presence of possible pentimenti under the painted surfaces [46], [52].

Raman and LIBS imaging

The effectiveness of using micro-Raman imaging, a technique that provides information about the molecular structure of surfaces, together with MSI, was evaluated by Maybury et al. [53] in an analysis of Armenian manuscripts. Deneckere et al. [54] used micro-Raman imaging coupled with the elemental technique of micro-XRF to acquire elemental and molecular images of a Belgian porcelain card. Bicchieri et al. [55] used MSI, FT-IR spectroscopy, micro-Raman and micro-XRF for the analysis of a degraded 18th century manuscript. Finally, Botteon at al. [56] used a variation of Raman microscopy called spatially offset Raman spectroscopy (SORS) to demonstrate the possibility of recovering painted images hidden by, for example, graffiti or other types of overpainting. In fact, any experimental technique capable of reconstructing spectrally resolved images of the surface of cultural heritage artefacts can be used for recovering hidden information. Elemental images obtained using Laser-induced Breakdown Spectroscopy (LIBS), a micro-destructive spectroscopic technique, were reported in [57], [58]. Among these, non-destructive approaches are obviously preferable, when applicable.

Digital processing techniques

Statistical analysis and source separation

Among the image processing techniques typically explored using MSI data, statistical analysis and dimension reduction have proven to be powerful tools for further enhancing and detecting hidden patterns in artworks or removing unwanted interferences. Dimension reduction can be both unsupervised, as in blind source separation (BSS) techniques [59], [60], and supervised, as in Fisher linear discriminant analysis (LDA) [61].

Indeed, unsupervised dimension reduction techniques, such as principal component analysis (PCA) and independent component analysis (ICA), linearly combine highly correlated spectral images to produce a different set of images that are uncorrelated and show decreasing variance. Furthermore, the output channels of ICA are statistically independent. Thus, the main principle underlying the enhancement capabilities of dimension reduction techniques is that, while the spectral components of an image are usually spatially correlated, the individual patterns (or classes, or sources) superposed onto the image are usually much less correlated. Hence, decorrelating the colour components gives a different representation, where the now orthogonal components of the image could coincide with single classes [62], [63], [64], [65], [66]. For example (Fig. 2), for palimpsests containing mixtures of two different texts and possibly further information layers (parchment texture, mould, etc.), dimensionality reduction often results in images in which each shows a single layer separated from the others. Because the statistical independence requirement of ICA is a stronger condition than the assumption of uncorrelation of PCA, it is possible that signals that are not well segmented by PCA may be separable by ICA or by ICA applied to a set of principal components, as done in [67].

Fig. 2

Folio 16v-17r of the Archimedes palimpsest. (a) RGB image under strobe lamp illumination. (b) Second component output (contrast-enhanced) from the 2 × 2 PCA of the red and blue colour channels, revealing the underwritten text and drawings (© The owner of the Archimedes Palimpsest, licensed for use under creative Commons Attribution 3.0 Unported Access Rights. Image processing: The Institute of Information Science and Technologies, National Research Council of Italy).

PCA, ICA and other orthogonalization methods, when applied to multispectral images, can increase the readability of degraded texts [68], [69], [70] or reveal hidden features not apparent in any of the individual input images, as in [66], in which a hidden text was shown to exist in a XVIII century painting, or in [71], in which the existence of many otherwise hidden details was demonstrated in wall paintings found in the Etruscan Tomb of the Monkey (Chiusi, Italy, 5th Century BCE). BSS techniques were particularly important in investigating the lost mural paintings in the Etruscan Tomb of Blue Demons (Tarquinia, Italy, 5th Century BCE), as reported in a recent paper by Adinolfi et al. [18]. In that study, a set of visible, infrared and fluorescence images was treated statistically by BSS algorithms, revealing a magnificent hunting scene with three hunters, a wild boar, a deer, a dog, and two felids, where the naked eye could perceive only a white wall (a detail of the scene, depicting the wild boar and the head of a hunter, is shown in Fig. 3).

Fig. 3

Detail of the hunting scene (a wild boar, running from right to left) recovered using BSS in the Tomb of Blue Demons in Tarquinia, using MSI and BSS. On the right, the visible image of the wall. Note the improvement in readability of the wild boar (muzzle with ear and fang is evidenced in the yellow circle) and of the head of one of the hunters (red circle) and vegetation at his right.

If the mutual independence assumption is not tenable, the ICA-based strategies for source separation may fail. One option in this case could be to rely on dependent component analysis (DCA), a class of model-based techniques that exploit other possible properties of sources or mixtures to reach their goal [72], [73]. This type of technique has been applied extensively to fields such as remote sensing [74], [75] and medical imaging [76]. Although some DCA approaches could be employed to analyse different types of cultural heritage-related images, only one proposal is present in the open literature in which a DCA approach is used for the digital restoration of colour images of double-sided documents [77].

Fisher linear discriminant analysis (LDA) can also be applied to reduce the dimensions of multispectral scans and to enhance degraded writings. Because Fisher LDA is a supervised dimension reduction tool, it is necessary to label a subset of multispectral data. To this end, in [78], a semi-automated label generation step was introduced based on an automated detection of text lines. This approach is thus based not only on spectral information, as in PCA and ICA, but also on spatial information and, when tested on two Slavonic manuscripts, has yielded better performance compared with that of unsupervised techniques.

The self-organizing maps (SOMs) method, introduced by Kohonen at the end of last century [79], represents a completely different approach to the blind separation problem. SOMs are artificial neural networks that achieve separation through the similarity of the (optical) properties of materials, which are represented in an n-dimensional space by the coordinates of the corresponding (hyper)-colours. Unlike in BSS, the number of images that can thus be extracted by a multispectral set can be greater than the number of images in the original set. No hypothesis is made on the linearity of the model, and the information layers are separated through an iterative, competitive process between the neurons that “move” in the hypercolour space arriving, after convergence, to assume the coordinates of the centroid of the corresponding cluster. This method requires the definition of a metric that determines the similarity of the hypercolours defining different materials (Euclidean, Angular, Manhattan, etc. [80]). The number of neurons is also left to the decision of the operator, based on the expected number of different materials/optical responses in the physical object [65]. An important advantage of the SOM approach is that no dimensional reduction must be performed for the classification of materials; the position of the neurons in the hyperspace represents the “prototype” of the optical properties of the corresponding material. The hypercolour associated with each pixel in the MSI image may have components corresponding to visible and infrared reflectivity, fluorescence, and elemental or molecular information. Applications of SOMs to elemental images obtained by the LIBS technique were reported by Pagnotta et al. [57], [58] (Fig. 4).

Fig. 4

SOM segmentation of a set of elemental images obtained on a Roman mortar sample using μ-LIBS [57]. The yellow square in the figure indicates the zone analysed.

In addition to BSS, SOM and LDA, non-blind spectral unmixing has proven useful in text analysis. This approach, popular in remote-sensed hyperspectral image analysis, is based on the availability of a dictionary containing the typical spectral signatures of the materials of interest and unmixing strategies such as spectral angle mapping (SAM) [81] capable of labelling the different sensed regions as belonging to specified classes [82], [83]. In document image analysis, pixel regions belonging to specific object classes, e.g., parchment, mould, overwriting, or erased text, are first identified by the user. An algorithm then computes the class membership of each pixel in the image based on the similarity of its spectrum to each of the specified classes. Although intensive both in terms of human interaction and computation time, this method was applied with success to the Archimedes Palimpsest [84], [85], [86]. Spectral unmixing for document image analysis can be particularly useful in situations in which different feature spectra are known or can be determined a priori, as in remote sensing for earth observation, where the spectra are known from field or laboratory measurements.

Pseudocolour imaging

A simple approach for enhancing hidden features in an artwork when appropriate non-visible bands are available is a rendering technique called false colour or pseudocolour. Because only three spectral bands can be displayed in a colour image, three suitable images are selected from the multispectral set and superimposed in the form of a (false) colour image. The most common combination of the multispectral images is infrared, red and green (IrRG), although the combination infrared, green and blue (IrGB) [87] is also used. The procedure implies that one of the visible colour channels is discarded (the blue band in IrRG false colour imaging or the red band in IrGB imaging), and the information it contains is not present in the pseudocolour image (Fig. 5).

Fig. 5

Schematic representation of the procedure used for building IrRG and IrGB false colour images from a multispectral set of images.

Pseudocolour imaging can be generalized in several ways. For example, to render the image data used in the study of the Archimedes Palimpsest, images captured through a blue filter under ultraviolet illumination, where the underwriting was mostly visible, and through a red filter under tungsten light, where the underwriting had nearly disappeared, were combined to render the overwriting in black and the underwriting in a reddish tint. In the resulting pseudocolour image, the two texts were then perceptually well separated because they featured highly contrasting colours, enabling the reader to distinguish between them [88].

When using data reduction methods, if layers are perfectly separated, each feature class would dominate the greyscale range in the related output channel, while pixels belonging to the other feature classes would exhibit the same grey value and thus merge with the background in that channel. More realistically, PCA or ICA may not succeed in separating features if the different feature patterns are not truly orthogonal or independent. Thus, in palimpsests, traces of overwriting usually appear in those channels in which the erased underwriting is most visible, and the variation in statistics across the scene, for example to variations in erasures, makes the erased text often appear in more than one output channel with varying intensity. This fact can, however, be exploited to generate pseudocolour rendering of extracted component images, where small variations in grey value may appear as large changes in colour, thus further improving the readability of the erased text [89].

A similar effect can be obtained by changing the pseudocolour rendering by varying the hue angle or by creating weighted combinations of images, including results from ICA and PCA and possibly original image bands.

Legnaioli et al. [71] introduced a false colour imaging technique called chromatic derivative imaging (ChromaDI), which exploits the subtraction of consecutive couples of 4 consecutive spectral images, namely, G-B, R-G and IR-R. This method was developed with the intent of building a false colour image that would take into account the information from all multispectral images acquired, without excluding a priori one of the four images in the multispectral set. The ChromaDI image provides information on the changes in reflectivity of an object with wavelength. With respect to the canonical false colour image, the differences between the optical behaviour of various pigments are enhanced, taking into account the changes occurring while passing from short wavelengths (blue band, which is more sensitive to surface details) to longer ones (green and red bands) in the visible image (see Fig. 6).

Fig. 6

Schematic representation of the procedure used to build a ChromaDI image.

ChromaDI has been successfully applied to images of a Roman painted sarcophagus, III century A.D., and to images of a mural painting of an Etruscan tomb in Chiusi (Siena, Italy), among artefacts. The method can easily be generalized to multispectral sets containing more than four images. For instance, in palimpsests, ChromaDI images could include one or more channels of UV fluorescence.

Another false colour imaging method, only experimented on paintings to date, aims at producing chromatically faithful pseudocolour images, which maintain good readability of the information contained in the infrared band. Examples of the application of this technique include the multispectral images acquired for the Pietà of Agnolo Bronzino (1569, Florence) and the analysis and visualization of the multispectral data obtained from Etruscan mural paintings (Tomb of the Monkey, Siena, Italy, V century B.C.) [90]. The method is called gradient transfer and, through a regularization strategy, merges the information from the IR band into the RGB image, preserving at best the chromatic similarity with the visible image (Fig. 7).

Fig. 7

Detail of the Pietà of Agnolo Bronzino. (a) RGB image, (b) infrared Image and (c) merged “true colour infrared image” [90].

A similar approach for image inpainting exploiting infrared information has been recently proposed by Calatroni et al. [91] for removing overpaintings in the visible image in the analysis of illuminated manuscripts and by Peng et al. [92] for mining patterns of painted cultural relics in ancient pottery and murals.

In the context of ancient manuscripts, e.g., palimpsests, the IR band could be substituted by the blue band of the UV fluorescence, where, presumably, the underwriting is best visible.

Even XRF data can take advantage of pseudocolour. For instance, in [93], a linear model was proposed to disentangle the four texts emerging from an XRF analysis of a recto-verso palimpsested manuscript. A pseudocolour rendering is then used to enhance the individual patterns in the resulting images. A nonlinear model for the superposition of texts in recto-verso scanned manuscripts [94] is envisaged to further improve the result.

Colour spaces for RGB imaging

The potential of MSI and other imaging techniques for the analysis of cultural artefacts is currently widely recognized and demonstrated. Furthermore, portable and inexpensive equipment is available. Nevertheless, the efficient use of these instruments requires specialized operators and mechanical apparatuses for the correct alignment of the camera and the artwork. Thus, in the majority of cases, simpler-to-use acquisition devices operating in the visible spectrum alone are employed, and more specialized probing techniques are limited to artworks of particular importance. In recent years, extensive digitization campaigns have been conducted in most museums, libraries and archives around the world, mainly for conservation purposes. An enormous number of digital reproductions of artworks as high-resolution RGB images are thus available. This situation poses the problem of finding fast, efficient and easily deployed image processing techniques that can meet the two requirements of being suitable for routine use and effective in helping scholars in the study and analysis of the artwork at hand.

Manuscripts often contain patterns such as underwritings, stamps, or paper watermarks that represent the most significant information from a cultural and historical point of view for establishing authorship and origin. Such marks should thus be undisclosed and enhanced. As previously mentioned, in many cases, explorations in the near-infrared or UV band can be extremely useful in this respect, as can further elaborations of the multiplicity of multispectral/hyperspectral images.

However, sometimes representing the only available RGB images in different colour spaces can be an efficient tool for “simulating” views outside the visible range, and even without introducing additional information from, for example, infrared images, the method can disentangle interesting information contained in the visible spectrum from masking interferences.

Indeed, although the RGB colour representation is the most frequently used colour space in image processing, it presents some limitations in terms of maximization of image information content. Hence, in the literature, many different colour spaces have been developed for different image analysis tasks, such as object segmentation and edge detection [95]. Some of these spaces are particularly suitable for the analysis of degraded documents, often allowing for the enhancement of document content, the improvement of text readability, and the extraction of partially hidden features.

In 1989, Xerox Corporation proposed a colour encoding called YES [96]. The YES colour space is a linear transformation of the RGB vector that matches the physiology of the human visual system. The space separates colour information and intensity information. The three coordinates are an achromatic luminance channel that is a weighted sum of the RGB values, called Y, and two opponent-colour chromatic coordinates given by spectral differences: the E channel is proportional to red minus green, while the S channel is proportional to yellow minus blue. YES has been specifically employed for the enhancement of degraded ancient parchments. When imaging the Dead Sea Scrolls, it was found that the contrast in the E map was significantly augmented, and hidden characters were revealed [16]. Other authors claim that subtracting the green component from the red, hidden characters in charred documents can be revealed, exhibiting a performance similar to that obtained using the near-infrared band [97]. A possible explanation for this behaviour is that the red channel may have recorded some infrared information, which is separated from the rest by subtracting the “red” part contained in the green channel as well [98].

The OHTA colour space was derived to approximate the PCA of RGB components [99]. The fixed coefficients of the OHTA matrix were experimentally found by a statistical study of the uncorrelated colour components in a large population of images of typical real-world scenes. The three coordinates are an achromatic luminance channel that is a homogeneous weighted sum of the RGB values, called O, and two chromatic coordinates given by spectral differences: the H channel is proportional to red minus blue, while the T channel is proportional to green minus magenta.

The YES and OHTA colour spaces and the red-minus-green and red-minus-blue operations were also useful for removing the bleed-through distortion in reddish documents [100]. The rationale for this application can be found by examining the histogram of this type of document, from which it can be observed that, in the background/bleed-through areas, red and green (or red and blue) are well separated, i.e., their difference is large. Thus, red-green/red-blue return nearly equal, high values for both the background and the bleed-through pixels such that they merge; conversely, much lower values are obtained for the text, resulting in enhancement.

Note that dimension reduction techniques, such as PCA or ICA, when applied to RGB images, can be interpreted as adaptive colour representations, in which the new colours, i.e., the components extracted, are mutually spatially uncorrelated or independent.

Modelling Multispectral/Multiview images

The use of statistical processing techniques to elaborate the multispectral/hyperspectral images of an artwork, with the aim of separating the various layers of information that it contains, implicitly assumes a linear, instantaneous data model. In other words, all available views of an artwork are considered linear combinations of a number of patterns. The recovery of individual patterns thus amounts to inverting this transformation. However, because the coefficients of the transformation are not known, a priori assumptions about the patterns must be exploited. Applying the various PCA and ICA operators corresponds to assuming mutual uncorrelation (or independence) between the patterns [62], [63], [101].

This basic linear instantaneous mixing model can be extended to account for nonlinearity, spatial non-stationarity, convolutional mixing, noise, etc., to better adhere to the physical characteristics of specific instances of pattern superposition in artworks. For example, some of the abovementioned variants have been explored to model the phenomenon of text overlap in recto-verso manuscripts affected by show-through or bleed-through distortion, with the aim of correcting the distortion. In such cases, solutions exploiting nonlinear ICA, non-negative matrix factorization, variational approaches, regularization, dependent component analysis, or other ad hoc strategies have been proposed [77], [94], [110], [111], [112], [102], [103], [104], [105], [106], [107], [108], [109].

Conclusions and future perspectives

In this paper, experimental methods and analytical techniques that can help in recovering hidden details in cultural heritage artefacts are presented and discussed. These methods are particularly suited for the analysis of degraded texts, palimpsests and paintings but can also be applied, for example, to the study of geological materials, pottery and mortars.

Regardless of the experimental technique used, if a representative set of images can be obtained, processing methods can be applied to treat these images and extract meaningful information. Blind source separation techniques, self-organizing maps, and linear discriminant analysis provide statistical algorithms that can reveal hidden features that, although present in the input set, might not be observable in the individual channel images. These techniques can also be applied to simple RGB images, possibly with the help of freely available software, such as the D-stretch ImageJ plugin [113]. Once the image set is obtained, pseudocolour images can be obtained or, using new techniques based on the gradient transfer method, even colour faithful images, embedding otherwise invisible information, can be obtained. 3D multispectral models can also be recovered using digital photogrammetry. Many examples of the application of the above described techniques in restoration, archiving and documentation processes can already be found in recent literature [114], [115], [116], [117], [118].

With the progress of instrumentation (improved CCD cameras, illuminators, and non-optical imaging systems such as micro or macro XRF/LIBS elemental imaging, Raman molecular imaging, etc.) and the introduction of simpler, faster and more performant statistical algorithms for the treatment of large image sets, it is reasonable to expect that in the near future multispectral imaging and the related techniques described here will likely replace colour digital photography for quick and information-rich documentation and study of cultural heritage.

Conflict of interest

The authors have declared no conflict of interest

Compliance with Ethics Requirements

This article does not contain any studies with human or animal subjects.