The modular small-angle X-ray scattering data correction sequence.

Data correction is probably the least favourite activity amongst users experimenting with small-angle X-ray scattering: if it is not done sufficiently well, this may become evident only during the data analysis stage, necessitating the repetition of the data corrections from scratch. A recommended comprehensive sequence of elementary data correction steps is presented here to alleviate the difficulties associated with data correction, both in the laboratory and at the synchrotron. When applied in the proposed order to the raw signals, the resulting absolute scattering cross section will provide a high degree of accuracy for a very wide range of samples, with its values accompanied by uncertainty estimates. The method can be applied without modification to any pinhole-collimated instruments with photon-counting direct-detection area detectors.

Introduction

Attaining a high standard for data quality is paramount for any insightful analysis. This is particularly important for small-angle scattering, where the largely featureless scattering patterns may easily be insufficiently corrected and/or over- or under-fitted by an inexperienced user. Previous work on data correction procedures tended to follow an ad hoc approach, incorporating a limited incomplete subset of the available corrections in an integrated correction step. Such methods offer neither flexibility nor the ability to trace the effects of a given correction step (Stothart, 1987 ▸; Strunz et al., 2000 ▸; Dreiss et al., 2006 ▸).

This can be resolved with the provision of a comprehensive modular set of elementary two-dimensional data correction steps. These steps can be chained together to form a bespoke and complete correction sequence for a given instrument or sample in any small-angle scattering laboratory or synchrotron. When enhanced with the ability to estimate and propagate well founded uncertainty estimates1 on (at least) the resulting scattering cross-section values, it allows said laboratory to rapidly evaluate and select the subset of significant corrections for their experiments or instruments. The two-dimensional nature of the corrections renders them appropriate for isotropic as well as anisotropically scattering samples, increasing their universality.

While the majority of these individual data correction steps have already been comprehensively collated for experiments using both X-rays (Pauw, 2013 ▸; Feigin et al., 1987 ▸) and neutrons (Hollamby, 2013 ▸; Brûlet et al., 2007 ▸), the recommended order in which they are to be applied has so far not been published. For X-ray experiments in particular, this was due to a historical lack of software that might benefit from such a scheme, and because such a sequence had not been finalized until now. With the recent emergence of various two-dimensional data correction software packages capable of performing such comprehensive modular corrections (Arnold et al., 2014 ▸; Basham et al., 2015 ▸; Benecke et al., 2014 ▸; Filik et al., 2017 ▸; Girardot et al., 2017 ▸; Nielsen et al., 2009 ▸; Taché et al., 2017 ▸; Solé et al., 2017 ▸), establishing a recommended starting point for implementing such a data correction schema seems pertinent.

This work, therefore, provides a near universally applicable, ordered schema of corrections with which the absolute scattering cross section can be extracted from raw detector information in a consistent and reproducible manner. This scattering cross section is determined multidimensionally (typically two dimensionally) and can optionally be reduced to one dimension, i.e. where the averaged scattering cross section is determined as a function of scattering vector Q or azimuthal angle χ. This result can then be fed to a wide range of data analysis programs for morphological elucidation, such as ATSAS, GIFT, Irena, McSAS, SASfit and SASView, to name but a few (Franke et al., 2017 ▸; Glatter, 1980 ▸; Ilavsky & Jemian, 2009 ▸; Bressler et al., 2015 ▸; Breßler et al., 2015 ▸; Alina et al., 2017 ▸). This schema can be used as the core of a data correction software package or as a reference correction sequence against which (faster) alternatives can be proven. It is hoped that adherence to this schema will improve the already exemplary comparability of results obtained at different instruments (Pauw et al., 2017 ▸; Rennie et al., 2013 ▸).

The near universality of the scheme implies that it is applicable to most sample types (cf. Table 1 ▸). This property stems from its three-stage correction process: The first two stages of the correction process are sufficient to correct homogeneous samples (single phase), and the addition of the third stage means it can also extract a particular scattering signal from both dilute and dense heterogeneous samples. Examples of such investigations include size and shape investigations of colloidal dispersions (Wagner et al., 2000 ▸), the physical state of self-assembled systems (Soni et al., 2006 ▸), the elucidation of structural and orientational details from polymers (Heeley et al., 2005 ▸), and protein sizing in solution (Rambo & Tainer, 2013 ▸). Additionally it is possible to use this schema to extract the scattering of dynamic systems such as evaporative drying (Gu et al., 2016 ▸), in situ chemical reactions (Chen et al., 2015 ▸), phase or state changes experienced by soft-matter systems with respect to change from external influences and changes (Bulpett et al., 2015 ▸), and deterioration in systems exposed to stress, strain, wear or age (Turunen et al., 2016 ▸).

Table 1

Examples of measurements to be used for the various processes, for a range of sample types

‘NIB’ stands for nothing in beam. This means that there is nothing in the beam path at the sample position; the normal flight-tube exit and entrance windows, for example, are kept in place. ‘Empty cell’ can be replaced with ‘empty capillary’ if capillaries are used. For sooty flames, the non-sooty flame is a best effort, since the burning conditions, and therefore the background, are by definition different.

To determine the scattering of	Process A	Process B	Process C
Solids
Freestanding solid (slab, plate or foil)	NIB	Solid	N/A
Metal alloy	NIB	Alloy	N/A
Nanoparticles embedded in a polymer	NIB	Polymer	Polymer with embedded nanoparticles
Porous membrane in vacuum (dry)	NIB	Membrane	N/A
Only pores in the membrane (dry)	NIB	Non-pore-containing membrane	Dried, porous membrane
Porous membrane in in situ cell (gas/wet)	Empty cell	Filled cell	Immersed membrane
Only pores in the membrane (gas/wet)	Empty cell	Immersed non-pore-containing membrane	Porous membrane
Powders
Powder in between sticky tape (dry)	Sticky tape	Powder in sticky tape	N/A
Powder in in situ cell (gas/wet)	Empty cell	Gas- or water-filled cell	Immersed powder in cell
Liquids
Pure liquids	Empty cell	Liquid	N/A
Nanoparticle dispersion in liquid	Empty cell	Solvent	Solvent + nanoparticles
Proteins in buffer	Empty cell	Buffer	Buffer + protein
Micelles in oil/water dispersion	Empty cell	Oil; water (separately)	Micelles in dispersion
Gases
Pure gases	NIB or empty cell	Gas	N/A
Particles in carrier gas (flow-through cell)	Empty cell	Gas	Gas + particles
Particles in carrier gas (free flowing)	NIB	Gas	Gas + particles
Sooty flames (see note in caption)	NIB or empty cell	Non-sooty flame	Sooty flame

Over the past few years, most elements of the schema have been developed, tested and refined in practice, on both laboratory- and synchrotron-based small-angle X-ray scattering (SAXS) instruments. Its development has focused on modern instruments, and a direct-detection photon-counting detector is, therefore, highly recommended in order to achieve the best results. The use of such a photon-counting detector in the following is implicit: the data correction steps necessary to compensate for the inadequacies of other detector types have been omitted for brevity.

We here present the ordered schema, each correction’s abbreviation is briefly described, and the reasoning behind their chosen position in the sequence is clarified.

The schema

The recommended data correction schema covering a wide range of practical samples is presented in Fig. 1 ▸. This data correction schema consists of three different series of corrections, denoted as ‘Process A’, ‘Process B’ and ‘Process C’. The measurements to be used in a particular process depend on the object of interest, with many examples given in Table 1 ▸. In general, Process A should be used on a measurement of the instrumental background (including empty sample cells when sample cells are used). Process B should be applied to (1) measurement of a non-dispersed material of interest or (2) a dispersant, be it either solid, liquid or gas. In the second case, Process C is used for the measurement of the dispersant with the analyte. The output from Process C is then the absolute scattering power of the analyte alone, and the output from Process B is the absolute scattering power of the dispersant.

Figure 1

The recommended data correction sequence for most types of samples. Output B for solids is the corrected data in absolute units, and for dispersions it is the dispersant (solvent) scattering in absolute units. Output C for dispersions is the sample scattering in absolute units. The azimuthal averaging step can be considered for isotropically scattering samples.

When applied to homogeneous polymer films, for example, the instrumental background measurement (measured with nothing at the sample position) is inserted in Process A and the polymer film measurement in Process B. From the output of that process, ‘Output B’, we then obtain the polymer film scattering cross section in absolute units. If, however, determination of the scattering arising from nanoparticles embedded within that polymer film were desired, then it would be necessary to collect a measurement of the non-nanoparticle-containing film for Process B and the nanoparticle-containing film in Process C. Output C will then be the scattering power of the embedded nanoparticles alone.

Likewise, for a dispersion of nanoparticles in a solvent contained in a capillary, measurement of the empty capillary would be used in Process A, the solvent in capillary in Process B and the dispersion in capillary in Process C. This leads to the acquisition of the absolute scattering cross section for the solvent in Output B and that of the pure analyte in Output C. The advantage of this approach is that the solvent scattering may be reused for other samples measured using the same energy by adding this to a solvent scattering library. A disadvantage of this approach is that the uncertainties of several operations are added twice to the scattering cross section for dispersions (Output C), such as the uncertainties on the flatfield and polarization corrections, which will be discussed below.

Note that the same capillary should be used for both Process A and B, when obtaining the dispersant scattering cross section. Likewise, the capillary used for the set of Processes A and C, i.e. when collecting the measurement of the dispersant with analyte, should also be identical. These conditions, therefore, necessitate the use of reusable containers or flow-through cells.

The steps and reasoning behind the sequence

The mathematical expressions for most of the corrections below are described by Pauw (2013 ▸), with the remainder given in the appendices. Here, we focus on the justification of the steps and highlight the position dependency of some of them.

(i) DS (data read-in): before starting any data corrections, the data must be read in correctly, where necessary compensating for the data storage peculiarities (Knudsen et al., 2013 ▸).

(ii) MK (masking): invalid pixels are masked so they are not considered in the following corrections.

(iii) PU (Poisson uncertainty estimator): the Poisson (counting) uncertainty needs to be calculated on the number of detected photons, and therefore this step is carried out before the deadtime, dark current or flatfield corrections. Some detectors automatically apply corrections and will require (software) adjustments before the uncorrected counts can be retrieved.2

(iv) DT (deadtime): the signal is subsequently corrected for the detector deadtime, returning the estimated number of photons arriving at each pixel on the basis of the detected count rate.

(v) DC (dark current): the subtraction of natural background radiation (including the steady flow of cosmic rays) forms the dominant component of the dark current correction. With the aforementioned recommended detector type, we should not see any significant contribution of the time-independent and flux-dependent dark current components.

(vi) TI (time): a normalization to render the measurement independent of the measurement duration.

(vii) FL (flux): a normalization to make the measurement independent of the incident beam flux.

(viii) TR (transmission): a scaling correction, correcting for the probability of absorption (and only absorption) within the sample. The transmission should, ideally, be calculated by dividing the flux of all transmitted, scattered and diffracted radiation by the incident flux. Note that the quality of the obtained scattering cross section is very strongly dependent on the quality of the transmission factor (in particular when the background subtraction operation is applied), and an accuracy of >99% should be aimed for.

(ix) SA (self-absorption): the sample self-absorption is the correction for the increased probability of scattered rays that are absorbed as they travel through slightly increased amounts of sample after the scattering event. This correction needs to be performed after the transmission correction: it represents a direction-dependent modification to the transmission correction and does not replace the TR correction itself. It is feasible to implement and use for samples of plate-like geometry (only), with the plate surface perpendicular to the X-ray beam direction. It is related and therefore placed next to the transmission correction.

(x) FA (frame averaging): it is recommended for photon-counting direct-detection systems that the measurements be split up into multiple shorter frames. This avoids saturation of the per-pixel counters and allows for temporal variations in signal due to beam, instrument or sample instabilities to be recognized. When no significant variation between the frames is observed, they can be averaged in this step. When the averaging is, furthermore, weighted by the scattering signal uncertainty of each pixel in each frame, frames collected with different exposure times can be averaged to obtain a high dynamic range (HDR) scattering signal. Any saturated pixels for a long-exposure frame will then have been masked by the MK process.

(xi) BG (background subtraction): the subtraction of the background signal is done only after the measurement-dependent corrections have taken place, as the various parameters (transmission, flux, time and therefore dark current in particular) may differ.

(xii) FF (flatfield): the flatfield correction, a multiplication matrix normalized to 1, corrects for inter-pixel sensitivity differences. An example of its magnitude in modern detectors is given by Wernecke et al. (2014 ▸).

(xiii) AE (angular efficiency): this correction compensates for variations in the detector efficiency depending on the photon angle of incidence onto the detector surface. It is detailed in Appendix C . This is the last of the corrections for detector imperfections.

(xiv) SP (solid angle): a (geometric) correction for the solid angle subtended by each pixel. This can be calculated on the basis of the instrument geometry alone.

(xv) PO (polarization): the polarization correction com­pensates for differences in the probability of scattering events, both for polarized and for unpolarized beams. In the latter case, it is an azimuthally uniform (isotropic) correction. The polarization correction is performed before the second background subtraction, so that older dispersant measurements can still be used for correction of future samples.

(xvi) TH (thickness): the thickness correction normalizes the data to units of reciprocal length. Note that the thickness used in this correction is the thickness of the solid sample or the liquid phase for dispersions only. A derivation supporting this is provided in Appendix D .

(xvii) AU (absolute units): the absolute units correction scales the data to units of scattering cross section, the fraction of radiation that is scattered per length of material per solid angle. This is commonly reported in units of or .

(xviii) DV (displaced volume): this correction has not been included by Pauw (2013 ▸), but is described in Appendix B . This correction can be applied for dispersions with high volume fractions of analyte, but must be done on the solvent scattering signal only.

(xix) RM (remapping): for background subtraction operations using a previously stored dataset, a mapping or interpolation operation may be necessary to match the dimensionality and angular range of the dataset in the processing step.

(xx) AV (averaging): this optional step reduces the dimensionality and size of the dataset, typically from two dimensions to a limited number of data points in one dimension. This can be done azimuthally (to obtain versus Q) or radially ( versus χ). The azimuthal averaging is suitable for isotropic data, whereas the radial averaging is typically applied to anisotropic data over a limited radial range, to extract a degree of orientation. This is commonly performed in fibre diffraction experiments.

Note that the averaging from two dimensions to one dimension is performed after the background subtraction step as (1) it is optional and (2) the background subtraction process in particular can subtract anisotropic signals such as flares. In that case, the practical uncertainty estimated during the averaging procedure is reduced if the background subtraction is done in two dimensions rather than after averaging, as shown in Appendix A . Note also that the solvent background subtraction in Process C can be performed either directly with two-dimensional data from Process B or using a stored solvent scattering signal from the absolute backgrounds library. In the latter case, the data may have been stored in azimuthally averaged form (one dimensional), in which case they will need to be mapped (interpolated) to two dimensions to match the dataset dimensionality and angular range of the data processed in Process C.

Many of the corrections are multiplications and therefore follow the law of commutation. The corrections that can be commutated have been grouped together (cf. Fig. 1 ▸), where such a commutation would not affect the result. The commutability becomes clear when we write Process B as a pseudo-equation, with a indicating a more involved operation, a indicating a subtraction, and a indicating a multiplication operation with either a scalar or a vector:

Regarding uncertainties

In the modular approach presented here, it is pragmatically assumed that the correction steps and their uncertainty propagation are uncorrelated to, and independent of, any preceding or subsequent correction steps. This assumption makes the uncertainty propagation easy to implement, as each correction step can remain unaware of its position within the complete scheme. However, this assumption of uncorrelated independent uncertainties for each step is not strictly true for those correction steps that are performed in each of the processes but use the same correction factors (as is the case for FF, PO, SA and AE, for example). Their multiple occurrence within the general scheme may lead to an overestimation of the propagated uncertainty (to which the uncertainty of each of these is added several times). The propagated uncertainty estimates in the final output may, therefore, be overestimated owing to the multiple occurrences of these corrections in the schema. The magnitude of this overestimation of uncertainties can be reduced by avoiding the total number of such duplicate correction steps, by considering the ‘merging points’ in the data correction sequence.

In the data correction sequence, merging points exist at the container and solvent scattering subtraction operations, where the uncertainties of the preceding two sequences are combined into the uncertainty of the resulting data values. Operations that can be deferred beyond these merging points are performed only once on the data as opposed to twice when placed before the merging points (i.e. where they are applied once per dataset), and therefore the total resulting uncertainty estimate will be smaller by avoiding the duplicate operations.3

The corrections in this sequence have thus been ordered to limit the number of duplicate operations, while retaining the usefulness of the results. In particular the choice was made to obtain both the solvent scattering (output B) and the analyte scattering (output C) in a reusable form at the cost of retaining some duplicate operations, whereas the scattering signal from the container itself was not considered important enough to obtain in reusable form. This has the added benefit that the correction values, for e.g. the flatfield, do not need to be the same between processes A + B and A + C. Pragmatic choices have been made for corrections which do not benefit from an early placement in the schema but can be deferred until after a merging point, saving time and limiting the uncertainty expansion. The authors believe that the simplicity of implementation of this approximate error estimate propagation method outweighs the drawbacks of the potentially increased uncertainty estimate. The increases can also be limited by accurate determination of the correction values, so that the correction value uncertainties are small and do not contribute significantly. Finally, as multiple uncertainty estimators are provided, including one determined during the final azimuthal integration step, the user remains free to choose the final estimator (or estimator combination) that they deem to be the most accurate reflection of the practical uncertainty for their instrument. The provided propagated uncertainties can only improve with better, more involved uncertainty propagation considerations in the future.

A further practical modification

In practice, the flux and transmission corrections can be combined. We define the transmission factor , with the incident flux denoted as and the emergent flux (the sum of the transmitted, scattered and diffracted radiation) as . Then, defining the prior detected signal and flux- and transmission-corrected signal , we get Combining these operations ostensibly negates the need for an upstream beam flux monitor, to the great relief of many instrument scientists. However, as the transmission factor still needs to be known for the self-absorption correction, their elation is likely to be short in duration.

Instrumental effects for consideration in the analysis rather than the data corrections

There are some effects which are, unfortunately, best considered in the scattering pattern analysis procedure rather than in the data correction procedure. There are three effects: the resolution function smearing, the multiple scattering effect and the scattering length density contrast. We will discuss each of these briefly.

The resolution effect originates from the uncertainty in the scattering vector for each individual photon. Some of the origins of these scattering vector uncertainties are well defined, such as finite beam size and divergence, and the scattering vectors for an ensemble of photons will, therefore, exhibit a well defined spread. This is known as the resolution function, and it can, in principle, be corrected for. The procedure to do this can be likened to a ‘sharpening’ procedure in image processing and carries the risk of introducing artefacts due to its ill-posed nature. As it is more prominent in the neutron scattering field, a workable solution has already been developed there: the mathematically safer method for including the resolution contribution is to include the resolution function in the analysis. For this reason, the resolution function should accompany the data, for example using the provisions in the NXcanSAS format. By then convoluting or ‘smearing’ the model scattering with the resolution function, the problem is tractable and can be taken into account without reservation (Rennie et al., 2013 ▸).

The same holds for the multiple scattering contribution (Warren & Mozzi, 1966 ▸). This is the probability that photons are scattered twice or multiple times, and is directly related to the scattering probability of a material for the energy used and its thickness. The multiple scattering contribution is hard to correct for in the original data. It is much easier to convolute the scattering pattern with the multiple scattering effect and likelihood, and to take it into account in that manner (Rennie et al., 2013 ▸).

The last effect is the energy dependence of the scattering length density contrast. This energy dependence implies that, while the scattering vector is described independent of the energy, the scattering cross section will still be correlated, particularly near to absorption edges. There is, to our knowledge, no current solution for this, and information on the used energy must, therefore, always accompany a scattering curve.

Conclusions

We have presented a comprehensive data correction sequence, which can be used as the core of a software implementation or as a reference correction sequence against which other, faster implementations can be proven. The sequence is chosen so that it returns useful information, in particular for dispersions, where the absolute scattering signal from the dispersant and that from the analyte are obtained independently.

By presenting this schema, we hope to encourage unity and consistency in the worldwide data correction efforts, to the betterment of the small-angle X-ray scattering field.