CHAPTER 4

Spot Integration

The last two chapters have dealt with diffraction geometry almost exclusively. Given a prediction of the location of each reflection as precise as it can be on detector surface, the procedure of spot integration is very much independent of diffraction geometry. This procedure is responsible for finding diffraction power of each reflection. Three major issues are critical to the accuracy, completeness, and speed of integration. First, an accurate evaluation of a local background scattering is needed. Background scattering is modeled by a tilting plane for each spot in Precognition. Second, closely spaced spots need to be resolved in order to split their diffraction powers. Third, a variety of irregular spot shapes shall be modeled to obtain accurate summation of intensity. Precognition offers a spectrum of integration modes to selectively perform these functions at different costs of time.

4.1 General Issues for Integration

A command script for integration is shown below.

diagnostic off

busy off

warning off

@ m37v3_8us_002.mar3450.inp

@ m37v3_8us_004.mar3450.inp

…

@ m37v3_8us_062.mar3450.inp

Input

Image lambda.lam

Spot 20 6

Quit

Dataset hybrid

Resolution 1.6 100

Wavelength 0.98 1.6

Quit

Stop

Yes

Listing 4.1.0.0.1 Command script for spot integration.

First, a list of input files for all images to be integrated is loaded in. On a dual-processor machine, split this list into two half and make both processors work. The Input section specifies a l-curve and an average spot length and width in pixel. The .lam file is a two-column ASCII file with wavelength in Å in the first column, and relative intensity in the second. If the spectrum of your beamline is unknown, a very rough box is sufficient.

0.7 0

1.0 1

1.2 1

1.6 0

Listing 4.1.0.0.2 A rough l-curve indicating the peak of the spectrum between 1 and 1.2 Å.

The Spot command in Input section may also accept a third number as a s-cut. In case of integration, the s-cut plays a role in selection of sample profiles. A low level of s-cut may allow many weak spots to contribute to profiling, therefore the quality of profile may be sacrificed. A large s-cut value may results in insufficient number of sample profiles, and prolonged execution time. The default s-cut is 3, which is good for most cases. You may adjust this value according to your image quality, and check the saved .spt file for evaluation. Obviously, s-cut affects those integration modes involving profile fitting only. See 4.5 through 4.7 for details of those integration modes.

The Dataset command discussed in the previous chapter can also accept integration modes. Once again, this command enters into a submenu. If an integration mode is given, resolution and wavelength ranges can be given here in addition to directory names. Despite what ranges is set in Input section, these ranges in Dataset section will determine the integration ranges. If no new ranges are given, the old ranges set in Input section will take effect. The directories where images should be loaded and where results should be saved can be specified in the same manner as described in the last chapter (3.3). The result of integration is a set of .ii (integrated intensities) files to be loaded by the scaling program Epinorm (Chapter 5). The command Dataset with an integration mode may also accept an optional numerical argument that specifies a starting ‘pattern number’ of LaueView. .sht files will also be saved, so that scaling can be performed by LaueView. By default, .sht files will not be generated starting from release 4.1.0.

An integrated intensity file is a point of entry to the system of Precognition. It is a free format ASCII file with arithmetic values only. Each record has a same format and represents an observed reflection. Harmonic or spatially overlapping reflections must be arranged consecutively. General users shall avoid any modification to these files.

The first three columns are Miller indices h, k, and l. The forth integer is multiplicity of harmonics. The fifth and sixth columns are center coordinates of the reflection spot on detector in pixel values. See 2.1.3 and 2.2.1 for definition of pixel coordinates. The content of these two columns may be changed in future releases. Column 7 and 8 are resolution and wavelength of the reflection in Å, respectively. The last two columns are integrated intensity and its standard deviation error in pixel² ´ detector unit, respectively.

…

5 7 -22 1 2076.7 2427.9 3.0961 1.257307463 328.44 54.93

6 8 -27 1 2050.0 2431.1 2.5606 1.029589599 24.93 123.97

4 4 -20 2 1923.3 2422.6 3.6535 1.380610908 16129.92 121.86

5 5 -25 2 1923.3 2422.6 2.9228 1.104488727 16129.92 121.86

6 6 -28 1 1956.5 2463.7 2.6009 1.040187484 4279.40 99.42

5 5 -23 1 1964.0 2472.5 3.1638 1.280254859 1055.61 93.11

4 4 -21 1 1902.4 2394.7 3.4859 1.265264294 8475.77 144.94

…

Listing 4.1.0.0.3 A section of integrated intensity file.

Algorithms of spot integration can be divided into two distinctive categories: summation and profile fitting. Summation methods are certainly faster. They are also very good for evaluation of strong spots. But summation methods have no ability of identifying and filtering noise, which make their results from weak spots poor. They also experience great difficulty when spots are spatially overlapping. Profile fitting methods are learning processes. They make connection between strong and weak reflections by learning profile from strong ones and applying profile to weak ones. Such process identifies and filters noise, so that they are in general better methods for evaluation of weak spots. Knowing spot profile also greatly facilitates deconvolution of spatially overlapping spots. However, noise filtering can be a risky business. This is because that the base of profile learning can be too wide or too narrow, as we discussed above. When a learned profile does not fit an individual spots very well, its noise filtering and therefore its integration become biased. This happens in more significant way to stronger spots. Between summation and profile fitting, and more toward the latter, numerical profile has the advantage of profile fitting, but it is not so powerful in terms of noise filtering, since it inherits some numerical features of summation methods.

Precognition offers the full spectrum of integration algorithms. More importantly, each implementation tries to extend its advantage and to eliminate its shortcomings as much as possible. For example, some summation methods may do its best to isolate two closely spaced spots. The analytical profile fitting methods are combined with numerical compensation. All these efforts aim at evaluation of diffraction power of as many reflections as accurate as possible.

4.2 Integration On-the-fly, box Mode

A fast integration mode named box is available for quick checking of data quality while data are being collected. The algorithm used here is the fairly common square box method. A circular partition inside the box is designated to be the peak area. The size of the square box is specified by the average length in Input section. The limitation of this algorithm is rather severe when used for integration of Laue spots, since no resolution of spatial overlap is attempted and background is only modeled by a non-tilting level. Many weak spots are not observable by this method.

4.3 Elliptical Peak Area, fixedElliptical Mode

A better method called fixedElliptical is to use an elliptical peak area in integration, since Laue spots are often streaking in radial directions. A rectangle box of size specified in Input section is reoriented for each spot being integrated. An elliptical peak area is partitioned in each rectangle box. The size of the box is specified by user. A typical size shall be used, that is, the input size shall represent the greatest population of spots. This algorithm can also resolve some spatial overlaps as long as no pixel is involved in more than two peak areas. Thirdly, background near each spot is modeled by a tilting plane, which makes summation of peak much more accurate.

4.4 Learned Elliptical Peak Area, variableElliptical Mode

This integration mode called variableElliptical offers another level of automation. The integration box for each image is determined by the program itself. The user input is only an initial hint. The major problem of fixedElliptical mode is therefore solved. This mode is more necessary when spot streakiness varies significantly in a data set.

4.5 Numerical Profile, numeric Mode

Spot profile is a typical example of pattern recognition. Weaker spots are recognized by learned profile from stronger ones. This integration mode numeric learns an averaged numerical spot profile within a small area on detector surface from some strong and well separated spots, which are sometimes called samples. See above for selection of sample profiles. This profile is then applied to all predicted spots. This algorithm makes resolution of spatial-overlaps possible. A lot of weak spots also become observable.

4.6 Analytical Profile, linearAnalytical Mode

Ren et al. (J. Appl. Cryst. 28, 461-81, 1995) have shown previously that a multi-parameter analytical function can be used as spot profile for Laue diffraction. This analytical profile has its flexibility and stability, and therefore advantage over a numerical profile. Even more weak spots can be integrated by analytical profile than by numerical one. This is because analytical profile naturally suppresses additional noise contained in numerical profile. The current analytical profile mode is called linearAnalytical since spatial overlap resolution is carried out by a linear model. This mode is the most similar one to the algorithm used in LaueView, and so far remains the best integration mode. The linear model of spatial overlap does very good job isolating spots from each other, it they are reasonably spaced. See Figure 4.6.0.0.1 for an example. However, deconvolution of spatial overlaps by profile fitting will eventually reach a limit when spots are getting extremely close to each other respect to their own size and the amplitude of prediction error. A second attempt to deconvolute those extremely-close spatial overlaps is implemented with the same strategy of harmonic deconvolution, that is, to take advantage of redundant and symmetry-related measurements. See 5.4 through 5.6 for detail.

Figure 4.6.0.0.1 Three spatially-overlapping spots fitted by analytical profiles. One strong spot and one weak spot are obvious. A third spot between the other two is shown as a bulge on the contour plot and a shoulder of the main peak on the surface plot. The surface plot is on a logarithmic scale. The colors show residual of fitting.

4.7 Simultaneous Profile Fitting and Spatial-overlap Deconvolution, nonlinearAnalytical Mode

This mode of integration pushes the envelope to an extreme. In case of large unit cell, it may be difficult to find enough well separated spots for profile fitting. Or in case of small unit cell, it may also be difficult to find enough sample spots, since there are not many spots to start with. This may be more true in some area of a frame than in others, for example, where major ellipses are passing through. As described above, the last mode linearAnalytical performs profile fitting only on selected samples and deconvolutes spatial overlaps when applying the previously obtained profile. This integration mode merges these two procedures, so that the profile is being constantly adjusted. Figure 4.7.0.0.1 gives one example that nonlinearAnalytical mode does a better job to minimize prediction error than linearAnalytical mode does. This mode in the current release has not yet been fully tested.

Figure 4.7.0.0.1 Profile fitting of one spot by linearAnalytical (left) and nonlinearAnalytical (right) modes. Color shows the residual of fit on each pixel. The left penal shows positive errors on one side of the peak and negative on the other, which indicates that positional error is not completely corrected. The right penal shows slightly different position and shape of the profile, and random error distribution.

4.8 Numerical Compensation to Analytical Profiles

Although profile fitting is so powerful for integration of weak spots, no single profile, numerical or analytical, can fit all spots at a same time. It is more obvious that strong spots are harder to be fitted perfectly. An algorithm of numerical compensation corrects such misfit to a large extent, while keeps loyalty to the analytical profile found. Both analytical modes linearAnalytical and nonlinearAnalytical feature this compensation.

Figure 4.8.0.0.1 The contour view of the same three spots shown in Figure 4.6.0.0.1. The main peak height is over 1000 detector counts. The color-coded fitting residual ranges ±2% after numerical compensation.

4.9 hybrid Mode

This integration mode hybrid is trying to find an optimal balance between summation and profile methods. This mode is a hybrid between variableElliptical, the best summation method and linearAnalytical, the best profile fitting method. This mode may be changed in future release.