CHAPTER
2
Indexing of Laue Pattern
The very first release of Precognition was dedicated to a more robust auto-indexing algorithm of Laue diffraction patterns. Compared to LaueView, a recent comprehensive test shows that over 90% of sparse images in a data set from a small molecule crystal can be independently indexed and refined with absolutely no user intervention; another 5% can be indexed and refined with some user assistance; while none of the images in the same data set was successfully indexed by myself and others using LaueView after days of trials, due to insufficient number of good nodal spots identified by eyes. The key improvement is automatic pattern recognition of diffraction spots, their common profile, direct-beam center, Laue ellipses, and significant nodal spots.
This chapter explains the usage of this indexing program, and leaves the underlying methodology elsewhere. This chapter is a starting point for a new user, since it describes many basic input and control parameters not only for indexing but also for the other tasks of the system.
2.1
Required Input
Input is a main level command to the program Precognition (Listing 1.5.3.0.1). It enters into a submenu that takes all input from files or from keyboard. In order to index a Laue pattern, the following input is required:
Input
Crystal /home/renz/xtal_info/bm.xtl
Distance 40.9 0.1
Center 1087 1368 1
Pixel 0.0792 0.0792
Format MarCCD
Image
/home/renz/work/bm1/images/BM1_1_100.img
Resolution 0.7 100
Wavelength 0.7 1.3
Quit
Listing 2.1.0.0.1 Required input.
2.1.1 Crystal information
cpmv
317 317 317 90 90 90
spgp i23 197 24 12 cubic
Listing
2.1.1.0.1
The first line of the crystal information file must only contain a single string without space. However, underscore (_) can be used, such as, cowpea_mosaic_virus.
The command
There is so far no effort made by us to index a Laue pattern without knowing cell constants, however, determination of cell constants and assignment of space group may be added in later release.
2.1.2 Crystal-to-detector distance
Distance takes one numerical argument in mm as the crystal-to-detector distance and another optional value as its uncertainty. The default uncertainty of distance is 1 mm. Error in distance contributes to the error in calculated angular parameters of diffraction rays, also known as reflected beams. At a Bragg angle of 22.5¡ã, this contribution is just as large as that from the uncertainty of a spot location on detector surface. Normally, the uncertainty of distance shall be smaller than 1 mm to avoid mis-indexing (see 2.5). The uncertainty value will be used in geometry refinement (See Chapter 3).
2.1.3 Direct-beam center
Center takes two numerical arguments in pixel that mark the location of direct-beam center. The first number measures from the left edge of an image to the center when viewing along the X-ray beam, and the second does from the top edge. See Figure 2.1.3.0.1 for detail. The command may also take one or two more optional values as the uncertainty of the beam center. If only one additional value is given, it is assumed that the uncertainty of beam center is isotropic. If two are given, they are the uncertainties of both coordinates, respectively. The default values are 1 pixel. Error in center affects the accuracy of diffraction ray at a much greater extent than error in distance does. This is the primary reason for the cause of failed or mis-indexing. Precognition improves the tolerance of error in center position compared to other programs, since center is automatically refined during the process of ellipse refinement. However, it is recommended to control the error to an extent as small as possible and no greater than half of the narrow dimension of a typical Laue spot on the image. For example, if a typical spot is 10-pixel long and 6-pixel wide, the tolerance of uncertainty in direct-beam center is not much greater than 3 pixels or 300 mm. However, you may recycle the refined values and rerun the program, if indexing fails. Future releases may include automatic cycling and a GUI that will accept visual assistance from user.
Figure 2.1.3.0.1 Coordinate system of an image file. Each square is a pixel. The top leftmost corner of an image has coordinates of (0.5, 0.5). The top leftmost pixel centers at (1, 1). The origin is outside of the image.
Direct-beam center as a required input is not general enough for abnormal geometry. The command Center may be replaced by others in future release.
2.1.4 Pixel size
Pixel takes two numerical arguments in mm that measure the size of an averaged raster in horizontal and vertical direction, respectively. These numbers usually come from your detector manufacture or your own calibration experiment. Error in pixel size is directly related to error in distance. Both error sources affect the angular parameters of the diffraction ray. The command may take one or two more additional numbers as uncertainties, but they are rarely set by users. The default value for horizontal pixel size is 1 mm, and that in vertical direction is 0.
2.1.5 Image format
Format is a command at Input level. MarCCD is the code for supported detector format. This code is case-sensitive. The supported detector/image formats are listed in Table 11.1.1.1.1. This list will grow if users request other formats to be supported. Check the updated reference (11.1.1.1) for the latest list of format codes.
2.1.6 Image file
Image expects a string (do not quote) that specifies an image filename with full path or relative path. If a relative path is given, it is relative to where the program was launched. Obviously, a given image must be consistent to all other input.
2.1.7 Resolution and wavelength ranges
Resolution and Wavelength are followed by ranges of resolution and wavelength in Å, respectively. The order of the minimum and maximum values is arbitrary. Wavelength may take an additional value as a reference wavelength, but this must be in the last place. The wavelength where the spectral peak occurs is usually selected as a reference. These ranges need not to be so accurate and realistic, rather, underestimated limits work better during the stage of indexing. For example, it would be better to use only modest resolution limit, and half to 30% maximum in bandpass. Prior knowledge about how far the crystal diffracts and the X-ray spectrum will be helpful. See 2.2.2 and 3.1 for automatic estimates of soft limits.
These required input do not have to be in any specific order, but they must be consistent with each other. Obviously, erroneous information in the input data will result in mis-indexing or failure to index. Minor error in the input data may prolong the process.
2.1.8 Exit from Input menu
Quit exits from Input menu. Obviously, this command must be the last.
2.2 First Indexing
If it is the very first time when you are working on a kind of crystal, Precognition needs to learn something about the typical spot profile in order to estimate the scale of realistic error level associated with your images. A command Profile is used to just do that. This command may take a few minutes to run, however, it is not always necessary for the subsequent runs, if you are working on the same kind of crystal with the same experimental setup. See 2.4 for more.
diagnostic
off
Input
¡
Quit
Spot re.spt
Profile
Ellipse
Nodal nodal.spt
Pattern pre.spt
Stop
Yes
Listing 2.2.0.0.1 Command script with profile learning.
This is an input script with a spot profile learning process prior to the ellipse and nodal recognition. The section omitted is a repetition of what we discussed in 2.1 Required Input. The first line diagnostic off turns off diagnostic messages, which may be annoying to general users. This command is the so-called global command. It is accepted at any level.
2.2.1 Spot recognition
Spot is a top level command that performs spot recognition. It is optional to add a filename to be created or overwritten to store the recognized spots. The filename can be absolute or relative, if you would like to direct the file to a specific location. The recognized spots contained in the file are sorted according to their signal-to-noise ratio.
0 0 0 1159.24 1880.96 48678.0 2365.1
0 0 0 998.11 1098.16 85323.8 4161.8
0 0 0 863.66 1322.68 26602.8 1375.3
0 0 0 1012.06 1595.98 99479.5 6198.8
¡
Listing 2.2.1.0.1 CPL reflection file that stores recognized spots.
The first three columns are h, k, l. They are all 0 since the pattern is not indexed yet. The next two columns are the location of recognized spots in pixel. The top-leftmost pixel of an image has coordinates (1, 1) when viewed along the X-ray beam (Figure 2.1.3.0.1). The last two columns are integrated intensities and associated uncertainties, respectively. This file is needed later when validating the indexing.
This command may also accept some optional numerical arguments. If one numerical argument is given, it is taken as the minimum acceptable I/s(I) value for the recognized spots, also known as sigma cut. The default sigma cut is 3. In case of week diffraction pattern, one may set this value lower. If you have very strong pattern, use a larger value to reduced number of spots. If two numerical arguments are given, they specify spot length and width of a typical spot in pixel. If three are given, they must be ordered as length, width and sigma cut. See 2.2.3 for spot profile.
A Python utility program pick.py wraps this command, and can be used independently. Type pick.py -h for its usage:
% pick.py -h
|||)) Welcome to CPL M0.15.2
|||\\ Renz Research, Inc.
SOURCE: /home/renz/code/adaptors/pick.py
TYPE: error
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Usage:
/home/renz/code/adaptors/pick.py [-h|help] image format [sigmacut [spot length
[spot width]]] [spotfile]
Picks diffraction spots from a
given image.
image: An image filename.
format: A string that specifies
the image format. Recognized
formats are:
['KodakStoragePhosphor',
'
'Q4',
'Quantum4',
'ESRFImageIntensifiedCCD',
'MarCCD',
'Kodak',
'ESRF',
'MAR3450Packed',
'MAR345',
'FujiImagingPlate',
'Mar345',
'MARCCD',
'MARCCD165']
sigmacut: Displayed spots must
have signal-to-noise ratio greater than this
value. The default is 1.
spot length and width: If
specified, must follow sigmacut.
The default is
10 pixels.
spotfile: Save recognized spots in
this file, if specified.
WARNING -
Existing file will be overwritten.
-h|help: Prints this usage.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Listing 2.2.1.0.2 Usage of spot picking utility pick.py.
Figure 2.2.1.0.1 Spots recognized at sigma cut of 2.0. This is clearly over-picked, since the spot arrangement at peripheral does not appear to be a diffraction pattern. The real pattern is submerged in noise.
Figure 2.2.1.0.2 Spots recognized at sigma cut of 2.3. Diffraction pattern obviously stands out from noise, but this is arguably still over-picked.
It would be a good idea to use this utility to find an appropriate sigma cut value before indexing. Figure 2.2.1.0.1 through 2.2.1.0.4 are a few examples of sigma cut.
Figure 2.2.1.0.3 Spots recognized at sigma cut of 2.6. It seems to be just right, since not much noise can be removed any more without removing spots from the real pattern.
Figure 2.2.1.0.4 Spots recognized at sigma cut of 2.9. It seems to be under-picked. Compared to the last Figure, we start to lose some real spots.
2.2.2 Auto-recognition of diffraction limit
I cannot over express the importance of accurate and consistent estimates of soft limits, such as crystal diffraction limit, source bandwidth, and a reliable minimum of signal-to-noise ratio, as early in the process as possible. Effective algorithm is implemented in Precognition to recognize highest diffraction resolution, maximum Bragg angle, and best sigma cut level before a pattern is even indexed and as soon as spot recognition is done. Recognition of other soft limits will have to wait until a pattern is indexed. However, this topic falls outside of the main flow of this section, first indexing. I would delay the discussion until the first part of Chapter 3. As it will be noted later, some soft limit recognition can be carried out at this point already, and the result may better assist the task of first indexing.
2.2.3 Learning spot profile
Profile is a top-level command for learning the typical spot profile. This command helps set up the typical spot length and width. It generates a report like this:
An
overall mean profile is recognized.
Semi-major
& -minor axes (pixel): 2.73176 1.59699
Non-elliptical correction:
-0.021812 0.00429671
-0.0552877 -0.0120681
Non-Gaussian
correction:
0.803568 0.636582
R.m.s.d.
(detector count):
0.00303301
Overall
spot length is set to 8 pixels.
Overall
spot width is set to 5 pixels.
Estimated
crystal dimension is 0.561176 mm.
Estimated
mosaic spread in FWHM is 0.0606017 degree.
Listing 2.2.3.0.1 Report generated by command Profile.
The newly recognized overall spot length and width may improve the last command Spot for spot recognition slightly. They can also be used in later runs, if working with similar patterns, in which case the command Profile is no longer necessary in subsequent runs. See 2.4 for more detail.
The estimates of crystal dimension and mosaic spread is based on a model of spherical crystal and isotropic mosaicity.
2.2.4 Ellipse recognition
Ellipse is a top-level command for ellipse recognition. Ellipse recognition must be carried out after spot recognition and before nodal recognition. It can take a filename as an optional argument that saves recognized ellipse centers, however, this is unnecessary if it is not a diagnostic run. This command also refines the direct-beam center while the ellipses are being refined. The refined center may be used in future runs.
2.2.5 Nodal recognition
Nodal does nodal recognition, if the ellipse recognition has been successfully performed. This top-level command can take an optional filename for saving the recognized nodal spots. This file has the same format as the spot file described earlier (Listing 2.2.1.0.1). This file is useful when a pattern is mis-indexed. See 2.5 for how to use this file to correct mis-indexing.
2.2.6 Indexing
Pattern does indexing, a key step in recognition of every detail of a Laue pattern. If some nodal spots are automatically recognized or manually given, this command tries to index the pattern and to refine the experimental geometry and cell parameters. This command takes some optional arguments, one of which is a filename for predicted spot locations, here pre.spt. This file can be used to compare with the observed spots for validation of indexing (See 2.3 for detail). This command generates a summarized report like this:
1
possible crystal orientation is recognized;
corresponding
cell constants and detector parameters are refined.
Indexing
1
Cell
lengths (Angstrom):
6.4232 7.3423 15.7870
Cell
angles (degree):
95.8000
96.5907 102.5697
Missetting
matrix:
-0.34351176
0.30330904 0.88882130
0.48259076 -0.75489410 0.44411829
0.80567105 0.58149680 0.11294086
Goniometer
omega, chi, phi(degree): 0.0000
0.0000 0.0000
Omega-axis
polar orientation (deg):
-90.0000
0.0000
Direct-beam
center (pixel):
1087.7794 1367.5079
Pixel
size (mm):
0.0792287 0.0792000
Detector
swing angles (degree):
0.0000
0.0000
Detector
tilt angles (degree):
0.2309
0.3280
Detector
bulge corrections:
-1.477e-05 1.113e-08
Missetting
matrix compatible with LaueView:
-0.1414,0.3708,0.9179
0.3662,-0.8418,0.3965
0.9197,0.3922,-0.01677
Listing 2.2.6.0.1 Report from command Pattern.
Please note that the missetting matrix used in Precognition and LaueView are different. This program offers a conversion.
The refined parameters in Listing 2.2.6.0.1 will be saved into a file, if a string argument follows the command Pattern. This filename is the string suffixed by .inp, here pre.spt.inp. This file has the correct format of a command script for loading back into Precognition, which is very similar to the default file of LaueView in terms of function. None of the parameters included in this file is specific to a particular frame, although they were obtained from refinement against one frame. These parameters are specific to a fixed crystal and detector mounting during a dataset. Presumably, all of them can fit another pattern after slight refinement. This parameter file is therefore called non-frame-specific, and could be used on any other frame in the dataset later. See Chapter 3 for frame-specific parameter file. The naming convention of this file may be changed later.
Input
Crystal 93.669
44.000 83.600 90.000 122.016 90.000 5
Matrix 0.048116 -0.055959
0.997273 0.072781 0.995972 0.052375 -0.996187 0.070062 0.051995
Omega 90.000 0.000
Distance 101.988 0.250
Center 1146.45 1143.35 1.00
1.00
Pixel 0.081717
0.081600 0.001000 0.001000
Swing 0.000 0.000
0.000 0.000
Tilt -0.636992
-0.194799 0.100000 0.100000
Bulge -0.000063569100
0.000000058691 0.000001000000 0.000000001000
Resolution 2.20 100.00
Wavelength 0.70 1.35
Quit
Listing 2.2.6.0.2 A Precognition generated input file that contains parameters that are not specific to a frame.
See the next chapter for usage of this file and the new commands Matrix, Omega, Swing, Tilt, and Bulge appeared in this file.
2.3
Validation of Indexing
How do I know whether it is mis-indexed? You can tell from the refinement residual, and it is immediately obvious from the observed and predicted patterns superimposed together. To plot such superimposed patterns, a utility program written in Python su.py is available. Type su.py ¨Ch for a message of usage:
Usage: su.py [-h or -help]
[background] [foreground] [dimension] [sigmacut]
Displays two patterns superimposed
together.
background, foreground: Two spot
files. The first one will be
displayed
in red circles;
the second in black dots. If
omitted, the defaults
are re.spt and
pre.spt.
dimension: Dimension of a square displaying
area. The default is 2048.
sigmacut: Displayed spots must
have signal-to-noise ratio greater than this
value. The default is 1.
Listing 2.3.0.0.1 Usage of su.py.
This utility prepares a project file for GRACE and launches it. A fully functional GRACE session will appear in a new window. Consult GRACE manual (http://plasma-gate.weizmann.ac.il/Grace/doc/UsersGuide.html), if you want to manipulate the graph. The default graph will look like this in Figure 2.3.0.0.1.
Figure 2.3.0.01. A GRACE-generated graph showing two Laue patterns superimposed together.
2.4
Routine Indexing
If you always work on a same type of crystals under a fixed experimental setup, you need to index a lots of patterns of a similar quality again and again. You must then have very good idea about the spot profile, direct-beam center, crystal-to-detector distance, etc. It would be quicker to use this script for such routine indexing:
diagnostic off
Input
¡
Spot 10 4
Quit
Spot re.spt
Ellipse
Nodal
Pattern pre.spt
Stop
Yes
Listing 2.4.0.0.1 Command script for routine indexing.
The only difference here is to set a typical spot length and width instead of running the command Profile. You may use the suggested values from a previous run or those yield success routinely.
Here, a new command Spot at Input level may accept up to three numerical arguments. The rule is the same as that for Spot command at the main level, but this Spot command at Input level does not perform spot recognition. Once again, if one numerical argument is given, it is taken as the minimum acceptable I/s(I) value for the recognized spots for future use. If two numerical arguments are given, they specify spot length and width of a typical spot in pixel. If three are given, they must be ordered as length, width and sigma cut.
2.5
Mis-indexing
Everything designed and built into this program is to guard from mis-indexing, however this will still be the most frequent topic as it has always been. If your pattern is mis-indexed, the first thing you should check is whether all input information are correct and consistent with each other, cell constants, distance, pixel size, and most importantly, direct-beam center, and whether you are indexing the image file you really intend. If after one has explored all options of the program, the pattern is still mis-indexed, a common mistake is often that wrong or inconsistent information are given. After all, if you do think it is the program¡¯s fault, this section explains what other options you have.
The process of Laue indexing is essentially a pattern matching game of diffraction rays in an angular space. All error sources will influence the outcome of the matching. Accidental mismatch does happen. The strategy to identify mis-indexing in this program is to rank all successful matching and to hope the top or near top matching is indeed the correct indexing. Mis-indexing at the final outcome of the program means that no successful matching near the top of the ranking list is a correct one. This can virtually never happen, if all input are error free. There must be a correct matching near the top of the ranked list, if it is not the very top one. In order to avoid mis-indexing, we must first minimize the errors, and second alter the length of the ranked list.
2.5.1 Check all possible
matching
A single numerical argument 0 added to the command Pattern will change the program¡¯s
behavior: Pattern 0 pre.spt
This command will run much longer to search for all possible crystal orientations and the top 10 of them will be saved for your inspection. The very top solution is saved in pre.spt. The other 9 will be 1_pre.spt, 2_pre.spt, ¡
If this numerical argument is positive, the program will keep searching until this many matching solutions are found. If this argument is 0 or negative, all possible solutions will be examined before the program exits. This option may cause very long running time. Success rate will be dramatically increased if one allows to search the first a few solutions instead of the very first one. I suggest to use a small number, say 3 or 4 before trying larger ones, 10 or 20 for this option.
Figure 2.5.1.0.1 shows an example of mis-indexing in an initial trial, and the argument 0 worked successfully. This pattern happens to display two major ellipses nearly tangential to each other, that is, the diagonal of two major axes falls on the X-ray beam. Patterns of this kind are usually harder to index, and shall be avoided if possible. Compare to Figure 2.3.0.0.1 for difference.
Figure 2.5.1.0.1 A Laue pattern mis-indexed with the default value of the numerical argument to Pattern, but correctly indexed with command Pattern 0.
2.5.2 Use more nodal spots
If none of the possible solutions identified are correct indexing, another option you may want to play with is the second numerical argument ranging from 8 to 16. 8 is the default, so that there is no need to repeat if you have run the default. This argument specifies the number of top quality nodal spots used in the indexing process. The greater this value is, the longer the program will run. If this second argument is greater than 10, it is recommended to combine with 1 or a small first argument. Otherwise, the program may run intolerably long.
2.5.3 Manual indexing
All above options insist on no user assistance on nodal selection. In fact, you may win the game much more easily if you are willing to help a little bit with your visual power. If you can personally inspect a pattern for a minute, pick a few major nodal spots, put them in a spot file, or if you can screen the program-generated spot file, select some major nodal spots, the program will have much better chance to get the right indexing from your hand-edited spot file. You can change the script slightly by specifying a spot file in Input section, and remove the command for nodal recognition.
diagnostic
off
Input
¡
Nodal nodal.spt
Quit
Spot re.spt
Ellipse
Pattern pre.spt
Stop
Yes
Listing 2.5.3.0.1 Command script to take user supplied nodal spots.
Please note that the new command Nodal at Input level differs from that at the main level. It does not perform nodal recognition, instead it loads a spot file that contains nodal spots only.
Although it is possible without the GUI version of Precognition, this task becomes so much easier when using the graphical interface Precognition.py. If you have the GUI, launch it by running Precognition.py. Choose an image type from File menu, browse and select an image file. The image will be displayed in a graphic window. From Tool menu of this graphic window, choose Spot and then Manual Picking. Adjust Spot Size in the newly appeared dialog window, and use left mouse button to pick desired nodal spots in the graphic window. Each selected spot will be marked by a symbol. Click on a selected spot will delete this spot. Click OK button when done. From File menu, choose Save and then Spots. Browse and select a filename for saving the selected nodal spots. Please refer to Chapter 12 for more details on the graphical user interface.
2.5.4 Other options
If your patterns are from a small molecule crystal or collected using a narrow bandpass source, they become so sparse that few ellipses are sensitive to human vision. You will be surprised that Precognition does far better job than your eyes.
If you cannot index one pattern, can you choose some other patterns? This usually solves the difficulty easily, if your crystal happened to be mounted at an odd orientation. Nevertheless, there are still something else one can try after all these options failed. For example, were the data collected in room temperature, but you never know the RT cell of your crystal, and you simply assumed it is as same as the low temperature cell? This happens quite often these days. Are you sure that the detector is indeed normal to the X-ray beam as people always assume so.
Finally, call me, if you give up. I do want to learn from your case and to make Precognition more powerful.
2.6
Quick Indexing
If some nodal spots are known, either from a previous run or from hand picking, and an accurate direct-beam center is available, the following script carries out a quick indexing without searching nodal spots and refinement of center. The program will load nodal spots from the given file nodal.spt.
diagnostic
off
Input
¡
Center 1087 1368
Spot 7 4
nodal nodal.spt
Quit
Spot re.spt
Pattern pre.spt
Stop
Yes
Listing 2.6.0.0.1 Command script for quick indexing.
2.7
Goniometer Setting
It is not an issue, if you only work on a single image. If you have a set of images to process, their relative geometric relationship must be known. You need somehow to tell Precognition how these images were spatially arranged. Goniometer setting is therefore needed. We delay this topic until the next chapter, when multiple images are concerned.