• 1. GETTING THE DATA INTO LAUEGEN
• 1.1. FTP ladi
• 1.2. Reorder
• 2. LAUEGEN
• 2.1. Displaying an image
• 2.2. Indexing
• 2.3. Refining the orientation
• 2.4. Refening the Soft Limits
• 3. INTEGRATING
• 3.1. Integrating the pattern
• 4. LAUENORM
• 4.1. Lauenorm Input files
• 4.2. Lauenorm Output
• 5. AGROVATA
• 5.2. Agrovata Output
• 6. Getting the HKL file
• 7. Backing up your data
• 8. PICTURES
• 8.1. Quasicrystal movie
• 8.2. Model of LADI
• 8.3. Birdseye View of Grenoble
• 8.4. Java Laue Patterns
• USEFUL LINKS

You have performed an experiment on LADI. The raw data is stored on the HP ladi control computer in a .edf file. You need to move the data to Ladisgi for processing.

1. You must be logged into Ladisgi in the directory in which you wish to process your data.

2. You need to ftp from the control computer to collect the .edf file.

3. The control computer is called simply 'ladi', the username is 'operator' and the password is 'ladi97'. Start FTP, the dialogue will look like this:-

(Bold letters indicate what you type in.)

<code>
ladisgi 13% ftp ladi
Connected to ladi.ill.fr.
220 ladi FTP server (Version 1.7.212.1 Sat Feb  1 01:30:15 GMT 1997) ready.
Name (ladi:guest): operator
331 Password required for operator.
Password:
230 User operator logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
</code>

4. The data will probabaly be in the directory 'data1', go there.

<code>
ftp> cd data1
250 CWD command successful.
</code>

5. Collect the file containing your image and quit FTP.

<code>
ftp> get  image01.edf
local: image01.edf remote: image01.edf
200 PORT command successful.
150 Opening BINARY mode data connection for image01.edf (16000650 bytes).
226 Transfer complete.
16000650 bytes received in 15.15 seconds (1031.19 Kbytes/s)
ftp> quit
221 Goodbye.
ladisgi 14% 
</code>

6. Now your file is on ladisgi you can start to process it.

Once the data is on Ladisgi you have to pre-process the files in order to make them viewable in LAUEGEN. This is done with the program REORDER.

1. Your are logged into Ladisgi and working in the directory which contains your raw data, the .edf files transferred over from the control computer.

2. The .edf file is a large binary file with a small text header at the beginning. The text header needs to be removed and the binary data reorganised. You need to know the size of the text header at the top of the .edf file inorder to remove it.

This header is quite useful, it contains information about the experiment, such as the date, temperature, exposure time, etc. And can be viewed with the command.

<code>
ladisgi 23% head -23 image01.edf
</code>

(Prints out the header on the screen. An example.)

The number of bytes of header is the file size minus 16000000. The filesize varies a little but is usually around 16000600. Find this with :-

<code>    
ladisgi 26% ls -l image01.edf
-rw-r--r--    1 guest    guest    16000650 Nov  4 19:57 image.reo
</code>

3. Run the program REORDER.

<code>
ladisgi 19% reorder 
Name of input file: image01.edf
Name of output file: image01.reo </code>

Change the filename ending for the output file to .reo, for reordered.

<code>
Film (f) or image-plate (i) file [i]: i </code>

i = An image plate from LADI.

<code>
No. of x rasters, no. of y rasters [1200 1200]: 4000 2000 </code>

The size of the detector in pixels. This is shown in the header

<code>
Order of axes in input data file [+x+y]: -y +x </code>

The order required to make the image the correct shape for the LAUEGEN window.

<code>
Byte swap data [n]: n </code>

<code>
Number of header bytes to skip = 650 </code>

The number of bytes in the test header of the .edf file to be removed. See above.

<code>
Translating image plate data
Name of input  file: image01.edf
Name of output file: image01.reo
No. x rasters = 4000, no. y rasters = 2000
Input axis order: -yf +xf
Number of header bytes to skip = 650
*Skipping header data*
*Reading file*
*Writing file*
*Translation successfully completed*ladisgi 20% 
</code>

4. To check that you have successfully translated the file you can look at its size. The .reo file should have exactly 16000000 bytes

<code>
ladisgi 5% ls -l image01.reo
-rw-r--r--    1 guest    guest    16000000 Nov  4 19:57 image01.reo
</code>

5. Your file is ready to be displayed and processed in LAUEGEN.

Jreo.

Alternatively use the program JREO, which works more automatically.

<code>
ladisgi 5% /d2ladisgi/people/cowan/progs/jreo
Enter the image file name.
image1.edf
Enter the name of the output file.
image1.reo </code>

Then the header will be printed on the screen and the file converted automatically.

The program LAUEGEN is used to display the diffraction patterns, which are now stored as .reo files.

To display an image you need two files.

1. A reo file containing the image.
2. An .ldm data file

The ldm data file.

The .ldm data file contains the parameters needed by LAUEGEN to view your image. Some parameters are unique to the detector and some are unique to your image. To display your first image you need an .ldm file with the detector information in it, which you can modify for your image. The easiest way to do this is to borrow an .ldm file used in another experiment and use LAUEGEN to modify it.

Copy an .ldm file to your directory. For instance:

<code>
	ladisgi 15% cp /d2ladisgi/lss/wilkinso/template.ldm . </code>

Now run LAUEGEN with the command.

<code>
	ladisgi 15% newlauegen </code>

Or run the old version of lauegen, which is needed if you wish to use the integration programs INTEGRATE+ or ARGONNE_BOXES.

<code>
	ladisgi 15% lauegen </code>

After you have chosen a few likely spots click the End Spots Input button. Then the Measurement Complete button. In the central window of the LAUEGEN screen you will be asked some questions. These appear automatically as you proceed through. At the first attempt accept the default values. If no solutions are found you can increase these values

<code>
Limit of h**2+k**2+l**2 (max=40) [10]: 
Error in spot positions (mm) [0.50]: 
</code>

After attempting to auto index the spots selected the program will offer you some possible solutions to indexing. Some may be equivalent, some will be wrong. It is not obvious form the list. To find the correct solution you must use the Show Solutions option. A correct solution will be obvious, all the real spots in the image will be marked with a coloured prediction in, or close to, the centre.

To easily see the predicted spots in relation to the observed data click on Colour in the image window and select White on Black. Zooming in also helps to see the fit more clearly.

When you believe that you have the correct solution, click on Select Solution and select the correct one from the list displayed.

If the indexing has not found an acceptable solution then :

1. Check that detector geom says cyl.
2. Try increasing the 'Error in spot positions' and 'Limit of h**2+k**2+l**2'.
3. Try more spots, make sure they are spread evenly about the about the image.

Try a few times, if you cannot find a solution then you may have the wrong unit cell. Or there may be something wrong with the crystal e.g. twinned. Try another image from the same crystal and rotate the solution until it fits. The rotations about Phi in the detector correspond to rotations about PhiZ or Spindle, but in the opposite sense.

A few more questions. Accept the defaults. The RMS should have reduced alot. If it has not then answer Y to "<code>Continue refinement with current spots (y/n)</code>" and repeat the procedure refining a few more things on the list. It is best at the beginning to stick to the top six options on the list (phix, phiy, phiz, c_to_f, x_c, y_c).

<code>
**Spots found outside Chebyshev limits**
PhiX = 177.36   PhiY = 4.79   PhiZ = 73.66
c_to_f = 159.13   x_c = -0.563   y_c = -1.310   w_c = 0.000   y_scale = 0.998
RMS = 0.492 mm. for the 18 spots used in the refinement
(from 1.783)
Display difference plot (y/n) [n]: 
Accept refined parameters (y/n) [y]: 
Continue refinement with current spots (y/n) [n]: 
</code>

Check the image, the predicted and observed patterns should match up quite well now. Continue with Nodals, Search and Refine.

2. Nodals Search and Refine.

With a good fit from the original indexing then it is easiest to use this option. The program searches for and matches the predicted and observed spots. To begin from a well indexed pattern click on 'Nodals Search and Refine'.

Accept the defaults for the questions in the centre window for the first refinement.

<code>
==Generating reflections and finding overlaps==
Number of spots = 921
Number of reflections = 1079
Number of singles = 811
Number of multiples = 110
Number spatially overlapped = 4
Number too close to integrate = 0
Histogram of nodal spots
========================
1   8          2   34         3   83         4   145
5   251        6   337        7   471        8   564
9   665        10  741        11  832        12  879
Nodal spot selection index [4]:  return 
145 nodal spots
Use radial masks in spot position determination (y/n) [y]:  return 
Threshold above background for c_of_g calc [100.0]:  return 
===Searching for Reflections===
RMS = 0.756 for 96 spots
Number of spots to use in refinement [96]:  return 
Difference plot required (y/n) [n]:  return 
RMS = 0.756 mm. for the 96 spots to be used for refining

</code>

The refinement menu should appear as above. Refine only the missetting angles: phix, phiy, phiz for the first time. Click Refine at the bottom of the menu.

<code>
**Spots found outside Chebyshev limits**
PhiX = 88.54   PhiY = -58.39   PhiZ = -32.87
c_to_f = 159.26   x_c = 1.233   y_c = -0.312   w_c = 0.000   y_scale = 0.998
RMS = 0.383 mm. for the 96 spots used in the refinement
(from 0.756) </code>

The results of the refinement are not bad. A reasonable drop in the RMS value. Accept the defaults for the rest of the questions.

<code>
Display difference plot (y/n) [n]:  return  
Accept refined parameters (y/n) [y]:  return  
Continue refinement with current spots (y/n) [n]:  return  </code>

Now repeat the process by clicking Nodals Search and Refine again.

When repeating the refinement you can make it more accurate by:

1. Increasing the nodal spot index, so including more spots in the refinement.
2. Refining other parameters: the offsets, the cell, polynomial correction, and the distortion.

Keep repeating the refinement until there is no improvement in the RMS value. When repeating it is usually good enough to continue using the same spots. (eg.<code> Continue refinement with current spots (y/n) [n]: y </code>).

After every few refinements, or after a large improvement in the RMS, you should restart with a new set of spots, even if you use the same 'nodal spot index'. This reindexs some the spots that may have been indexed incorrectly before. Often the RMS value increases after this, but will drop again after a refinement.

When processing a set of images from the same crystal it is best to refine the cell only for one image. If the cell is refined for different images then this changes the predicted wavelength of the reflections and introduces errors for the wavelength normalisation stage.

When you are satisfied that you have the correct value of 'dmin' click Accept new soft limit or Input new soft limit if you think the value if different. Then return to Return to previous menu.

Occasionally the computer fails to determine 'dmin'.

<code>
Value determined for 'dmin' is 0.00
*Warning* No bin has 10 or more significant spots
</code>

When this happens try to guess 'dmin' from the histograms. If the histograms look odd try changing 'lmin', if it is very low increase it a little and repeat Determine 'dmin'.

Determine the Minimum Wavelength.

The minimum value for the wavelength is found in an identical way to 'dmin'. Click on Determine 'lambda-min' in the 'Soft Limits Options' menu.
After finding 'dmin' and 'lambda-min' once, repeat the process and find 'dmin' again. The value the program calculates for 'dmin' depends on the starting value of 'lambda-min', and vice-versa. Keep determining each value in turn until neither result changes. This should only take a few cycles. If the values both go to 0.00, then change the values manually in the 'Crystallographic/pack Parameters' section of the LAUEGEN screen to two values significantly above the real values (try maybe dmin=1.0, lmin=1.0) and repeat the process until you are satisfied with the results. Remember to keep checking the histograms and the images during these iterations.

Once the pattern has been well indexed, and the soft limits determined satisfactorally the intensities of the spots can be measured by integrating them.

There are two integration routes. Either use the built in integration routine in the new version of LAUEGEN, or use the integration program, INTEGRATE+ or its modification ARGONNE_BOXES.

• flags+ file. A simpler and easier to read file than the .geasc containing the intensities.
• neighbours+ file. A file containing details of how each reflection was integrated.
• models+ file. A file containing the models used in the integration.

The flags+ file may be useful and you should rename it, otherwise it will be overwritten next time you run INTEGRATE+.

<code>
ladisgi 23% mv flags+ image01.flags+ </code>

After the _mod.geasc file has been created it needs to be converted back into binary. Use the program EXCHGE again, as below.

<code>
ladisgi 9% exchge
Translate to ascii (t), from ascii (f) or print header (p) [t]: f
Name of input ascii file (default ext=.geasc): image01_mod
Root name for output .ge1/.ge2 files: image01_mod
*File translation completed*
ladisgi 10% 
</code>

The _mod.ge1 and _mod.ge2 files are the ones needed for the normalisation.

The .ldm files have a variable format, only one kind works with INTEGRATE+. The most common problem I have with INTEGRATE+ is if I have the wrong kind of .ldm file. A file that works is:

<code>
/d2ladisgi/people/cowan/templates/int+.ldm
</code>

This file has to be read into LAUEGEN (old version) all the parameters corrected, then written out again.

ARGONNE BOXES

ARGONNE_BOXES is an updated version of INTEGRATE+ with more scope for altering the integration parameters. The same files are required to run ARGONNE_BOXES as INTEGRATE+, so you must make the .ldm and .geasc files and collect a fuji_borders file as above.

Finally click on Write Intensities File to write the output .ge files

Once all the images in the dataset have been integrated and the .ge files written, then LAUENORM can be used for the wavelength normalisation. It works by comparing the equivalent reflections and refining a wavelength normalisation curve. This curve takes into account any factor only dependent on wavelength, mainly the different flux and different wavelengths, but also the response of the image-plates to different wavelengths.

To run LAUENORM you need 3 types of files.

1. An lnorm.com file.
2. An lnorm.dat file.
3. All the .ge files from the images.

The lnorm.com file.

The Lnorm.com file is the command file which runs LAUENORM. Collect a file from a previously processed dataset and modify it.

ladisgi 15% cp /d2ladisgi/people/cowan/templates/lnorm_template.com . 

The file has to be modified for each new dataset. Here is an example of an lnorm.com file.

After you have modified the file save it as something unambiguous.

ladisgi 15% mv lnorm_template.com lnorm_image.com 

The lnorm.dat file.

The Lnorm.dat file contains the parameters for running LAUENORM. Collect a file in the same way.

ladisgi 15% cp /d2ladisgi/people/cowan/templates/lnorm_template.dat . 

This file has to be modified before running the program. Below is an example. Its not a very easy file to work with, as you have keep modifying the numbers at the end of the file. And you have to try to remember the effect of each one.

Template for lauenorm
7.000 9.000 11.000  70.000 80.000 90.000
NORMALISE 6 GE 0
UNITY
UNITY
2 1 8 4 6 4 1
1 1 2 0.0 0 0 0 3.0
1.0 1.9 20 6 3
3
0 25.0 0 0 0

Here is a run down of the most important parameters.

This web page has a more detailed description of the dat file. LAUENORM Users Documentation

After you have modified the file save it under the name referenced in the .com file.

ladisgi 15% mv lnorm_template.dat lnorm_image.dat 

Now you are ready to run the program.

ladisgi 15% lnorm_image.com
5.421u 0.192s 0:13.86 40.4% 0+0k 0+1io 0pf+0w
ladisgi 16% 

The program should have worked. A common error occurs if an output file (eg. image.mtz) already exists. This must be deleted or renamed before the program will run.

LAUENORM should produce 2 files.

• image.mtz - A binary file containing the normalised data.
• lnorm.log - A log file giving details of what the program did.

The log file should be a huge file containing lots of tables full of numbers. Some parts of the file are useful to check and see which parameters to change in the .dat file.

An example of an lnorm.log file.

The first thing to check is if the program ran successfully. If you see this message then the program has failed because the output file, image.mtz already exists. You must remove or rename image.mtz and run the program again.

<code>
ladisgi 77% lnorm_image.com
 LAUENORM:  (Q)QOPEN NEW file already exists: image.mtz                                                

0.027u 0.050s 0:00.18 38.8% 0+0k 0+1io 0pf+0w
ladisgi 78% rm image.mtz </code>

Next check what the end of the lnorm.log file looks like by using NEDIT. The last line should say how many reflections are output. If it does not then something has gone wrong somewhere.

<code>
ladisgi 77% nedit lnorm.log


NUMBER OF REFLECTIONS WRITTEN TO OUTPUT FILE  =   7936
</code>

If the program failed to work properly and at the end of the lnorm.log file there is a message that says.

<code>
**MISSING DATA FOR SOME BATCHES**
</code>

Then the program nearly worked, and you are on the right track. If you have any other message it probabaly means there is a mistake in the .dat or .com file, and you should check these carefully.

If you are 'MISSING DATA FOR SOME BATCHES' you must change some of the parameters in the .dat file to numbers appropriate for you dataset. If the program worked properly you should also change some of the numbers in the .dat to try to optimise the output. Both cases are similar.

To try to find which parameters to change you should examine parts of the lnorm.log file.

Sections to check in the lnorm.log file:

<code>
</code>
 SUMMARY OF WAVELENGTH BINS DATA:

There should be a table reporting how many reflections are in each wavelength bin. There should be a fairly even distribution of the reflections across the bins. If you got the 'Missing Data' message then this table gives you an idea what parameters to alter.

The rest of the file contains details of how the refinement progressed. The most interesting are at the end of the file, where the final results are:

<code>
</code>
 REFINE FILM PACK SCALE FACTORS

There is a scale factor refined for each image. These should be proportional to the exposure time of each image. If a scale factor looks odd perhaps that image has something wrong with it, eg. wrongly indexed, bad integration, crystal out of the beam; and the image should be checked.

<code>
</code>
 LAUE INTER-FILMPACK MERGING R-FACTORS = SUM(ABS(I(I)-I(J)))/SUM((I(I)+I(J))

A large symmetric matrix showing the agreement between images. All the numbers should be similar. A number wildly different from the rest indicates that an image has something wrong with it, perhaps for the same reasons as above.

<code>
</code>
 LAUE MERGING R-FACTORS = SUM(ABS(I-IMEAN)/SUM(I)

You would like these R-factors, similar to the merging R-factor in SHELX, to be as small as possible.

<code>
</code>
 NUMBER OF REFLECTIONS WRITTEN TO OUTPUT FILE  =   7936

You obviously would like as many reflections as possible.

LAUENORM is a trial and error business. Run the program, examine the output, change some parameters, run the program again. The aim is to maximise the number of output reflections and minimise the R-factors. The most useful parameters to change are:

• Wavelength range. Increasing the wavelength range includes more reflections, but usually increases the R-factors. If you increase the range too much then you may not have enough data to refine at the high and low lambda limits.
  
  
• Number of bins. Try changing this to find the best value.
  
  
• Order of polynomial. Again just change this until you have the best value.
  
  
• Sigma cutoffs. If you reject reflections with the I/sigI cutoff then the R-factor will increase, but you will lose data. Altering the I/sigI cutoff for the refinement is more useful, so you can refine the wavelength curve on your most accurate data.

Keep running the program until you are certain that you have the best results. Remember to delete or rename the output .mtz file before you rerun the program.

AGROVATA should produce 2 files.

• image_out.mtz - A binary file containing the merged data.
• agro.out - A log file giving details of what the program did.

The agro.out file, similar to the lnorm.log file, is long listing of what the program did. Like the LAUENORM you need to examine the output to try to optimise the input parameters.

An example of an agro.out file.

Similar to LAUENORM, you must have removed any previous output _out.mtz file for the program to run.

Once the program has run correctly, view the agro.out file with NEDIT to examine the results.

Sections to check in the agro.out file:

<code>
</code>
  * Number of Reflections = 1861

At the bottom of the file it should tell you how many reflections were output.

<code>
</code>
  Number of reflections for each symmetry operation

 symmetry operation   Number of hkl    Number rejected   Fraction

           1                4284                69         0.0161
           2                3409                35         0.0103

There should be this table, explaining how many reflections have been rejected. There is an explantation about how the reflections are rejected above this table. Basically, if the reflections are more than the value SDREJECT away from the mean, then they may be thrown away, but there will not be more than 'BADREJECT PERCENT' of the reflections thrown away, and not more than 'BADREJECT NOBS' thrown away. This also may give you a hint if you have the space group wrong. Not too many reflections should be discarded, if more than ~5% are rejected, then you should increase SDREJECT.

<code>
</code>
   $TABLE: Analysis against resolution:

Under this heading there should be a table that shows the R-factors compared to Resolution. If some of it is bad then maybe change the RESOLUTION limits.

<code>
</code>
   $TABLE: Analysis against intensity:

Under this heading there should be a table that shows the R-factors compared to Intensity. At the bottom of this table should be a line summing up the results. It gives an overall R-factor for the merged dataset. Here 0.141.

<code>
TOTALS   0.141    0.000      0    3181.   946.2     3.4   500.9    7434.   1706    
</code>




<code>
</code>
   $TABLE: Completeness & multiplicity v. resolution:

A table showing how complete the data is. At the bottom is a total. Here there are 7589 accepted input observations, with 1861 uique reflections, and a total completeness of 95.2% within the resolution range.

<code>
Total      7589     1861     95.2      4.1
</code>




<code>
</code>
   $TABLE : Analysis of standard deviation v. Intensity:

AGROVATA gives you the opportunity to 'fudge' the standard deviations of the intensities. Altering SDINFLATE SDFAC + SDADD, changes the standard deviations. In the agro.out file are instructions on how to alter them. Then total mean should be 0.0 and the total sigma should be 1.00. Reduce sigma by increasing SDADD. Increase mean by increasing SDFAC. Trial and error is the best way to get the values right.

<code>
 Range   Imin     Imax              Irms    Number      Mean     Sigma  $$
                                    Fully_recordeds
 TOTALS:
    0       8.   64726.             6766.    7434.      0.09      1.04
</code>

In the same way as with LAUENORM, keep repeating the program altering the parameters, until you are satified with the results. Remember to delete the output .mtz file each time you rerun the program.

Agrovata Program documentation

MTZ2VARIOUS

The program MTZ2VARIOUS can be used to convert the an .mtz file into a wide variety of formats.

To convert to SHELX hkl format.

To convert the output .mtz file from AGROVATA to SHELX hkl format you need a command file, which must be edited.

<code>
ladisgi 28% cp /d2ladisgi/people/cowan/templates/mtz2v . </code>

Only the filenames in the command file need to be changed.

<code>
mtz2various HKLIN image_out.mtz HKLOUT image.hkl << EOF
OUTPUT SHELX77
LABIN FP=IMEAN SIGFP=SIGIMEAN

END
EOF
</code>

Then run the program.

<code>
ladisgi 28% mtz2v </code>

Mtz2various CCP4 documentation.

If the 'Data Source Directory' is not /d1ladisgi/DATA, then type 1 and change it.

<code>
Please, enter the full path of source directory
(you can use an NFS mounted directory)

Path:/d1ladisgi/DATA </code>

Next check to see if there is enough space on the CD with option 2.

<code>
Please wait...

Result: 265740 Kbytes (259 Mb)

         <<< Please, press Enter to continue>>>
</code>

This is fine. One CD can hold upto 650Mb. Its best not to go to close to this limit, as the program adds a few extra files.

To make the CD, select option 345Q. The program will run and in about 10-15 minutes the CD will be made.

<code>
Your Choice ? 345Q </code>

After making the CD, and checking that it has worked, remove your files from /DATA.

Download ladi.wrl

The Laue Suite Homepage

Max von Laue  - Biographie

IUCr Homepage

CCP4 Homepage