Filing for VMS/UNIX

A tutorial

The TAS software package Home page	Installing the TAS software package on a Macintosh	PkFit for the MacOS A tutorial	Filingfor the MacOS A tutorial
Downloading	Installing the TAS software package on VMS and UNIX	PkFit for VMS/UNIX A tutorial	Filing for VMS/UNIX A tutorial

1. Goals and definitions

Filing is an interactive tool for the handling of TAS data files. It makes it possible to create a new data set from one or more data files, the contents of which can be modified, grouped and/or combined. The main features of the program are listed below.

Modifying a data set

With Filing, almost all the data set inputs can be modified manually:

the header parameters can be edited,
the resolution parameters can be edited,
the temperature value can be changed,
data points can be removed or modified,
data points can be grouped.

The latter option is especially usefull to improve the statistic or to make the plot more understandable e.g., when the used scan step is too narrow compared to the resolution width.

Option "Add files"

When performing a TAS experiment many measuring strategies can be used. For example, a given scan may be repeated several times or performed by parts. Scan ranges can be identical, contiguous, non-contiguous, overlapping, etc. Step widths and/or counting times can be identical or not. The goal of the option "Add files" is thus to merge, concatenate, append, rescale, etc, data to produce a single data set from several data files whenever this makes sense.

Option "Combine files"

The goal of the option "Combine files" is to perform a linear combination of the contents of several data files. This is especially useful: - to subtract a background, either observed or given. - to remove unwanted peaks. E.g., the same scan may be repeated above and below a phase transition or with different neutron polarizations. In such a case combining data sets may be used to remove all peaks but those present only in one of the states.

Filing limitations

Filing can open and create data sets with a maximum of 500 points.
Filing can simultaneously open up to 7 files in the formats described in the manual of PkFit v2.1 for the Macintosh.
The resulting data set can be saved only in the ILL TAS format.

2. Algorithms

To divide the sheep from the goats, Filing performs a set of compatibility tests when running the options "Add files" and "Combine files". Some tests require that some file characteristics be strictly identical. Others are controlled by user defined tolerance levels.

Compatibility between plot-axis variables

Filing can only add or combine data files if they have identical scan variables and identical X and Y plot-axis variables. When this compatibility test is not satisfied, the program refuses to process the files.

Header parameters' compatibility

This second set of compatibility tests is applyed to the header parameters of the input files. Each header parameter of each input file is compared to the corresponding header parameter of the first input file on the basis of a custom set of tolerance levels, Tol. A file File_j is said compatible with the first one File_1 if all header parameters are equal within the specified tolerance. If the files are not compatible, Filing displays a warning message and writes a failure report to the text window (see below). The user can then decide either to ignore the parameter discrepancy or to abandon the file processing.

Superimposed and not-superimposed points

Once the compatibility tests have been performed successfully, the contents of the input files can be concatenated, added or combined. However, during the processing, an extra tolerance level is used to decide wether or not very close data points should be considered as superimposed or independent. This tolerance level, Tol, is applied to the mean value of the scan step, DX. Two points from different input files, X(File1) and X(File2), will be superimposed if the following condition is satisfied:

<dd>

Note that, as explained below, the options "Add files" and "Combine files" use different algorithms to process superimposed and not-superimposed points.

Option "Add files"

When n files are added all data points are kept. Superimposed points are summed and rescaled. For 2 files, the add algorithm is as follows:

Not-superimposed points are copied unchanged to the resulting data set.

</dd>

while two superimposed points are added as follows:

with, for a given scan point from file i,

X(i)

the abscissa

Y(i)

the neutron counts

DY(i)

the Y error

Mon(i)

the monitor counts

and for the resulting data point

X	the abscissa
Y	the neutron counts
DY	the Y error
Mon	the monitor counts

Option "Combine files"

When n files are combined, each point of the resulting data set is the scaled sum of n "superimposed" data points. When less than n data are observed at a given x-value, the resulting point cannot be computed and the corresponding input points are simply skipped.

For two files the combine algorithm is as follows:

with, for a given scan point of file i

X(i)

the abscissa

Y(i)

the neutron counts

DY(i)

the Y error

Mon(i)

the monitor counts

Fact(i)

a multiplying factor choosen by the user

and for the resulting data point

X	the abscissa
Y	the neutron counts
DY	the Y error
Mon	the monitor counts

In the resulting data set all points are scaled to the monitor counts <tt>Mon</tt>.

Note that, with the option "Add files", the Poisson statistics of the data is kept while, with the option "Combine files", it is generally lost. As a consequence, be careful when you use a resulting data set created by the option "Combine files". E.g., using the option "Add files" after the option "Combine files" may result in wrong Y-errors.

3. Launching Filing

To launch PkFit, simply double-click on its icon or type the name of the program as the command.

Then type a return or "/xserve" if you are using an X terminal otherwise see the graphics device help. The PGPLOT window is displayed with a menu bar in its bottom part.

4. Setting general options

Select the item "General" from the menu "Options". A dialogue allows you to fix the monitor to a given value, or to set the default file format and the default data-directory.

5. Setting tolerance levels

Select the item "Tolerances" from the menu "Options". A dialogue allows you to set the tolerance levels.

Click the button OK to exit from the dialogue. Then click the item "Menu1" to go back to the main menu.

6. Ex. 1: Grouping a set of scans

As a first example of the use of Filing, we will add the contents of four data files. These files are compatible, since they correspond to a Q-scan performed in four parts with different Q-ranges and with different Q-steps.

6.1. Opening the set of data files

Select the item "Open" from the main menu.
Use the item list "Current file format" to select the file format ILL TAS file.
Select the edit text "Directory name" and type the pathname of the directory Tutorial Examples
Press the tab key to jump to the edit text "Filename" and type Add_1.tas
Type a carriage return to end the edit session and click the button "Open".

Filing immediately reads the file and displays its contents in a graphic window. This window is divided into three parts: the plot region (top left), the caption region (top right), and the message region (bottom), which is presently empty.

The open-dialogue is shown again to allow you to successively open the remaining three files (Add_2.tas,Add_3.tas, and Add_4.tas). For each file, select the edit text "Filename" and move the cursor on the integer with the key "<". Note that typing a key will erased the selected one.

Once the four files have been opened, click the button "MENU1". Filing is now ready to process the series of opened files.

6.2. Adding files

Select the item "Add" from the menu "Filing". A dialogue box allows you to select the files to be added.

Check the boxes of the four files and click the button OK.

The compatibility tests are performed. In the present case, Filing displays a warning message and reports that the parameter ALF1 from file Add_2.tas is not compatible with the same parameter from file Add_1.tas:

A more detailed incompatibility report is given in the text window with, the parameter name, its values in the different input files and the tolerance value.

The user can now decide either to ignore the parameter discrepancy or to abandon the file processing. In the present case the parameter ALF1 is not an important one. Thus we can bo ahead by clicking the button "Continue". Note that the default output value of an "incompatible" parameter is that from the first file. However this value can even be edited using the item "Set resolution parameters" from the menu "Options".

The graphic window displays the resulting data set. Note that the comment region shows the equation of the add process.

To allow you keeping track of the various processing performed during a sesssion, the program writes to the text window:

the equation of the add process
both the fractional and the physical value of the tolerance-level on the abscissa.

the resulting data set.

To help you understand the add process, the table below shows how the resulting data set was built from the contents of the input files.

PNT	QL	CNTS	DELT	M1	Add_1	Add_2	Add_3	Add_4
1	-1.0702	0.00	1.000	379	X
2	-1.0599	1.00	1.000	382	X
3	-1.0503	2.00	1.414	383	X
4	-1.0401	2.00	1.414	382	X
5	-1.0301	2.00	1.414	382	X
6	-1.0253	0.00	1.000	382		X
7	-1.0200	5.00	2.236	381	X
8	-1.0198	3.00	1.732	380		X
9	-1.0153	2.00	1.414	384		X
10	-1.0101	92.00	9.592	1147	X	X		X
11	-1.0098	16.00	4.000	382			X
12	-1.0083	290.00	17.029	765			X	X
13	-1.0061	1695.00	41.170	764			X	X
14	-1.0049	2725.00	52.202	381		X
15	-1.0041	7129.00	84.433	759			X	X
16	-1.0022	20609.00	143.558	765			X	X
17	-1.0002	20854.00	144.409	382	X
18	-1.0000	45630.00	213.612	756		X		X
19	-0.9982	27676.00	166.361	380				X
20	-0.9962	25141.00	158.559	380				X
21	-0.9950	15789.00	125.654	380		X
22	-0.9941	14502.00	120.424	382				X
23	-0.9921	4768.00	69.051	379				X
24	-0.9901	3090.00	55.588	762	X			X
25	-0.9899	870.00	29.496	383		X
26	-0.9849	20.00	4.472	379		X
27	-0.9801	1.00	1.000	379		X
28	-0.9749	2.00	1.414	380		X

Note that while point 7 has a QL value very close to that of point 8, they were not added. Why? Because the differences between the two abscissa (here 0.0002) is larger than the absolute tolerance level of 0.000152. From the above table we can see that the abscissa differences are also small for point pairs 7-8, 10-11, 17-18, and 24-25. These differences are less than 0.0003, while the narrowest scan step in the input files is 0.002.

6.3. Changing the tolerance level on the scan step

Thus we may decide to relax the abscissa tolerance-level by a factor of two so that the pairs 7-8, 10-11, 17-18, and 24-25 can be added.

Select the item "Tolerances" from the menu "Options". Type 0.2 instead of 0.1 in the edit text "Scan step" of the dialogue "Tolerance levels". Note that the physical tolerance level of the scan step is the product of this "fractional" tolerance level and of the mean value of the scan steps.

Click the button OK and select the item "Add files" in the menu "Filing". Since the last resulting data set was not saved, a warning message is displayed.

Click the button "Yes", check the boxes of the four files in the dialogue "Add data sets" and click the button OK to add again the four file contents. Ignore the warning message about the file incompatibility. The new resulting data set is displayed and you can verify that the four pairs of nearly superimposed points have been combined.

PNT	QL	CNTS	DELT	M1	Add_1	Add_2	Add_3	Add_4
1	-1.0702	0.00	1.000	379	X
2	-1.0599	1.00	1.000	382	X
3	-1.0503	2.00	1.414	383	X
4	-1.0401	2.00	1.414	382	X
5	-1.0301	2.00	1.414	382	X
6	-1.0253	0.00	1.000	382		X
7	-1.0199	8.00	2.828	761	X	X
8	-1.0153	2.00	1.414	384		X
9	-1.0100	108.00	10.392	1529	X	X	X	X
10	-1.0083	290.00	17.029	765			X	X
11	-1.0061	1695.00	41.170	764			X	X
12	-1.0049	2725.00	52.202	381		X
13	-1.0041	7129.00	84.433	759			X	X
14	-1.0022	20609.00	143.558	765			X	X
15	-1.0001	66484.00	257.845	1138	X	X		X
16	-0.9982	27676.00	166.361	380				X
17	-0.9962	25141.00	158.559	380				X
18	-0.9950	15789.00	125.654	380		X
19	-0.9941	14502.00	120.424	382				X
20	-0.9921	4768.00	69.051	379				X
21	-0.9900	3960.00	62.929	1145	X	X		X
22	-0.9849	20.00	4.472	379		X
23	-0.9801	1.00	1.000	379		X
24	-0.9749	2.00	1.414	380		X

At this stage it would be a good practice to save the resulting data set as explained below.

7. Ex. 2: Subtracting a background

As a second example of the use of Filing, we will subtract a background scan (file Combine_2.tas) from a measurement scan (file Combine_1.tas) by means of the option "Combine files". These two files are compatible since they correspond to the same energy scan performed at two different temperatures, above and below a magnetic transition.

7.1. Opening the two data files

Select the item "Open" from the menu "File" (keyboard shortcut command-O). The open-file dialogue is shown.
Click the button "Remove all" to remove all the opened files from the list in the lower frame.
Use the popup menu "Current file format" to select the file format ILL TAS file.
Navigate to the directory Tutorial Examples.
Select the file combine_1.tas and click the button "Open". The file is immediatly read and its contents is displayed in the graphic window. The open-file dialogue is displayed again.
Select the file combine_2.tas and click the button "Open". The file is immediatly read and displayed in the graphic window. The open-file dialogue is displayed again.
Click the button "Done". Filing is now ready to process the two opened files.

7.2. Combining files

Select the item "Combine" from the menu "Filing". A dialogue box allows you to select the files to be combined.

Check the two boxes "Select".
Set the multiplying factor to be applied to the file combine_2.tas to -1.0 and leave the other at its default value of 1.0.
Click the button OK. The graphic window displays the resulting data set. The intensities from file combine_2.tas have been subtracted from the intensities of file combine_1.tas.

Click in the text window to make it the front one. The message below tells us that two points from file combine_2.tas were not transfered to the resulting data set since they have no equivalent in file combine_1.tas.

At this stage it would be a good practice to save the resulting data set as explained below.

7.3. Plotting data sets

Once the background has been subtracted, you may want to display all data sets on the same plot..

Select the item "Plot" from the menu "Plot". A dialogue box allows you to select the data sets to be ploted.

Check the three boxes "Plot" and click the button OK. The window contents display the selected data sets as shown below.

7.4. Saving data and results

Select the item "Save" from the main menu. A new menu bar allows you to decide what should be saved: the contents of the text window, the contents of the graphic window, the resulting data set, the "grouping data set", or the resolution parameters.

Note that the "grouping data set" is the resulting data set obtained after performing the option "Group points".

For example select the item "Graph". A dialogue allows you to edit the file name. In the example below, the contents of the graphic window will be saved to the PostScript file PGPlot.Plt.

7.5. Printing results

Select the item "Print" from the main menu. The graphic window is automatically printed on the current printer. Note that the current printer is defined by the environment variable PGPLOT_ILL_PRINT_CMD.

7.6. Quitting the program

Select the item "Exit" from the menu. A dialogue box allows you to confirm or to cancel.

Created by
Alain Bouvet and Alain Filhol L

ast update April 1997

Computing for Science

All Software

Filing for VMS/UNIX

A tutorial

Contents

1. Goals and definitions

Modifying a data set

Option "Add files"

Option "Combine files"

Filing limitations

2. Algorithms

Compatibility between plot-axis variables

Header parameters' compatibility

Superimposed and not-superimposed points

Option "Add files"

Option "Combine files"

3. Launching Filing

4. Setting general options

5. Setting tolerance levels

6. Ex. 1: Grouping a set of scans

6.1. Opening the set of data files

6.2. Adding files

6.3. Changing the tolerance level on the scan step

7. Ex. 2: Subtracting a background

7.1. Opening the two data files

7.2. Combining files

7.3. Plotting data sets

7.4. Saving data and results

7.5. Printing results

7.6. Quitting the program