Close This Window

Please download official ILL logos here

 

For using on the web or on a screenFor printing in high resolutionWhite version, for dark backgrounds

Download PNG

Download AI

Download white PNG

Download JPG

 

Download white AI

finddata

The Computing for Science (CS) group supports ILL scientists, students and visitors in a number of activities including data analysis, instrument simulation and sample simulation.

Back to ILL Homepage
English French Deutsch 

raw data

Searching for and retrieving data

Version 5  
R. E. Ghosh, November 2006 

Summary

IDA - Internet Data Access

The easiest way to examine these directories of data is to use IDA from your web browser.

On standard workstations data may be found in the directory trees from

 
/usr/illdata/data/instrument for the current cycle
/usr/illdata/data-1/instrument for the preceding cycle
/usr/illdata/975/instrument for cycle 975 (compressed data)
etc.

Certain instruments, e.g. D16 have very large numbers of files, which are subdivided:

/usr/illdata/data/d16/d16_0    for files 000001 to 009999
/usr/illdata/data/d16/d16_1 010000 to 019999 etc

Older data are stored in compressed format. Filenames terminate in .Z, and must be copied onto your own disks before using the uncompress command: the uncompressed file replaces the .Z file locally e.g. for a short sequence...

        %cp /usr/illdata/951/d11/00437*.Z .
%uncompress 00437*.Z

Data from Instruments - ASCII format (post 1994)

The raw data are automatically transferred and stored on a data-server serdon. This also holds data from the previous reactor cycle. The server path is defined on standard workstations and the directories can be examined with standard commands.

Catalogue

The regularly updated file /usr/illdata/data/DATA_CATALOG permits experimenters to find where their data files are stored.

Experimenters may use unix commands (cat,more,grep,awk...) to search files in the fashion of an SQL command.

Note: all input is case sensitive


To top

Examples

example 1:to find where are in5 data files with date:JUN
---------
Use grep command:
grep "in5" /usr/illdata/data/DATA_CATALOG | grep "JUN"
-----=====---------------------------=====
cycle exp numor numor from date to date identifier

953 in5 002656 002688 10-JUN-95 10:29 13-JUN-95 09:04 PRAGERH
953 in5 002690 002690 13-JUN-95 19:49 13-JUN-95 19:49 Herma
953 in5 002691 002694 13-JUN-95 20:51 14-JUN-95 07:54 HERMA
953 in5 002695 002722 14-JUN-95 14:25 19-JUN-95 08:45 DERIU MF
953 in5 002723 002770 19-JUN-95 11:28 23-JUN-95 08:34 COULOMMF
953 in5 002771 002796 23-JUN-95 14:43 27-JUN-95 09:03 J SuckJCC
953 in5 002797 002831 27-JUN-95 10:42 3-JUL-95 09:03 FGJPR AJD

example 2:to find in5 data files with date:JUN and experimenter name Herma.
---------
Use grep command:
grep "in5" /usr/illdata/data/DATA_CATALOG | grep "JUN" | grep "Herma"
-----=====---------------------------=====--------=======
953 in5 002689 002689 13-JUN-95 18:02 13-JUN-95 18:02 Herma HERM
953 in5 002690 002690 13-JUN-95 19:49 13-JUN-95 19:49 Herma

awk -F '/in5/&&/JUN/{print $0}' /usr/illdata/data/DATA_CATALOG
--------============------------------------------
953 in5 000009 002629 6-JUN-95 15:34 7-JUN-95 08:33 Cook Ferr
953 in5 002630 002655 7-JUN-95 12:39 10-JUN-95 08:50 ENSS MF
953 in5 002656 002688 10-JUN-95 10:29 13-JUN-95 09:04 PRAGERHERM
953 in5 002689 002689 13-JUN-95 18:02 13-JUN-95 18:02 Herma HERM
953 in5 002690 002690 13-JUN-95 19:49 13-JUN-95 19:49 Herma
953 in5 002691 002694 13-JUN-95 20:51 14-JUN-95 07:54 HERMA
953 in5 002695 002722 14-JUN-95 14:25 19-JUN-95 08:45 DERIU MF
953 in5 002723 002770 19-JUN-95 11:28 23-JUN-95 08:34 COULOMMF
953 in5 002771 002796 23-JUN-95 14:43 27-JUN-95 09:03 J SuckJCC
953 in5 002797 002831 27-JUN-95 10:42 3-JUL-95 09:03 FGJPR AJD

example 4:to find in5 data files with date:JUN or with date:JUL
---------
Use the awk unix command:

awk -F '((/in5/&&/JUN/)||(/in5/&&/JUL/)){print $0}' /usr/illdata/data/DATA_CATALOG
--------================================------------------------------
953 in5 000009 002629 6-JUN-95 15:34 7-JUN-95 08:33 Cook Ferr
953 in5 002630 002655 7-JUN-95 12:39 10-JUN-95 08:50 ENSS MF
953 in5 002656 002688 10-JUN-95
953 in5 002689 002689 13-JUN-95 18:02 13-JUN-95 18:02 Herma HERM
953 in5 002690 002690 13-JUN-95 19:49 13-JUN-95 19:49 Herma
953 in5 002691 002694 13-JUN-95 20:51 14-JUN-95 07:54 HERMA
953 in5 002695 002722 14-JUN-95 14:25 19-JUN-95 08:45 DERIU MF
953 in5 002723 002770 19-JUN-95 11:28 23-JUN-95 08:34 COULOMMF
953 in5 002771 002796 23-JUN-95 14:43 27-JUN-95 09:03 J SuckJCC
953 in5 002797 002831 27-JUN-95 10:42 3-JUL-95 09:03 FGJPR AJD
953 in5 002832 002850 3-JUL-95 10:25 4-JUL-95 22:54 BERND MF
953 in5 002851 002929 5-JUL-95 02:01 7-JUL-95 20:12 BA,JZLMF
953 in5 002930 002952 10-JUL-95 02:55 13-JUL-95 07:24 VRGZ MF
953 in5 002953 003007 13-JUL-95 10:38 19-JUL-95 15:27 CORNELGJK
953 in5 003008 003017 19-JUL-95 20:09 21-JUL-95 07:59 VRGZ MF
953 in5 003018 003020 21-JUL-95 08:57 21-JUL-95 11:26 HGBGJKHGB
953 in5 003021 003037 21-JUL-95 14:06 23-JUL-95 08:27 COOK COOK

example 5: to find all experimenters with name beginning by "O" or "o"
---------
use awk unix command:

awk -F' ' '($9~/^O/)||($9~/^o/){print $0}' /usr/illdata/data/DATA_CATALOG
-----------------=----------=--------------------------
951 d17 003411 003583 10-MAR-95 18:21 13-MAR-95 10:28 OTTEWILL
951 d7 001284 001286 04-MAR-95 15:28 04-MAR-95 20:18 OVCUR0
952 d17 004547 004598 18-APR-95 17:01 20-APR-95 08:23 oeser/mont
952 d1b 021515 022229 25-APR-95 10:46 27-APR-95 08:43 OULADDGOME
952 d1b 022230 022419 27-APR-95 10:14 29-APR-95 08:19 OULADDHUBE
952 d1b 022420 023238 29-APR-95 10:22 2-MAY-95 08:30 OULADDSAYE
952 d22 012759 012761 19-APR-95 22:25 19-APR-95 22:52 oeser
952 d22 012762 013159 20-APR-95 18:43 24-APR-95 19:43 Ottewill
952 d7 001534 001534 02-MAY-95 20:54 02-MAY-95 20:54 OFFENV
952 d17 004547 004598 18-APR-95 17:01 20-APR-95 08:23 oeser/mont
952 d1b 021515 022229 25-APR-95 10:46 27-APR-95 08:43 OULADDGOME
952 d1b 022230 022419 27-APR-95 10:14 29-APR-95 08:19 OULADDHUBE
952 d1b 022420 023238 29-APR-95 10:22 2-MAY-95 08:30 OULADDSAYE
952 d22 012759 012761 19-APR-95 22:25 19-APR-95 22:52 oeser
952 d22 012762 013159 20-APR-95 18:43 24-APR-95 19:43 Ottewill
953 d11 015394 015632 16-JUL-95 09:47 18-JUL-95 09:21 Ottewill

Data transcribed from pre-1994 binary

The archive and transcription tools were maintained on the Alpha-VMS cluster, and data have been progressively transcribed to ASCII files similar to those in current use. These data are held on-line in directories as described above. Notably for SANS, this archive matches current standards and data may be examined with present tools from year/cycle 743 and 771 for D11 and D17 respectively. Since these data have not all been catalogued requests should be sent to the Service Informatique concerning these data.

To top