Page 89 - ILLS Annual Report 2018

MODERNISATION PROGRAMMES AND TECHNICAL DEVELOPMENTS
86-87
IT service J.F. Perrin (ILL)
Since 2012, the ILL has experienced
a significant increase in the volume of experimental data generated by our instruments. In 2011, a full cycle of
50 days and 40 instruments generated roughly 1TB of data. Today, we are generating 2−3TB per day (figure 1). This increase is mainly due to improvements
to the guides and instruments as a
result of the Millennium and Endurance programmes. However, these fantastic results are not without consequence for the IT infrastructure, especially our network, storage and computing facilities.
The IT service has initiated three main activities in an attempt to accommodate this evolution.
1. Network
In August 2018, we refurbished our core network equipment. We are now able to reach a capacity of 100 Gb/s in the core. For now, this is probably not particularly noticeable to end-users but it allows us to further upgrade the network. In 2019, we intend to upgrade the distribution equipment (where the computers are connected) in order to offer 1Gb/s to everyone, while in more specific cases 10 Gb/s will be available for those instruments generating a large volume of data.
2. Storage
After six years of service, Serdon, the ILL’s well-known experimental data archive, became unable to cope with the volume of data and performance required. It was therefore refurbished in August 2018, after a year of preparation for the 1PB storage capacity that is now available. This new system was able to handle the pace of the third cycle in 2018, but some optimisation is still ongoing. We are also working closely with our instrument control colleagues to allow users immediate access to the data in the archive.
3. Computing facilities
This is the third pillar in our ability to process data.
The expansion in data volume has hit not only the ILL but all other European analytical facilities (neutron scattering centres, Synchrotrons, free electron lasers, etc.). We have agreed to tackle this subject together, with the support of the European Commission, as part of a collaboration project called PaNOSC. An ILL-only pilot began in 2018 on two instruments, while the European collaboration will officially start in January 2019. More detailed and practical information will be communicated during the course of 2019.
We hope that the ILL’s users will greatly appreciate these efforts, which will allow them to continue to deliver excellent science despite the ‘data deluge’ reported by the media.
Figure 1
Volume of experimental data/cycle.
www.ill.eu