White Rabbit logo blue

White Rabbit Network at The Large High Altitude Air Shower Observatory, LHAASO (SyncTechnology)

To guarantee the angular resolution of the reconstructed tracks of cosmic rays, the over 8000 detectors distributed over a 1.3 Km2 area on a plateau at 4410 meters above sea level must be synchronised with the same sampling frequency and timestamp by a clock network. All detectors are self-triggered and the global timestamp is attached to each single hit that is used for further software trigger algorithm. In addition, the clock system should be able to: automatically calibrate and compensate the link delay due to different cable lengths and temperature variations, withstand harsh environments, be easy to install and maintain while remaining low cost.
White Rabbit technology, still in its very early stage, came to our sight from the very beginning of the LHAASO experiment and it finally became the key and essential supporting technology of the experiment. WRPC are integrated in all detectors and connected as a complete WR network with over 550 WR Switches separated in three layers; the Level-3 WRS are installed in electric-distribute-racks distributed over the field and connected by ~500 meters fiber cable to the Level-2 and hence Level-1 WRS in a electronics room. The GPS receiver and Rubidium Oscillator are also installed in the room to provide external UTC and frequency reference for the WR network.
Timing and data from the detectors are transmitted through the single WR connection thus only one photoelectric composite cable is needed for each detector, which greatly simplifies the cable deployment; the timing and data arriving Level-2 WRS are then separated through different ports to Level-1 WRS and commercial data switches respectively to overcome the 1Gbps bandwidth limitation of WRS.


The WR network has fully installed and operated since 2021.

Background

The Large High Altitude Air Shower Observatory (LHAASO) is one of the largest experiments in the history of cosmic ray physics. It is located 4,410 meters above sea level (ASL) on Mt. Haizi in Daocheng County, Sichuan Province, and covers about 1.36 km^2. It consists of three arrays: a shower particle detector array composed of 5,216 electromagnetic particle detectors and 1,188 ground-installed muon detectors covering 1 km^2; a water Cherenkov detector array covering 78,000 m^2; and a telescope array composed of 18 wide-field-of-view Cherenkov telescopes.
The key scientific goal is to search for cosmic ray origins. After 3 years of operation, LHAASO discovered many ultra-high-energy cosmic accelerator candidates in the Milky Way and detected the highest energy photon ever recorded, thus launching an era of "ultra-high-energy gamma astronomy". LHAASO also accurately measured the luminosity of the "standard candle" in the ultra-high-energy band and discovered gamma rays with energies over 1 quadrillion electron volts, which challenges the theoretical limit. Recorded the precise energy spectrum for the highest-energy gamma-ray radiation from Gamma-Ray Burst (GRB) 221009A, the brightest GRB observed to date.

Challenge

  • More than 8000 detectors distributed in a field of 1.3Km2 need to be synchronised
  • High altitude wild field deployment with daily and yearly temperature modulation
  • The management of hundreds of WRS

Solution

By introducing 3 layers of WRS, the network can support more than 8000 nodes; the data stream is separated from timing stream at Level-2 WRS and further transmitted to high performance commercial switches to meet the bandwidth requirement.

The hardware of WRS is modified to improve the heat dissipation performance that can work without fans, an optical management port is added to support fiber connection; other modifications include:

  •  Improved device temp. range to -20 °C ~ 70 °C
  • Improved low jitter clock circuit
  • Improved placement and layout
  • 18 WR ports with light guide for better EMC
  • Power button on front panel

Despite the hardware modifications, the fan-less version of WRS still keep software and firmware compatible with standard WRS.

White Rabbit logo blue