The PSRIX Instrument

Introduction

PSRIX was developed as a part of the LEAP project and is in operation since January 2011. Since LEAP needs raw voltages from the participating telescopes, the instrument supported baseband mode since its inception. With the release of multithreaded DSPSR and the availability of multicore CPUs, realtime coherent dedispersion became possible as well. The instrument was expanded with a second ROACH board in late 2012 allowing the processing of 500 MHz bandwidth thereby supporting any receiver that outputs its signal at 750 MHz IF.

System outline

The system consists of three main parts - two ROACH boards, a 10GbE switch and a computer cluster. The ROACH boards are equipped with a dual-channel 1GSPS analoge-to-digital converter (called iADC), a programmable device (Xilinx Virtex 5 FPGA) and a modest CPU (PowerPC) running the Linux operating system. The iADC samples both polarisations of the astronomical signal using 8-bits and at a programmable rate. On the Virtex 5 FPGA, the digitised signal is then split in to several sub-bands using a polyphase filter bank. These sub-bands are then shipped out as UDP packets, with each sub-band tagged by a unique IP/port combination. The switch then streams one or two sub-bands to a node, where the actual data is handled as defined by the PSRIX operational mode.

The ROACH boards

 The ROACH board shown here without the enclosure.

There are three main setting (i.e combinations of gateware/clock) for the ROACH boards.

  • 16 channel, 16 IP version running at 800MHz for pulsar timing
  • 32 channel, 32 IP version clocked at 1024 MHz for LEAP mode
  • 32 channel, 32 IP firmware clock at 1000 MHz for receivers using 750 MHz IF/direct RF receivers (UBB/50cm).
Mode Firmware clock (MHz) subband/bandwidth (MHz)
Timing 16c16i1024s 800 25/200
LEAP 32c32i1024s 1024 16/128
50cm 32c32i1024s 1000 15.625/600
50cm 32c32i1024s 1024 16/128
UBB 32c32i1024s 1000 15.625/750
6cm/3.6cm/2cm 32c32i1024s100015.625/500

Realtime coherent dedispersion/folding or baseband recording is possible in all of the three settings. Depending on the type of observations, various sections of the cluster are active.

10gbE switch

HP5412zl serves to connect the ROACH units and the computer cluster. With the recent firmware upgrade the backplane is capable of 768Gpbs throughput. The switch also supports jumbo frames, VLANs and multicasting. Since the ROACH boards pack data in 8202-byte UDP packet, jumbo frame support is essential both in the switch and the receiving computer. VLAN support is required to isolate the astronomical data network as any accidental flooding of institute-wide computer network with the ROACH data can potentially bring down the infrastructure. Multicasting is supported in the switches via IGMP. This feature is very useful if there are multiple clients for a single source of data.

The computer cluster

The cluster of computers associated with the PSRIX instrument consists of three different sub-clusters.

The psrfb nodes: This is the oldest part of the computing infrastructure and has eight nodes. Each node consists of 2x 6-core AMD processors, and a 3Ware RAID card with 16TB of storage space. Currently this cluster is used both for Timing and LEAP observations.

The automatix nodes: these are 16 nodes, each with a 2x 16-core AMD CPU. These are dense computing units with 4x nodes in a 2U rack space. This is used for observations requiring UBB or 500MHz IF (eg. 2cm/3.6cm/6cm receivers.

Codeix blade system: This is the newest in the cluster. The unit consists of 8x high performance blades. The aim is to move all realtime folding/coherent dedispesion to this system.

 
pulsarinstruments/psrix_psrix2/psrix/overview_old.txt · Last modified: 2019/07/05 13:32 by ramesh     Back to top