The PSRIX Instrument

PSRIX was developed in the context of the LEAP project and is operational since January 2011. Since LEAP needs raw voltages from the participating telescopes, the instrument supported baseband mode since its inception. With the release of multithreaded DSPSR and the availability of multicore CPUs, realtime coherent dedispersion became possible as well.

The system consists of three main parts - two ROACH1 boards, a 10GbE switch and a few high performance computers. The ROACH boards are equipped with a dual-channel 1GSPS analoge-to-digital converter (called iADC), a programmable device (Xilinx Virtex 5 FPGA) and a modest CPU (PowerPC) running the Linux operating system. The iADC samples both polarisations of the astronomical signal using 8-bits and at a programmable rate. On the Virtex 5 FPGA, the signal is digitised and may or may not be (depending on the observer requirement) split in to sub-bands. The polyphase filter bank technique is used not he FPGAs to create sub-bands if needed. The data are then output as UDP packets tagged by an unique IP/port combination. The switch then streams the data to one or few nodes, where the actual data is handled as defined by the PSRIX operational mode.

The image on the right shows various parts of the PSRIX system. The two PSRIX ROACH1 boards can be run in parallel to realise timing and baseband modes simultaneously. The second ROACH board can be configured to send out data without the subbands, allowing 1ns time resolution. The high speed data is then routed via the 40Gb switch and captured by the paf0/paf1 nodes. These nodes have access to the high performance beegfs filesystem, which can sustain 4GB/s transfer rates.

As shown, the system can process a maximum of 500MHz, depending on the receiver. For the large bandwidth secondary focus receivers (eg, S45, S20 etc.), only the 500MHz band allowed by the MultiFiba is processed by the ROACH boards.

There are three combinations of gateware/clock for the ROACH boards.

• 16 channel, 16 IP version running at 800MHz for pulsar timing
• 32 channel, 32 IP version clocked at 1024 MHz for LEAP mode
• 32 channel, 32 IP firmware clock at 1000 MHz for receivers using 750 MHz IF and pulsar search modes.

Realtime coherent dedispersion/folding and/or baseband recording is possible in all of the three settings. Depending on the type of observations, various sections of the cluster are active.

HP5412zl serves to connect the ROACH units and the computer cluster. With the recent firmware upgrade the backplane is capable of 768Gpbs throughput. Since the ROACH boards pack data in 8202-byte UDP packet, jumbo frame support is essential both in the switch and the receiving computer.

The cluster of computers associated with the PSRIX instrument consists of three different sub-clusters.

The automatix nodes: these are 16 nodes, each with a 2x 16-core AMD CPU. These are dense computing units with 4x nodes in a 2U rack space. These nodes are use for realtime coherent dedispersion and folding of pulsar data. Additionally a software based filter bank is also possible. Recording the sub-bands is the third possibility.

Codeix blade system: The unit consists of 8x high performance blades and used as a stand-by for the automatix timing mode.

This is most flexible mode of the instrument, and uses the most disk space. Depending on the firmware and sampling rate, each node receives either 64MB/s, 100MB/s or 125MB/s data streams, that is recorded to the disk. This data processed offline depending on the scientific need. The data can now be folded with a give number of bins, converted to pulsar search data or searched for giant pulses from a known pulsar.

For known pulsars, DSPSR is used to coherently dedisperse and average over a given time, write out all single pulses, or search for pulses that only have a specified signal-to-noise ratio. The last technique is used to search for giant pulses from fast spinning pulsars like Crab or B1937+21. In the post processing stage, single pulses from all sub-bands can be combined using PSRCHIVE.

For pulsars with spin period greater then 300ms, the signal can coherently dedispersed (number of channels depend on the DM of the pulsar), and the single pulses can be written out with full polarisation information and with a predefined number of phase-bins. One file per single pulse is created on disk. Usually the observation length is adjusted such that the number of files in a directory is ~10000.

Among the three modes, this is the least flexible and consumes the least amount of disk space. In this case, the pulsar signal is coherently dedispersed and averaged over 10 seconds. One file every 10 seconds is written to disk, and can have all four Stokes parameters. The number of frequency channels depends on the DM of the pulsar.

1. OLD Overview of the instrument