====== Noise variations ======

In beamformed observations of Ter5 ([[https://mattermost.mpifr-bonn.mpg.de/s-band-survey/pl/71om789hxtna7m9a4cxx7cwf9e|mattermost link]]) peaks and variations of the noise level of coherent and incoherent beams were found. Investigations using the interferometric data confirmed their presence, albeit with lower time resolution and therefore less detail.

It became clear that most antennas are behaving normally, and only a few are causing these problems. No strange effects are found in the cross-correlations, the only parameter that seems to vary is the system noise per antenna.

Going through all existing data manually would be too time consuming, so we have to find a way to detect problems with some automatic statistics. Here is an attempt in that direction.

I take the autocorrelations of all antennas. Gaps in the data (between scans) are filled but masked. Data are then normalised (per scan, because different targets can have very different noise levels) as follows: The median of the XX and YY levels are taken (per frequency), and XX and YY are normalised by this. XY is normalised with the geometric mean of the two. This brings the median noise in X and Y to a value of unity. The XY correlation can have arbitrary phase due to delay differences. Because of this, it is rotated using the median of XY (real and imaginary). This means XY will generally be close to positive real. The amplitude can differ from antenna to antenna, but it should be relatively constant. The imaginary part should be constant close to zero. Finally, 1 is subtracted from XX and YY to bring them nominally to zero.

===== All results =====

[[https://cloud.mpifr-bonn.mpg.de/index.php/s/eHdawNC3DZ6Hex8|Results from all observations that have been processed (for this purpose) so far are available at this link.]]

A description of the individual plots in the PDF files can be found below.

In addition to files for each dataset, there is a file [[https://cloud.mpifr-bonn.mpg.de/index.php/s/wnCcqweKcH8rMyg|ALLFIRST_auto_noisevar.pdf]], which consists of the first overview pages from all PDF files (from all data sets). For some reason the colours look a bit strange in this file.

**Note that displaying the PDFs in the browser may interpolate the colours, e.g. between stations. If that is the case, please download and display offline.**


===== Individual plots =====

For an overview of all antennas, I take the median of these normalised data over all frequencies and plot them per antenna and time. An example for the Ter5 data set 1648081798, in which the variations were found originally, is shown here:

{{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0_freqmed.png?direct&800|median over frequencies}}

The total power measurements in XX and YY are easiest to interpret. Without variation, they would be zero. Over longer scans (marked by vertical green lines), we sometimes see gradients. These can be due to the changing elevation and are not of concern. Some antennas, in this observation most prominent in YY, have stronger variations, in particular intermittent high levels and sudden changes of levels.

real(XY) shows the linear polarisation level of the noise. It is generally relatively constant over time. The orthogonal component, imag(XY), on the other hand, often changes rapidly with the variations in YY.

In order to see the frequency dependence for all antennas, I also produce moments for XX and YY of different order. These are computed around the normalised data, which means around the median. Different orders are sensitive to different effects:

{{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0_moment1.png?direct&500|moments of order 1}}
{{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0_moment2.png?direct&500|moments of order 2}} {{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0_moment3.png?direct&500|moments of order 3}} {{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0_moment4.png?direct&500|moments of order 4}} {{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0_moment5.png?direct&500|moments of order 5}} {{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0_moment6.png?direct&500|moments of order 6}}

Different colormaps are used for the even and odd orders, because even ones are always non-negative. Problematic antennas show up differently in the different orders.

Finally we have plots in time and frequency for all antennas.

m001 is looking normal (some frequency ranges are affected by RFI): \\
{{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0_00_m001.png?direct&800|m001}}

Some antennas (m030,m050,m061) have jumps in the level. We show m030 as example: \\
{{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0_16_m030.png?direct&800|m030}}

Some antennas (m031,m036,m051,m053) have peaks that look like dispersed (any sign) or like FRBs. Here they are much stronger in YY. We show m031 as example: \\
{{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0_17_m031.png?direct&800|m031}}

m038 shows jumps in a frequency slope of the level: \\
{{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0_23_m038.png?direct&800|m038}}

Some antennas (m054,m063) show combinations of effects. We show m054 as example: \\
{{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0_32_m054.png?direct&800|m054}}

m059 has a weirdness of its own: \\
{{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0_35_m059.png?direct&800|m059}}



===== All plots combined =====

A pdf with all plots as individual pages can be found {{:projects:mkat_sband:pub:auto_noisevar_1648081798_sdp_l0.pdf|here.}}