Digital decimation filters

We have five filters (low-pass, high-pass and three bandpass) for the five subbands. Each filter is a 31-tap FIR filter. The current design has very low leakage in the stop band, but falls off significantly in the passband, limiting the usable width. This means that after gain calibration the aliased out-of-band signal is scaled up, so that the relative aliasing becomes non-optimal.

We may want to optimise this behaviour and at the same time also take care of some other aspects, e.g. flattening the analogue system response to some degree and increasing the filter order (with improved quality), which becomes possible due to a new implementation.

Here is an example of one of the current filters (low-pass or high-pass, the two are mirror-reversed versions of each other) compared with a potential alternative design:

(Click on plot for full-size version.)

The top panel shows the filter coefficients (red for current design, green for an alternative). The middle panel shows the response in frequency space. The frequency axis is relative to the full band. We defined a certain edge range (grey, 70 MHz in this plot) as transition range that is considered lost and will not be optimised for. We see that the current filter rolls off significantly near this edge, which causes sensitivity loss at some point. The green curve falls off less, but has a larger ripple in the pass-band (peak-to-peak amplitude 5.3 dB). In the stop-band the green curve has stronger attenuation.

Most relevant is the last plot, which shows the stop-band folded into the pass-band as caused by the frequency decimation (keeping every second sample), after dividing by the pass-band level (bandpass calibration). This relative aliasing should be as low as possible in the good range (out of the grey area).

This alternative filter shown here is only meant as illustration. It was optimised for low max ripple in the pass-band and low max leakage in the stop-band, but not explicitly for low relative aliasing.

Which parameters do we want to optimise (for each of the five filters)?

1. transition region (should be narrow)
2. pass-band response (should be flat with low ripple)
3. stop-band response (relative aliasing should be low)
4. others?

Regarding the pass-band response and relative aliasing we can either minimise the maximum, or some average over the band.

Which relative weighting of these parameters should we use to define the optimal filter? Are there strict limits that must be met (e.g. absolute maximum acceptable leakage)? Can we find an optimal filter that is good for all applications?

Instead of defining a transition region that is excluded from the optimisation, we may also use some weighted combination of criteria over the entire band.

How much should we use the filter to equalise the analogue response?

Shall we leave the edge filters as high/lowpass filters (with aliasing from analogue filters) or shall we cut on both ends?

Because optimising for relative aliasing is numerically awkward, I start with (much easier) optimisimg for absolute aliasing while keeping the passband ripple under control. I set the usable bandwidth and then fit a range of solutions for aliasing/ripple.

Here is the result for 90% usable bandwidth:

The rectangle marks 1 and 3 dB maximum deviation of the passband gain and -30 and -45 dB for the aliasing (good for pulsar searching and timing). Recall that the filters are optimised for absolute aliasing, the relative aliasing is only plotted but not minimised. Results for various bandwidths can be found here.

Good filters are below/left the corners of the rectangle. We see that for this bandwidth we can keep the aliasing and the ripple low at the same time. Above 95% bandwidth (see PDF file) we do not even reach the less strict limits of -30/3dB.

These plots are for the current limit of 31 coefficients. We will probably be able to increase this number, with a significant effect on the filter quality. See here for an (arbitrary!) number of 41 coefficients. At the moment it is not clear how many will be possible.

Here are specific filters using 41 coefficients for 95% bandwidth, both with 2.8 dB maximum deviation in the passband. The first minimises the absolute aliasing, the second the relative. In terms of absolute aliasing they have -42.5/-43.9 dB, in terms of relative aliasing it is -39.7/-46.7 dB.

Here are curves for the 16-channel PFB coefficients used in the pulsar ROACH1 backend:

The top panel shows the coefficients (8 taps), the second shows the response. The third panel has the response after correcting for the in-band gain. The bottom panel shows the relative aliasing. Grey is for leakage from individual subbands, black is the sum over all out-of-band subbands. The vertical orange lines mark the range (-0.4,0.4) of the bandwidth, corresponding to 80%. At the edge the in-band attenuation is -1.25 dB (relative to the maximum), the relative aliasing is -16.2 dB. The range ±0.303 of the bandwidth (61%) has relative aliasing below -45 dB.

The main disadvantage of standard optimisation methods (e.g. least-squares fits, Remez algorithm) is that they constrain absolute leakage and/or passband ripple, but not relative leakage (after calibrating for passband gain). In the following I use a new algorithm that is based on least-squares optimisation. After one fit, it adapts the weight function to reflect the (current) passband gain, and then iterates, so that eventually the relative leakage is optimised. At the same time it can also downweight areas that are too good (and increase weights for areas that are too bad), so that the final result is not a least-squares solution, but an optimised min-max fit. This can be switched on or off for passband and stopband individually.

With this procedure I produced tradeoff plots for a range of bandwidths (for the S4 band with 35 coefficients). The box does again mark -30 and -45 dB relative aliasing, but now 2 and 6 dB for the passband ripple, because it is now parametrised as max-min (total range).

Given the very bumpy analog bandpass, very strict limits on the digital passband ripple do not seem to be of the highest importance. In the following I will thus use min-max for the relative aliasing, but least-squares for the passband. This effectively produces a larger gain decline near the passband edge, but this is acceptable.

I also tried to include the analog gain curve into the optimisation procedure so that the passband of a true source signal is flattened and that the relative aliasing is after correcting for digital and analog gain.

In order to determine the analog gain, I used most of our test observations of PKS 1934-63. The gains determined with the usual procedure are then combined first per antenna over all subbands, and then over all antennas, in both cases by correcting for different scales between observations and antennas and then taking the median of all measurements.

Here is the result for all antennas combined:

This plus results for individual antennas can be found in a PDF file.

The gain dip at the high frequency end can hardly be corrected by the digital filter, and the bumps generally make a min-max optimisation for the passband less efficient, so that least-squares should really be used for the passband. Also should we consider downweighting the edge region, because it is also affected by aliasing from the original sampling in combination with the analog filters. For the stop-band, including the gain dip can reduce the relative aliasing significantly.

For a comparison I ran the fits without the analog gains, and then the analog gains included. This is with min-max for the relative aliasing and least-squares for the passpand. The tradeoff plots do now include the rms ripple as well:

tradeoff for fits without analog gain
tradeoff for fits including analog gain

Without the analog gain, there seems to be a sweet spot for 92% bandwidth near -50dB leakage. This is how the filter looks like (second plot shows passband ripple in more detail, on a log and linear scale, with total scale modification to bring all to the same mean level):

But the analog gain does exist, and this is how the result looks like for the same filter, but with the analog gain included (not in the fit, only in the plots):

Not so great, and it would be good to correct some of the analog gain with the decimation filter.

For the fit including the analog gain, the min-max passband ripple cannot be reduced below 8.5dB, even for 90% bandwidth, which is a result of the low analog gain near 2.5 GHz. Looking at individual filters, one at 92% bandwidth with -50 dB aliasing is still looking acceptable. The rms passband ripple is about 1.2dB:

On the linear scale, the large-scale passband ripple looks a bit scary, even though it is probably still better than the analog gain itself or the analog gain with the current filter. The large waves are exactly known and reproducible, but the general astronomer would not like to see them anyway. Can they be corrected in the F engine? If not, we may have to reduce them.

The analog filters produce their own aliasing near the band edges 1750 and 3500 MHz. Using the S21 coefficients of the IF processor we can compute the aliasing by comparing the true response with the one mirrored at 1750 and 3500 MHz:

The upper panel shows the response function in red and the mirrored versions in blue and green. The sum is shown in red dashed. For the lower panels the mirrored ones are divided by the true one (difference on dB scale) and the result is plotted only for the band edges. We reach -30 dB relative aliasing at 1916 and 3442 MHz and -45 dB at ca. 2019 and 3401 MHz. The vertical lines indicate a loss of 10/20/30 % of the bandwidth.

We want to include the analog-aliasing in the total analog gain and exclude the range that is bad anyway from the optimisation of the digital filters. The limit for the latter is a bit arbitrary, let us try to exclude the > -30 dB range.

After consistency checks of a preliminary version (ver0) by Eddy, final filters were produced. Relative leakage was limited to -50dB (per frequency) including the effect of the analog bandpass. Near the edges the analog bandpass was cut with a lower limit to avoid upscaling too much. For the outer bands (LP and HP) the usable band was defined as 92%, for the bandpasses it is only 83%, because losses on both sides have to be accepted. This numbers were chosen to keep the passband ripple at a reasonable level, similar to the one already produced by the analog gain. For alternative filters we could trade passband flatness for usable bandwidth. With the available 35 coefficients, the following filters are optimal in this sense.

Here is one example plot from the actual fitting process, showing where the analog gain curve was limited:

All plots for S0 in one PDF file.

All plots for S1 in one PDF file.

All plots for S2 in one PDF file.

All plots for S3 in one PDF file.

All plots for S4 in one PDF file.

— Olaf Wucknitz 2022/01/26 11:36

Here are the corresponding plots for the filters used until end of May 2022.

All plots for S0 in one PDF file.
All plots for S1 in one PDF file.
All plots for S2 in one PDF file.
All plots for S3 in one PDF file.
All plots for S4 in one PDF file.

The new coefficients were tested in five 10-min scans, one per subband, of J1939-6342 on 8th June 2022. Here are combined results for all subbands. For each subband we have one observation with old and one with new filter. The new ones have timestamps in the title that start with 1654…, old ones are smaller. Note that not all antennas participated in all observations.

PDF with SEFD curves
PDF with bandpass curves
PDF with bandpass curves, amplitude corrected for digital decimation filter for a better comparison
PDF with SEFD and bandpass curves (dB)
PDF with SEFD and bandpass curves (dB), amplitude corrected for digital decimation filter for a better comparison