Digital decimation filters

We have five filters (low-pass, high-pass and three bandpass) for the five subbands. Each filter is a 31-tap FIR filter. The current design has very low leakage in the stop band, but falls off significantly in the passband, limiting the usable width. This means that after gain calibration the aliased out-of-band signal is scaled up, so that the relative aliasing becomes non-optimal.

We may want to optimise this behaviour and at the same time also take care of some other aspects, e.g. flattening the analogue system response to some degree and increasing the filter order (with improved quality), which becomes possible due to a new implementation.

Here is an example of one of the current filters (low-pass or high-pass, the two are mirror-reversed versions of each other) compared with a potential alternative design:

(Click on plot for full-size version.)

The top panel shows the filter coefficients (red for current design, green for an alternative). The middle panel shows the response in frequency space. The frequency axis is relative to the full band. We defined a certain edge range (grey, 70 MHz in this plot) as transition range that is considered lost and will not be optimised for. We see that the current filter rolls off significantly near this edge, which causes sensitivity loss at some point. The green curve falls off less, but has a larger ripple in the pass-band (peak-to-peak amplitude 5.3 dB). In the stop-band the green curve has stronger attenuation.

Most relevant is the last plot, which shows the stop-band folded into the pass-band as caused by the frequency decimation (keeping every second sample), after dividing by the pass-band level (bandpass calibration). This relative aliasing should be as low as possible in the good range (out of the grey area).

This alternative filter shown here is only meant as illustration. It was optimised for low max ripple in the pass-band and low max leakage in the stop-band, but not explicitly for low relative aliasing.

Which parameters do we want to optimise (for each of the five filters)?

1. transition region (should be narrow)
2. pass-band response (should be flat with low ripple)
3. stop-band response (relative aliasing should be low)
4. others?

Regarding the pass-band response and relative aliasing we can either minimise the maximum, or some average over the band.

Which relative weighting of these parameters should we use to define the optimal filter? Are there strict limits that must be met (e.g. absolute maximum acceptable leakage)? Can we find an optimal filter that is good for all applications?

Instead of defining a transition region that is excluded from the optimisation, we may also use some weighted combination of criteria over the entire band.

How much should we use the filter to equalise the analogue response?

Shall we leave the edge filters as high/lowpass filters (with aliasing from analogue filters) or shall we cut on both ends?

Because optimising for relative aliasing is numerically awkward, I start with (much easier) optimisimg for absolute aliasing while keeping the passband ripple under control. I set the usable bandwidth and then fit a range of solutions for aliasing/ripple.

Here is the result for 90% usable bandwidth:

The rectangle marks 1 and 3 dB maximum deviation of the passband gain and -30 and -45 dB for the aliasing (good for pulsar searching and timing). Recall that the filters are optimised for absolute aliasing, the relative aliasing is only plotted but not minimised. Results for various bandwidths can be found here.

Good filters are below/left the corners of the rectangle. We see that for this bandwidth we can keep the aliasing and the ripple low at the same time. Above 95% bandwidth (see PDF file) we do not even reach the less strict limits of -30/3dB.

These plots are for the current limit of 31 coefficients. We will probably be able to increase this number, with a significant effect on the filter quality. See here for an (arbitrary!) number of 41 coefficients. At the moment it is not clear how many will be possible.

Here are specific filters using 41 coefficients for 95% bandwidth, both with 2.8 dB maximum deviation in the passband. The first minimises the absolute aliasing, the second the relative. In terms of absolute aliasing they have -42.5/-43.9 dB, in terms of relative aliasing it is -39.7/-46.7 dB.

Here are curves for the 16-channel PFB coefficients used in the pulsar ROACH1 backend:

The top panel shows the coefficients (8 taps), the second shows the response. The third panel has the response after correcting for the in-band gain. The bottom panel shows the relative aliasing. Grey is for leakage from individual subbands, black is the sum over all out-of-band subbands. The vertical orange lines mark the range (-0.4,0.4) of the bandwidth, corresponding to 80%. At the edge the in-band attenuation is -1.25 dB (relative to the maximum), the relative aliasing is -16.2 dB. The range ±0.303 of the bandwidth (61%) has relative aliasing below -45 dB.

The main disadvantage of standard optimisation methods (e.g. least-squares fits, Remez algorithm) is that they constrain absolute leakage and/or passband ripple, but not relative leakage (after calibrating for passband gain). In the following I use a new algorithm that is based on least-squares optimisation. After one fit, it adapts the weight function to reflect the (current) passband gain, and then iterates, so that eventually the relative leakage is optimised. At the same time it can also downweight areas that are too good (and increase weights for areas that are too bad), so that the final result is not a least-squares solution, but an optimised min-max fit. This can be switched on or off for passband and stopband individually.

With this procedure I produced tradeoff plots for a range of bandwidths (for the S4 band with 35 coefficients). The box does again mark -30 and -45 dB relative aliasing, but now 2 and 6 dB for the passband ripple, because it is now parametrised as max-min (total range).

Given the very bumpy analog bandpass, very strict limits on the digital passband ripple do not seem to be of the highest importance. In the following I will thus use min-max for the relative aliasing, but least-squares for the passband. This effectively produces a larger gain decline near the passband edge, but this is acceptable.

I also tried to include the analog gain curve into the optimisation procedure so that the passband of a true source signal is flattened and that the relative aliasing is after correcting for digital and analog gain.

In order to determine the analog gain, I used most of our test observations of PKS 1934-63. The gains determined with the usual procedure are then combined first per antenna over all subbands, and then over all antennas, in both cases by correcting for different scales between observations and antennas and then taking the median of all measurements.

Here is the result for all antennas combined:

This plus results for individual antennas can be found in a PDF file.

The gain dip at the high frequency end can hardly be corrected by the digital filter, and the bumps generally make a min-max optimisation for the passband less efficient, so that least-squares should really be used for the passband. Also should we consider downweighting the edge region, because it is also affected by aliasing from the original sampling in combination with the analog filters. For the stop-band, including the gain dip can reduce the relative aliasing significantly.

For a comparison I ran the fits without the analog gains, then applied them after the fits, and finally ran fits with the analog gains included. This is with min-max for the relative aliasing and least-squares for the passpand:

tradeoff for fits including analog gain

For the fit including the analog gain, the min-max passband ripple cannot be reduced below 8dB, even for 90% bandwidth. Interestingly, there is a point at 92% bandwidth and ca. -62.5dB aliasing with about the same min-max passband ripple. The rms passband ripple is about 1.5dB: