zeroone wrote:
If every 2 cycles causes stability problems in the roll-off after your brick wall, use decimation between the brick wall and the roll-off.
The stability problems I referred to are a consequence of numerical accuracy. Discrete, high-order Butterworth filters require maintaining a lot of decimal places for certain cutoff frequencies. It's like a bifurcation event in chaos theory; the inaccuracies are rapidly magnified until you just get garbage out. I think that is why James only used a 4th order Butterworth filter and combined it with an Elliptic filter; he said he was able to do all the computations using only 32-bit precision, where as the 16th order Butterworth requires 64-bit.
Prior to experimenting with this filter, I was using a simple box filter and for the most part, it sounded just fine. For practically reasons, I may have to do what you suggested: putting a box filter or some other easy to compute filter in front of the Butterworth filter to try to get something out of all the available samples.
Edit: Another alternative is to use a lower order Butterworth and just apply it multiple times.
Edit 2: A 64-bit, 8th-order Butterworth filter remains stable for input sampling rates of 19687500.0 / 11.0 (NTSC) and 53203425.0 / 32.0 (PAL) and output sampling rates of 44100.0 or 48000.0. The filtering characteristics seem reasonable, but I do not have access to Matlab or any other special filter generating software; I wrote a test program that simply feeds the decimator with a slowly increasing frequency signal and it prints out the ratio of output-to-input gain. Timing still suggests that it should not eat up that much CPU, but I won't really know until I plug it into my emulator.