Filter Oscillation Tech Talk....

[Hollow Sun]

From Gwenhwyfaer....

As regards the self-oscillation issue, I think it has to do with the filter topology Alesis use and the difference between fixed and floating point DSP. Here's my (albeit amateur and limited) understanding of that:

With fixed point DSP, your signal varies between -1 and +1, but typically you'd reserve a couple of bits for headroom so it could actually go from -8 to +8 and then clamp it to -1 or +1 at the output (either by just hard clipping it, or by applying a saturation curve). But the key thing is that whatever you do to the signal on the way through it never steps outside that -8..+8 boundary, which means that you have to worry a lot less about keeping everything else in check. That's actually pretty similar to the way analogue circuits behave - no matter what you do, no matter where you are in the circuit, you can't ever step outside the bounds of the power rails.

But for floating point DSP, even if you clamp it at the output, your signal can end up varying over the whole capacity of the floating point range (typically -10^38 to +10^38), and if you're not careful about where you limit it, your signal can end up running riot over the +/-1 boundary. So long as your signal path is always going forward, that's not too much of an issue - the problems come when you start fiddling with feedback loops...

And that's the thing about resonant filters. Resonance in a filter is pretty much the same as resonance in anything else - the frequency to which the filter is "tuned" will ring when an impulse is applied to that filter. Self-oscillation occurs when the noise in the filter circuit itself provides a continuous stimulation of the filter, and its resonance is turned up high enough that it will "ring" with even that tiny input. (Think of guitar or mike feedback - it's the same idea.) In essence, what happens is that the circuit is feeding back on itself until it reaches a steady state, which is defined by where the signal ends up being limited within its signal path (each diode or transistor in a filter will provide a limiting step, for example).

But of course, a floating-point digital filter doesn't have those internal limiting steps unless someone inserts them... but inserting what's equivalent to a lookup table or a cubic function at every stage gets computationally expensive damn quickly, and also has the potential to introduce unbounded aliasing! But without it, certain filters (in particular the SVF, one of the most common digital filters because it's otherwise pretty easy to control) will just resonate themselves to bits (metaphorically speaking) unless you take steps to limit the resonance level to - well, probably to about 80%, which is what Alesis appear to have done. Any higher and at certain points your carefully crafted wave will just turn into unusable digital mush. And hence even the RP filter, which will whistle away quite happily on the Micron, won't reach self-oscillation on the Fusion.

Did Alesis make a mistake when they chose floating-point DSPs for the Fusion? No, I don't think so - the Texas TMS320c6713 is an incredibly powerful part (1350 MIPS per chip at its maximum, and if anything can take advantage of perpetual parallelism it's generating a bunch of synth voices!); also, in the article that's been floating around for a couple of years describing the Fusion's genesis, Alesis say that they initially started with the "wrong platform", and had to change halfway through. I'd hazard a guess that the "wrong platform" in question might have been one of the fixed point 320s. Could be wrong, though. In any case, that's the only downside of the filter implementation - it sounds amazing everywhere else, smooth, pure and as "analogue" as one could want. And let's not forget that there were a bunch of analogue filters that didn't self-oscillate either - the SEM, the EDP Wasp, the Alpha Juno, even the TB303 apparently (though I'd have to say that's hearsay - I guess it gets shriekily and grungily resonant, but doesn't go off on its own)... it's far from a defining characteristic of analogue filters that they self-oscillate. Not to mention the filters that go into shrieking distorted self-oscillation far too early, like the MS20 one; or the ones where turning up the resonance causes the proportionate disappearance of the passband signal; or the ones that can't accurately track the keyboard to save their lives... not every analogue filter is a Moog filter.
And even the Moog filter is an ugly hack, in electronic terms. It just happens to sound gorgeous.

(I should point out that I've never actually seen most of the gear I just namechecked in the flesh; I've just heard about it all, and heard a few samples. I've written a couple of filters in the past, and kind of remember how, and enjoyed it; but... mostly the whole DSP thing is a bit of an obsession of mine. I've been designing digital synths in my head for half my life, but none of them have yet made it into the light of day. So any correspondence between my theories and reality is entirely accidental. )

[Hollow Sun]

From Nuno

Hi all First post here

gwenhwyfaer said:

With fixed point DSP, your signal varies between -1 and +1, but typically you'd reserve a couple of bits for headroom so it could actually go from -8 to +8 and then clamp it to -1 or +1 at the output (either by just hard clipping it, or by applying a saturation curve). .

Gwenhwyfaer, I'm sorry, but what do you mean here? When you say your signal varies +1 and -1 do you mean that this is the smallest possible variation? Where do you get +8 and -8? And what do you mean with the clamping +1 and-1 at the output?

[Hollow Sun]

From Gwenhwyfaer

Say we have a 24-bit DSP, and 20-bit D/A convertors. We'd want to stick the 20 bits of D/A range as a nominal -1.000 to +1.000 (where the lowest voltage coming out of the DAC is -1 and the highest is +1). However, occasionally (when we're adding a number of signals, say, or when we're calculating a feedback loop for a filter), we might need intermediate products to be a bit more than +1 or a bit less than -1... but not too much, otherwise things get hopelessly out of control. So we take the middle 20 bits of our 24-bit word as what the DAC will actually output, and we have 3 bits at the top (so that we can represent values up to 8 times as large as our DAC can output, for intermediate calculations) and one bit at the bottom so that we can round accurately. In fixed point terms, we have 3 bits for the integer part and 21 bits for the fractional part, the most significant 20 of which are sent to the DAC to give a nominal -1.000 to +1.000 output range. -1 to +1 isn't the smallest variation, it's a peak-to-peak swing... just as it is in floating point.

However, what we don't want is to have a sample lying outside the -1 to +1 range "escaping" to the DAC! So in that situation, it's comparatively easy to test the top 3 bits to see whether it is actually outside the range or not, and if it is set the rest of the word to the highest or lowest value it can possibly be. That's hard clipping (or "clamping", as I referred to it). A slightly more sophisticated scheme would use a lookup table which mapped -8.000 to +8.000 onto a tanh() curve (a transistor's response follows such a curve) to produce a saturation or soft-clipping effect. The fact that we're using 24-bit words, couped with the fact that the DSP automatically sets a result to +/-8.000 when it would otherwise overflow, is what keeps us from escaping a "sane" range.

Think of it as a bit like an analogue circuit - traditional analogue modules use a signal standard of 10v (+/-1.000) peak-to-peak, but take power supply rails of -15v and +15v (+/-3.000). ...With the disclaimer that I don't know anywhere near enough about analogue circuits to know if that's actually how things work or not - one of my personal goals for 2007 is to buy a soldering station, a copy of Horowitz and Hill and a PAiA Fatman, with the aim of coming to understand all three.

Does that make it any less confusing? Sorry for the vagueness of explanation - I'm not the best person at actually getting across what I'm on about!

(And if anyone does DSP programming for a living, please tell me where and how I've gone wrong or misunderstood something.)

[Hollow Sun]

From Nuno

gwenhwyfaer said:

Say we have a 24-bit DSP, and 20-bit D/A convertors. We'd want to stick the 20 bits of D/A range as a nominal -1.000 to +1.000 (where the lowest voltage coming out of the DAC is -1 and the highest is +1)

Thanks for the reply

So You meant Volts? It would be easier to get there if you actually mentioned the unit. lol

I got your point. I am actually a software developer and lately I've been working a lot with ADC and DAC so it's not all chinese to me. It is curious that in my field we don´t really do anything like that. DAC will have a maximum value it can output and exceding it will really do nothing else than outputing the maximum Volts or Amps that it can actually output.

gwenhwyfaer said:

(And if anyone does DSP programming for a living, please tell me where and how I've gone wrong or misunderstood something.)

lol, I do develop software, but things vary a lot depending on the application and type of hardware use, so I´m being caucious here as I'm actually not sure if you are right or not.

Anyway, where does the connection with the self oscilation filter comes from? Why do you say that the self oscilation may be lost by using the floating point arithmetics?

Thanks again and sorry for hijacking the thread.

[Hollow Sun]

From Gwenhwyfaer

nuno said:

It is curious that in my field we don´t really do anything like that. DAC will have a maximum value it can output and exceding it will really do nothing else than outputing the maximum Volts or Amps that it can actually output.

So it will in audio processing - but the intermediate values, used in the calculation, might be outside the range of the DAC, and yet need to be preserved to retain the integrity of the sound. Consider the DSP56000 - it uses a 24-bit word length representing -1.0 to +1.0, but has a 56-bit accumulator divided into 8 "integer" bits (-256.0 to +256.0) and a 48-bit fraction (so that the exact result of a 24-bit multiply can be preserved - the "normal" 24 bits are the high-end ones). Or the EMU10K1, which likewise has a 32-bit word and a 67-bit accumulator.

Anyway, where does the connection with the self oscilation filter comes from? Why do you say that the self oscilation may be lost by using the floating point arithmetics?

Well, basically, it's not lost, it's just that it can't be permitted. Self-oscillation is basically when resonance is allowed to go out of control; in a fixed-point processor or an analogue filter that's OK, because there's a natural limit (the limits of the poles) to how far resonance can go out of control, and that limit is quite low. So you get a pleasing whistle at the output. In a floating-point system, there's no such limit - so if you let self-oscillation go out of control there, you pretty quickly end up with your filter oscillating between 10^38 and -10^38 - which as you can imagine, isn't pretty on a scale of -1 to +1. So the only practical option is to stop resonance from going so high as to escape out of control - which means limiting the range of the resonance control, and no self-oscillation. Not because it can't be done, but because were it permitted it would be horrid and useless, not the charming little whistle of an analogue filter.

Whilst we're on the subject... I did a couple of experiments with my Micron last night. It has 6 low pass filters - ob (12dB/oct), tb (18dB/oct), mg, jp, rp (24dB/oct) and al (48dB/oct). Of those, only the mg and tb filters go into genuine self-oscillation - the kind that continues when the signal is taken away - and the tb only does so at higher frequencies (it dies away below about 250Hz). And because it's a digital circuit, even the mg filter needs an initial impulse to get it going. The ob, jp and al filters never get close to self-oscillation, and although the rp will give the appearance of self-oscillation in the presence of a signal, it won't sustain it once the signal goes away.

(And according to Gordon Reid, the mg filter should respond rather more like the tb filter, with self-oscillation decreasing from about 1kHz down until it disappears at about 200Hz.)

[Hollow Sun]

From Nuno

gwenhwyfaer said:

nuno said:

It is curious that in my field we don´t really do anything like that. DAC will have a maximum value it can output and exceding it will really do nothing else than outputing the maximum Volts or Amps that it can actually output.

So it will in audio processing - but the intermediate values, used in the calculation, might be outside the range of the DAC, and yet need to be preserved to retain the integrity of the sound.

But the value passed on to the DAC can easily be limited to acceptable values in the software, don't you think?

gwenhwyfaer said:

nuno said:

Anyway, where does the connection with the self oscilation filter comes from? Why do you say that the self oscilation may be lost by using the floating point arithmetics?

Well, basically, it's not lost, it's just that it can't be permitted. Self-oscillation is basically when resonance is allowed to go out of control; in a fixed-point processor or an analogue filter that's OK, because there's a natural limit (the limits of the poles) to how far resonance can go out of control, and that limit is quite low. So you get a pleasing whistle at the output. In a floating-point system, there's no such limit - so if you let self-oscillation go out of control there, you pretty quickly end up with your filter oscillating between 10^38 and -10^38 - which as you can imagine, isn't pretty on a scale of -1 to +1. So the only practical option is to stop resonance from going so high as to escape out of control - which means limiting the range of the resonance control, and no self-oscillation. Not because it can't be done, but because were it permitted it would be horrid and useless, not the charming little whistle of an analogue filter.

I still fail to see why I must be missing something. Can't whatever intermediate values, large as they may be, be sent within an acceptable range to the DAC?

gwenhwyfaer said:

nuno said:

Whilst we're on the subject... I did a couple of experiments with my Micron last night. It has 6 low pass filters - ob (12dB/oct), tb (18dB/oct), mg, jp, rp (24dB/oct) and al (48dB/oct). Of those, only the mg and tb filters go into genuine self-oscillation - the kind that continues when the signal is taken away - and the tb only does so at higher frequencies (it dies away below about 250Hz). And because it's a digital circuit, even the mg filter needs an initial impulse to get it going. The ob, jp and al filters never get close to self-oscillation, and although the rp will give the appearance of self-oscillation in the presence of a signal, it won't sustain it once the signal goes away.

I have a Micron too Great little synth

[Hollow Sun]

From Gwenhwyfaer

nuno said:

I still fail to see why I must be missing something. Can't whatever intermediate values, large as they may be, be sent within an acceptable range to the DAC?

But the intermediate values never get sent to the DAC at all - they're the values in the "in-between" bits of the filter, for example between the first and second stages or in the feedback loop itself. They're used to work out what the final output is (eg. a product of the difference between two comparatively large values - quite frequent in filters), and quite often when they're within a sensible range (a bit bigger than the DAC, but not much), the final value will end up being within the range of the DAC, and will stay within that range, without having to make any other corrections.

But when the intermediate values can be unbounded, quite often they end up heading off to the extremes of the floating point range... which is an awful lot bigger, of course. When they finally escape their range they become +infinity and no further calculations are possible on them... but even before that, when they're at the extremes of the FP range, the precision of the model is completely destroyed because the all-important +1 range is about 80 bits off the low end of the mantissa. Thus either the final calculation ends up as a flat 0, or as a propagated infinity - and no amount of massaging can repair that.

In contrast, fixed-point calculations never lose precision on their LSBs... but then, neither would floats if their range was similarly limited (limiting a 7-bit exponent to a maximum value of +4 would have more or less the same effect as the EMU10k1's accumulator). Whilst expanding the "normal" range of signals up to +2^123 or so (about 4 or 6 from the highest possible exponent) would seem to get round it, that would cause similar problems any time you had to multiply two signals together... so for a filter, the choices are to cap the resonance range (which is free), or put a comparatively expensive range-limiting step after each MAC that could allow the result to grow arbitrarily.

There is one other possible solution. In IEEE-754, the "actual" value of the exponent for a number is the contents of the exponent field minus a "bias" - 127 in 32-bit floats (with 8-bit exponents). If the bias could be adjusted manually in the FPU - even as a rather expensive operation - it would solve a lot of problems; for an 8-bit exponent, an audio application could set it to 250 (giving a maximum range of +32.0). Likewise, it'd be nice to see a switch that forced the FPU to "autocorrect" +infinity to +MAX_FLOAT. Those two modifications would enable floating-point DSPs to behave a lot more like fixed point DSPs where it matters. (Of course, both of these possibilities might very well be present and correct already in the 320c67 - I'll have to check. )

[Hollow Sun]

This thread has been moved from another where the discussion went slightly off-topic

Please continue your discussions here - fascinating reading!


Steve

[nuno]

admin said:

From Gwenhwyfaer

nuno said:

I still fail to see why I must be missing something. Can't whatever intermediate values, large as they may be, be sent within an acceptable range to the DAC?

But the intermediate values never get sent to the DAC at all - they're the values in the "in-between" bits of the filter, for example between the first and second stages or in the feedback loop itself. They're used to work out what the final output is (eg. a product of the difference between two comparatively large values - quite frequent in filters), and quite often when they're within a sensible range (a bit bigger than the DAC, but not much), the final value will end up being within the range of the DAC, and will stay within that range, without having to make any other corrections.

But when the intermediate values can be unbounded, quite often they end up heading off to the extremes of the floating point range... which is an awful lot bigger, of course. When they finally escape their range they become +infinity and no further calculations are possible on them... but even before that, when they're at the extremes of the FP range, the precision of the model is completely destroyed because the all-important +1 range is about 80 bits off the low end of the mantissa. Thus either the final calculation ends up as a flat 0, or as a propagated infinity - and no amount of massaging can repair that.

In contrast, fixed-point calculations never lose precision on their LSBs... but then, neither would floats if their range was similarly limited (limiting a 7-bit exponent to a maximum value of +4 would have more or less the same effect as the EMU10k1's accumulator). Whilst expanding the "normal" range of signals up to +2^123 or so (about 4 or 6 from the highest possible exponent) would seem to get round it, that would cause similar problems any time you had to multiply two signals together... so for a filter, the choices are to cap the resonance range (which is free), or put a comparatively expensive range-limiting step after each MAC that could allow the result to grow arbitrarily.

There is one other possible solution. In IEEE-754, the "actual" value of the exponent for a number is the contents of the exponent field minus a "bias" - 127 in 32-bit floats (with 8-bit exponents). If the bias could be adjusted manually in the FPU - even as a rather expensive operation - it would solve a lot of problems; for an 8-bit exponent, an audio application could set it to 250 (giving a maximum range of +32.0). Likewise, it'd be nice to see a switch that forced the FPU to "autocorrect" +infinity to +MAX_FLOAT. Those two modifications would enable floating-point DSPs to behave a lot more like fixed point DSPs where it matters. (Of course, both of these possibilities might very well be present and correct already in the 320c67 - I'll have to check. )

Gwenhwyfaer, thanks for the explanation. Will have to look into this and get back to you with some further questions if you don't mind. Many thanks for taking the time to write all that

[falcon]

Options I see for non-oscillating filters are:

1. Harder and/or less effecient to implement in FPU, as gwen mentions.
2. The lack of an amp at the end of the voice structure, to save some CPU.
3. The chosen filter implementations sounds good at high resonance levels on regular waves, but not very musical on arbitrary sounds (samples).

However,

-1- There are no real obstacles for implementing a full oscillating filter in FPU, other than perhaps some extra CPU usage AFAIK. And even if the extra headroom was a problem, you could shift the signal upward before processing and backward after that.

-2- Won't save much CPU and might even be less effecient in some cases.

-3- Would be a marketing choice. I'd prefer to have self oscillation, even it it sounded way ugly on some sources.

Well... it's a bit of a guess

FWIW: I suspect that overflow-to-maxfloor value is a common DSP option, but haven't hard-checked any specs. But the fusion probably already uses this internally. Else the signal would sound VERY distorted (due to round-the-clock overflow) when the clip indicator is up.