## Gauss Table Creation

Discussion of hardware and software development for Super NES and Super Famicom.

Moderator: Moderators

Forum rules
• For making cartridges of your Super NES games, see Reproduction.
nocash
Posts: 1234
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

### Gauss Table Creation

Sony's soundchips (as used in SNES and PSX) are using a "gauss_table" with 512 entries for pitch interpolations:
SNES: http://nocash.emubase.de/fullsnes.htm#s ... spbrrpitch
Above docs are containing the table contents as extracted from real hardware.

So far, the table contents are known, and everything is fine. But, just for curiosity: How did Sony create those tables?

The guy who did originally dump one of that tables (and who branded the name "gauss" table) did later admit that he doesn't have a clue if the table/interpolation has any relation to Gauss at all. Anyway, the table content does seem to resemble to something that is called "gaussian normal distribution" - so after all, the name "gauss_table" seems to be correct.

Here's a formula that does - more or less - reproduce the contents for table[0..511]:

table = (e^(-((siz-i)^2)/curv)) * (volume) - offset

e = Euler's Number (2.718281828...)
siz = index of last table entry (511) (or maybe 512 in case the table excludes the highest point)
offset = some small offset, needed to get table=0 for SNES, and table=-1 for PSX
volume = volume factor (should be equal to "table[siz]+offset")
curv = some constant that somehow affects the shape of the curve

The "volume" should be kinda obvious since
table[siz]=(e^(0/curv) * (volume) - offset
If siz should be 512 rather than 511, then (for obtaining the "volume" value) one must guess the value for table, the value should be same or maybe one bigger than table.

The "offset" value is needed to get table=0 for SNES, and table=-1 for PSX. Without subtracting that offset, table would be some positive value (the formula won't reach zero until somewhere at table[-infinite]).
with some experimentation, I ended up with these offset values
offset=circa 10 for SNES
offset=circa 50 for PSX

that assuming that offset is constant for all table entries. I might be also variable, something like "offset=(512-i)/10" instead of "offset=50" or whatever. But anyways, the offset is needed only for fine-tuning.

And "curv" can be calculated as:
curv = -(siz-i)^2 / log.e((table+offset)/volume)
aka
curv = -(siz-i)^2 / ln((table+offset)/volume)
aka, when picking i=256 for example,
curv = -(siz-256)^2 / ln((table+offset)/volume)
and then I got this curv values from above formula:
curv=circa 53240 for SNES
curv=circa 42484 for PSX

So, with above stuff, three table entries are used as reference points:
table[siz]=highest point, used to calculate "volume"
table=some random point, used to calculate "curv"
table=lowest point, used to compute "offset" (done by experimentation, not really calculated)

and with the computed volume, curv, offset values, it should be theoretically possible to calculate all other table entries.

Unfortunately, the results are still far away from perfection. Maybe I got something wrong, or maybe Sony used some rounded value like e=2.7 rather than e=2.718281828... or the overall formula isn't correct at all.

Any ideas?

PS. credits to Felix Laepple for pointing me on the basic formula.

blargg
Posts: 3715
Joined: Mon Sep 27, 2004 8:33 am
Location: Central Texas, USA
Contact:

### Re: Gauss Table Creation

Here's a generator I wrote a while back that comes close (about 12 units off at the extremes):

Code: Select all

``````double e = 2.718281828;
for ( int i = 0; i < 512; i++ ) {
double x = i / 511.0 * 2.31 - 0.05;
double y = pow( e, -x * x ) * 1305.64;
gauss [i] = y - 8.54;
}``````

nocash
Posts: 1234
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

### Re: Gauss Table Creation

Thanks! That looks similar...
Your "511.0*2.31" would be equivalent to "curv=(511.0/2.31)^2" (ie. curv=48934.8) (a good bit different than the curv=53240 that I came up with for SNES).
The "-0.05" is something that I didn't have, did you use that to maintain the highest point at 1305 despite of the -8.54 subtraction?
Big difference is that "i" instead of "siz-i" will reverse the table, ie. table=highest point instead of lowest point.

tepples
Posts: 22075
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

### Re: Gauss Table Creation

I opened numpy, graphed the log FFT of the whole thing, and found a huge notch in the frequency response around period 256 (one input sample). It's as if they took their original curve and tweaked it to resolve one special case of whining at the sample frequency. I was able to produce a very similar-looking (and similar-functioning but nowhere near bit-exact) curve by convolving four boxcar functions of length 256, or three 256-boxcars and two 128-boxcars. Next I might see what I can do with products of the bell curve at various scales and various window functions (Hann, Blackman, etc.).

Is one of the goals some way to compress the table for use in an emulator?

nocash
Posts: 1234
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

### Re: Gauss Table Creation

No, no goal, just for curiosity. The whole table takes only 1kbyte, the floating point math needed to calculate the table might even eat up more memory.

The table contains a 'smooth' waveform without notches, where did you see notches? Or did you go some step farther and did analyze interpolation filtering characterisics... or whatever?

Tried the SNES table with my formula and curv=53240 that gave me errors around +/-16, a bit worse than blargg's results. Then I tried curv=53240-3000+i*6000/512 that dropped the error to around +/-8, that's even a bit better than blargg's results. Maybe this is the right direction and gives perfect results when fine-tuning the "-3000+i*6000" values.

tepples
Posts: 22075
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

### Re: Gauss Table Creation

nocash wrote:The table contains a 'smooth' waveform without notches, where did you see notches? Or did you go some step farther and did analyze interpolation filtering characterisics... or whatever?
Yes, I took the Fourier transform to analyze its filtering characteristics. By "notch", I meant near-zero response at a particular frequency.

nocash
Posts: 1234
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

### Re: Gauss Table Creation

Another try, curv=46440+i*24 (and offset=5, siz=511, volume=1305.5+offset), gives error in range -0..+6, yet a good bit closer.
tepples wrote:By "notch", I meant near-zero response at a particular frequency.
Yikes, frequency responses are sounding difficult.
With my plus/minus/xor integer math skills, even multiplications and exponents are already looking horribly complicated to me :-)

nocash
Posts: 1234
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

### Re: Gauss Table Creation

Tried to calculate "curv" values for three points...
i=128, table=029h=41, curv=-383^2 / ln((41+5)/1310.5) = 43794
i=256, table=176h=374, curv=-255^2 / ln((474+5)/1310.5) = 64607
i=384, table=3C9h=969, curv=-127^2 / ln((969+5)/1310.5) = 54351

Hmmmm... not the expected the result. I was expecting that curv would increase alongside with i.
But of that three values, it's reaching the highest point at i=256.
NB. at i=256, the "ln(...)" result is close to -1.00.

EDIT: Oops, typo, used 474 instead of 374. Correct should be:
i=128, table=029h=41, curv=-383^2 / ln((41+5)/1310.5) = 43794
i=256, table=176h=374, curv=-255^2 / ln((374+5)/1310.5) = 52412
i=384, table=3C9h=969, curv=-127^2 / ln((969+5)/1310.5) = 54351

So curv does increase alongsides with i, but not linearily.
Or maybe curv is constant and I am just trying to workaround a mistake elsewhere in the formula.

Near
Founder of higan project
Posts: 1550
Joined: Mon Mar 27, 2006 5:23 pm

### Re: Gauss Table Creation

Just noting, in case you weren't aware ... but the S-DSP stores the gaussian table in a 512x12-bit (6144-bit) mask ROM table.

Unfortunately we haven't been able to extract the table yet, the die scans needed another layer removed to see it and this hasn't been done.

It is possible, though it'd be extreme and unlikely, that they hand-tweaked entries in this table.

nocash
Posts: 1234
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

### Re: Gauss Table Creation

Okay, I've written a utility that can display the graphs, and calculate graphs using FPU opcodes, and display differences between original and calculated graphs... (source code
Gauss.zip
for Borland TASM).

First of, here are the original SNES and PSX tables, shown at their 12bit/16bit ranges (max=800h/8000h for SNES/PSX). And scaled to max=519h/59B3h, ie. highest points of the SNES/PSX graphs).
The sum of entries gauss[000h+i]+gauss[0FFh-i]+gauss[100h+i]+gauss[1FFh-i] is approximately 800h/8000h for SNES/PSX, the difference is that the PSX graph having a steeper+higher peak level, but less steep+high bottom). gauss1.gif (7.59 KiB) Viewed 11223 times
Next, here are some attemps to compute the SNES table by software.
The upper picture shows what happens when using bigger/smaller "curv" values. The bold lines are the actual graphs, the thin lines are showing the difference between original (red graph) and computed graphs at higher resolution - ideally this should be a straight horizontal line (=no difference).
The lower picture shows some more attempts:
curv=55968 is quite fine on the left side, but goes wrong at the right side
curv=58700+x*24 is quite fine, but still has some up/down error

nocash
Posts: 1234
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

### Re: Gauss Table Creation

This is computing the required "curv" values... the jitter at the left/right sides is due low resolution of the 12bit table entries... but even without that glitch, the result doesn't look too useful :-/ gauss3.gif (5.39 KiB) Viewed 11223 times
So next, attempt, replacing the "((x)^2)/curv" idea by "(something)^2", and now computing that "something":
The upper picture is using factor=256, looks fine. The lower one factor=235, which looks even better.
And, the graphs with offset=0 (cyan) are finally showing some "constantly" raising waveform, yeah :-) And, my first attempt to calculate "(something)" by software. Not perfect, but it looks as if it chould give perfect results when fine-tuning the 7000 and 532 values.
EDIT: And, the "235" in the reference-graph may also need some fine-tuning. gauss5.gif (5.79 KiB) Viewed 11223 times

nocash
Posts: 1234
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

### Re: Gauss Table Creation

Got bored of calculating or guessing numbers, and instead, used dumb brute-force approach for finding better values than 7000, 532, 235. This did threw out values 11580, 551, 244. Giving these graphs: gauss6.gif (10.15 KiB) Viewed 11199 times
The difference to the original SNES table is now within -1..+3. The formula is most probably correct, and errors may be due to rounding issues on the final result, or fractional parts of the constants (like maybe 551.4 instead of 551, or 1305.5 instead of 1305, etc).

tepples
Posts: 22075
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

### Re: Gauss Table Creation

The remaining waviness may be a post-processing step to make sure all sums of four corresponding values are near \$800, so that DC interpolates to DC.

nocash
Posts: 1234
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

### Re: Gauss Table Creation

tepples wrote:The remaining waviness may be a post-processing step to make sure all sums of four corresponding values are near \$800, so that DC interpolates to DC.
Theoretically yes, but the formula does spit out sums near 800h anyways (don't know how and why, but it does do so). And Sony definetly didn't mind about getting exact sums of 800h (causing the nasty hardware glitch when sum becomes 801h).

I've changed the brute-force stuff a bit, allowing it to span bigger range at better resolutions (with steps smaller than 1.0). The problem is that some of the FPU opcodes are quite slow, computing a few million graphs with 512 points each can take up a whole minute, or even several hours when using slightly bigger ranges for the separate constants.

Some nice constants are 16185, 580.0, 255.0, 1305.0. Used like this:
n = (x + 16185/(580-x) - 16185/580) / 255
table[x] = (e^(-(n^2))) * 1305

Results are very close to the original snes table (with errors are in range -1..+1).
Though there are various other constants that give similar (or possibly even better) results, so it's hard to tell which values are best.

I think some of the remaining errors could be blamed to rounding errors. One thing that is definitely wrong is that my tool spits out table=(e^0)*1305=1304. And Sony's original program may have similar rounding errors, which would make it difficult to get the same results without knowing the original FPU rounding mode and FPU resolution.

Oh, and I've replaced "e=2.718281828" by "e=(1.0)*(2^log.2(e))" (using fld1,fldl2e,fscale opcodes, which is hopefully more accurate).

EDIT: Just changed the resolution of FPU memory operands from 64bit (qword) to the full 80bit (tbyte) resolution - that has fixed the "(e^0)<1" error.

EDIT: I was somehow thinking that table might be rounded down to 1305. But actually, it might be rounded up to 1305. With constants like so 16151.9, 580.1, 255.0, 1304.5. Results are possibly looking a bit better that way. Only, again the FPU is giving me table=(e^0)*1304.5=1304, same for (e^0)*1304.6, despite of round-to-nearest mode, the damn thing just isn't rounding as desired.

nocash
Posts: 1234
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

### Re: Gauss Table Creation

Tried to do the same for the PSX table - should have been doing that anyways since the 16bit entries are having 16x better precision than the 12bit SNES values. gauss7.gif (4.67 KiB) Viewed 11129 times
Hmmmm, the error (the thin line at the bottom of the image) ranging from +25..-10 doesn't look good. I got a similar error for the SNES (but barely visible, ranging only from +1..-1 due to the lower resolution).
Anyways, I am afraid that +25..-10 can't be blamed on rounding errors, so there's probably still something wrong/missing in the overall formula :-/
EDIT: As tepples mentioned the entries should sum up properly (to 7F7Fh..7F81h for the PSX), which won't work with the above errors. Either Sony has applied post processing to get the 7F7Fh..7F81h range, or they used a better formula that didn't need post processing...