NTSC color palette emulation, with the square wave modulator

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

Post Reply
Bisqwit
Posts: 249
Joined: Fri Oct 14, 2011 1:09 am

NTSC color palette emulation, with the square wave modulator

Post by Bisqwit »

Reading the NTSC encoder page at http://wiki.nesdev.com/w/index.php/NTSC_video , I decided to create my own palette synthesizer based on the description. I first looked upon Blargg's nes_ntsc, but it seems to use a sinewave rather than the described squarewave for the color synthesis.

Here is my code, in C++11:

Code: Select all

    unsigned MakeRGBcolor(unsigned pixel)
    {
        // The input value is a NES color index (with de-emphasis bits).
        // We need RGB values. Convert the index into RGB.
        // For most part, this process is described at:
        //    http://wiki.nesdev.com/w/index.php/NTSC_video

        // Decode the color index
        int color = (pixel & 0x0F), level = color<0xE ? (pixel>>4) & 3 : 1;

        // Voltage levels, relative to synch voltage
        static const float black=.518f, white=1.962f, attenuation=.746f,
          levels[8] = {.350f, .518f, .962f,1.550f,  // Signal low
                      1.094f,1.506f,1.962f,1.962f}; // Signal high

        float lo_and_hi[2] = { levels[level + 4 * (color == 0x0)],
                               levels[level + 4 * (color <  0xD)] };

        // Calculate the luma and chroma by emulating the relevant circuits:
        float y=0.f, i=0.f, q=0.f, gamma=1.8f;
        auto wave = [](int p, int color) { return (color+p+8)%12 < 6; };
        for(int p=0; p<12; ++p) // 12 clock cycles per pixel.
        {
            // NES NTSC modulator (square wave between two voltage levels):
            float spot = lo_and_hi[wave(p,color)];

            // De-emphasis bits attenuate a part of the signal:
            if(((pixel & 0x40) && wave(p,12)) 
            || ((pixel & 0x80) && wave(p, 4)) 
            || ((pixel &0x100) && wave(p, 8))) spot *= attenuation; 

            // Normalize:
            float v = (spot - black) / (white-black) / 12.f;

            // Ideal TV NTSC demodulator: 
            y += v;
            i += v * std::cos(3.141592653 * p / 6);
            q += v * std::sin(3.141592653 * p / 6); // Or cos(... p-3 ... )
            // Note: Integrating cos() and sin() for p-0.5 .. p+0.5 range gives
            //       the exactly same result, scaled by a factor of 2*cos(pi/12).
        }

        // Convert YIQ into RGB according to FCC-sanctioned conversion matrix.
        auto gammafix = [=](float f) { return f < 0.f ? 0.f : std::pow(f, 2.2f / gamma); };
        auto clamp = [](int v) { return v<0 ? 0 : v>255 ? 255 : v; };
        unsigned rgb = 0x10000*clamp(255 * gammafix(y +  0.946882f*i +  0.623557f*q))
                     + 0x00100*clamp(255 * gammafix(y + -0.274788f*i + -0.635691f*q))
                     + 0x00001*clamp(255 * gammafix(y + -1.108545f*i +  1.709007f*q));
        return rgb;
    }
Outcome:
Image
To quote someone, I'd say it "is looking pretty good"... OLD TEXT: Though I am not sure whether the effect of combining multiple de-emphasis bits is correct. But at least the code for doing so is neater than in nes_ntsc :)

EDIT: Not to accuse nes_ntsc of anything. I did not compare the results, and it may very well do the very same thing through precalculated mathematics; I just thought it'd be good idea to actually replicate the process in detail. Also, this function is not meant to be called for each pixel. Ideally, you would precache the colors or calculate&cache them as needed.

EDIT: Replaced the YIQ conversion matrix with the FCC one, added guard for NaNs and removed the extended-range hack. This addresses problems
pointed out in beannaich's post(s), below.
EDIT: Replaced the attenuation code with single-attenuator and updated the screenshot. This addresses the problem with whether the combination of de-emphasis bits works properly or not.
Last edited by Bisqwit on Sun Oct 16, 2011 11:36 am, edited 7 times in total.
beannaich
Posts: 207
Joined: Wed Mar 31, 2010 12:40 pm

Post by beannaich »

One of the biggest problems I can see with this already is that color $20 is slightly off, $20 is $FDFDFD. Colors $xE and $xF are also incorrect, they should be flat black, but they are $050505.

Also, I see you're using the color decoder matrix found on the YIQ page of wikipedia. If you'd like to use the actual FCC decoder, then here it is:

Code: Select all

R   1.000000  0.946882  0.623557   Y
G = 1.000000 -0.274788 -0.635691 x I
B   1.000000 -1.108545  1.709007   Q
The above matrix was created by simplifying and inverting the RGB to YIQ matrix described here (Number 20).

So far it looks pretty good though, aside from the slight errors previously noted.
Bisqwit
Posts: 249
Joined: Fri Oct 14, 2011 1:09 am

Post by Bisqwit »

beannaich wrote:One of the biggest problems I can see with this already is that color $20 is slightly off, $20 is $FDFDFD. Colors $xE and $xF are also incorrect, they should be flat black, but they are $050505.
I actually did write that I intentionally extended the range very slightly (for debugging or other quirk purposes). The "black" and "white" variables control that. To get the function you describe as correct they should be set .518f and 1.962f respectively. And probably the last "signal high" value sehould be set to 1.962f rather than the 1.970f I set it to. Here is a screenshot of the outcome if those changes are implemented.
Image Image
Left: The "popular" matrix; Right: FCC sanctioned matrix.
beannaich wrote:Also, I see you're using the color decoder matrix found on the YIQ page of wikipedia. If you'd like to use the actual FCC decoder, then here it is:
Thanks. I actually used the matrix from nes_ntsc, which happens to be the same as on Wikipedia (quoted "popular" matrix).
I do not know which matrix it would make more sense to use, but the differences seem to be in the same realm as rounding errors....

EDIT: The clamp rule seems to be rather heavily invoked. I wonder if this is a problem.
Last edited by Bisqwit on Fri Oct 14, 2011 6:47 am, edited 1 time in total.
beannaich
Posts: 207
Joined: Wed Mar 31, 2010 12:40 pm

Post by beannaich »

Bisqwit wrote:To get the function you describe as correct they should be set .518f and 1.962f respectively.
This does fix the issues, the palette does look great with games I have a good memory of from my childhood (I should note that I haven't seen an actual NES on a tube television for about 13 years).
Bisqwit wrote:I do not know which matrix it would make more sense to use, but the differences seem to be in the same realm as rounding errors....
I should have noted that the two matrices are very similar, but there are some subtle differences. Most significantly are (2,1) and (2,2). And while the difference in output may be negligible, I still think it's proper to get the most accurate values.
The clamp rule seems to be rather heavily invoked. I wonder if this is a problem.
From what I can tell, you need to add in a check for NaN into your clamp function. This is sometimes generated when the value passed to pow(double) is negative. But clamping is necessary, and expected.
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Post by Near »

Really amazing work! I've been looking for just such a function for quite a while now.

Is anyone here able to confirm the behavior when multiple color emphasis bits are set? With that change, it'd be nice to have a definitive palette generator. Throw in a GUI to tweak some of the constants for user tastes, nes_ntsc for color bleeding and artifacts, and we'd be all set.
Bisqwit
Posts: 249
Joined: Fri Oct 14, 2011 1:09 am

Post by Bisqwit »

beannaich wrote:
The clamp rule seems to be rather heavily invoked. I wonder if this is a problem.
From what I can tell, you need to add in a check for NaN into your clamp function. This is sometimes generated when the value passed to pow(double) is negative. But clamping is necessary, and expected.
Thanks. I updated the code in the opening post with this change and with the other changes discussed earlier.
beannaich
Posts: 207
Joined: Wed Mar 31, 2010 12:40 pm

Post by beannaich »

I second what byuu said earlier, this palette is absolutely amazing! Really good job man. Next you should add hue/tint/saturation control, as byuu mentioned. So I can finally make a tv tuner dialog for my emulator ;)
Bisqwit
Posts: 249
Joined: Fri Oct 14, 2011 1:09 am

Post by Bisqwit »

Here is a version with saturation and hue controls (as function parameters). I don't know what the tint control could possibly be that differs from hue. If you want to adjust the de-emphasis bits, they are the bits 6,7,8 of the "pixel" parameter.
To adjust the hue by N°, pass N * 12 / 360 (e.g. 3 for 90°, -1.5 for −45°) as the hue_tweak parameter.
Pass 0.0 as saturation to get grayscale rendering; 2.0 for super-poppy colors.

Code: Select all

    unsigned MakeRGBcolor(unsigned pixel,
                          float saturation = 1.0, float hue_tweak = 0.0f,
                          float contrast = 1.0f, float brightness = 1.0f,
                          float gamma = 1.8f)
    {
        // The input value is a NES color index (with de-emphasis bits).
        // We need RGB values. Convert the index into RGB.
        // For most part, this process is described at:
        //    http://wiki.nesdev.com/w/index.php/NTSC_video

        // Decode the color index
        int color = (pixel & 0x0F), level = color<0xE ? (pixel>>4) & 3 : 1;

        // Voltage levels, relative to synch voltage
        static const float black=.518f, white=1.962f, attenuation=.746f,
          levels[8] = {.350f, .518f, .962f,1.550f,  // Signal low
                      1.094f,1.506f,1.962f,1.962f}; // Signal high

        float lo_and_hi[2] = { levels[level + 4 * (color == 0x0)],
                               levels[level + 4 * (color <  0xD)] };

        // Calculate the luma and chroma by emulating the relevant circuits:
        float y=0.f, i=0.f, q=0.f;
        auto wave = [](int p, int color) { return (color+p+8)%12 < 6; };
        for(int p=0; p<12; ++p) // 12 clock cycles per pixel.
        {
            // NES NTSC modulator (square wave between two voltage levels):
            float spot = lo_and_hi[wave(p,color)];

            // De-emphasis bits attenuate a part of the signal:
            if(((pixel & 0x40) && wave(p,12))
            || ((pixel & 0x80) && wave(p, 4))
            || ((pixel &0x100) && wave(p, 8))) spot *= attenuation;

            // Normalize:
            float v = (spot - black) / (white-black); 

            // Ideal TV NTSC demodulator:
            // Apply contrast/brightness
            v = (v - .5f) * contrast + .5f; 
            v *= brightness / 12.f;

            y += v;
            i += v * std::cos( (3.141592653f/6.f) * (p+hue_tweak) );
            q += v * std::sin( (3.141592653f/6.f) * (p+hue_tweak) );
        }	

        i *= saturation;
        q *= saturation;

        // Convert YIQ into RGB according to FCC-sanctioned conversion matrix.
        auto gammafix = [=](float f) { return f < 0.f ? 0.f : std::pow(f, 2.2f / gamma); };
        auto clamp = [](int v) { return v<0 ? 0 : v>255 ? 255 : v; };
        unsigned rgb = 0x10000*clamp(255 * gammafix(y +  0.946882f*i +  0.623557f*q))
                     + 0x00100*clamp(255 * gammafix(y + -0.274788f*i + -0.635691f*q))
                     + 0x00001*clamp(255 * gammafix(y + -1.108545f*i +  1.709007f*q));
        return rgb;
    }
EDIT: The attenuation explanation in the "NTSC video" article is ambiguous. In case there is simply just one attenuator that is selectively enabled at times, rather than three, one for each bit, the code to attenuate would be as follows [EDIT: Now included on the code above].

It would look like this (left: individual cascading attenuators for each bit; right: one attenuator):

Image Image
Last edited by Bisqwit on Fri Oct 14, 2011 2:50 pm, edited 7 times in total.
beannaich
Posts: 207
Joined: Wed Mar 31, 2010 12:40 pm

Post by beannaich »

Bisqwit wrote:I don't know what the tint control could possibly be that differs from hue.
Oops, I meant brightness.

Also, I don't think the article on NTSC video is ambiguous at all:
When signal attenuation is enabled by one or more of the channels and the current pixel is a color other than $xE/$xF (black), the signal is attenuated as follows
I believe that means that any of the three modulation channels can enable the attenuator, but the signal is only attenuated once.

This just keeps getting better and better!
Bisqwit
Posts: 249
Joined: Fri Oct 14, 2011 1:09 am

Post by Bisqwit »

Ok. Made the single-attenuator code default, and added brightness and contrast controls.

The single-attenuator code is mathematically unequivocal even in the presence of multiple de-emphasis bits.
Now can someone confirm whether the outcome from this de-emphasis code actually agrees with observations on real hardware? I don't have a dev cart, or even an NTSC system to begin with.
User avatar
thefox
Posts: 3134
Joined: Mon Jan 03, 2005 10:36 am
Location: 🇫🇮
Contact:

Post by thefox »

Bisqwit wrote:Ok. Made the single-attenuator code default, and added brightness and contrast controls.

The single-attenuator code is mathematically unequivocal even in the presence of multiple de-emphasis bits.
Now can someone confirm whether the outcome from this de-emphasis code actually agrees with observations on real hardware? I don't have a dev cart, or even an NTSC system to begin with.
See this post for a picture of blargg's palette demo. The colors are in different order but otherwise it's looking pretty much the same as yours.
Bisqwit
Posts: 249
Joined: Fri Oct 14, 2011 1:09 am

Post by Bisqwit »

I changed the color order to same as Blargg's and added the sawtooth scanline effect (the graphics is generated by native code, not NES code). Oddly, the color-de-emphasis bits are issued in a different order.
I don't know why Blargg used that particular order. Are mine possibly wrong?
His seem to be: none, blue, blue+red, red, red+green, green, green+blue, all; which is not binary; it's gray code.
Mine is: none, red, green, red+green, blue, blue+red, blue+green, all.
I added a saturation level of 1.15 to get better match.

Left: My generator with direct-to-PNG rendering
Right: Blargg's generator on actual NES
ImageImage
Last edited by Bisqwit on Sat Oct 15, 2011 4:00 am, edited 1 time in total.
beannaich
Posts: 207
Joined: Wed Mar 31, 2010 12:40 pm

Post by beannaich »

User avatar
Zepper
Formerly Fx3
Posts: 3262
Joined: Fri Nov 12, 2004 4:59 pm
Location: Brazil
Contact:

Post by Zepper »

Could someone post a .PAL file of this, please? Must be 192 bytes, no color emphasis entries.
beannaich
Posts: 207
Joined: Wed Mar 31, 2010 12:40 pm

Post by beannaich »

ask and ye shall receive
Post Reply