NTSC color palette emulation, with the square wave modulator

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

Bisqwit
Posts: 248
Joined: Fri Oct 14, 2011 1:09 am

Post by Bisqwit » Fri Oct 28, 2011 1:47 pm

lidnariq wrote:
Bisqwit wrote:It raises a new one, instead: Would that not mean that a stripe of constant color would appear having a fluctuating brightness?
No, because the color phase doesn't reset with every pixel:

Code: Select all

(hhhhhhll)(llllhhhh)(hhllllll)
\_pixel1_/\_pixel2_/\_pixel3_/
Doesn't that only confirm that it would appear to have a fluctuating brightness? Pixel 1 has the total amplitude of 6*h + 2*l. Pixel 2 has 4*h + 4*l. Pixel 3 has 2*h + 6*l.

lidnariq
Posts: 8791
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Post by lidnariq » Fri Oct 28, 2011 1:48 pm

Sorry, I realized I'd misread your post and explained the wrong thing, please go back and read my fixed version.

Bisqwit
Posts: 248
Joined: Fri Oct 14, 2011 1:09 am

Post by Bisqwit » Fri Oct 28, 2011 1:53 pm

lidnariq wrote:No, because the reconstructed chroma is subtracted from the received composite signal to calculate the true brightness.
Hmm... Thanks for the explanation. I did not understand it, though... I am unsure how to phrase a question. I would think this in terms of an algorithm and I cannot figure out how to translate your words into an algorithm. I guess I will try to write something according to what I guess you meant and see if it produces anything that makes sense.

lidnariq wrote:Also the duty cycle is 50% unless you're doing something funny, not 33%.
Ah, sorry, my bad.

tepples
Posts: 21755
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples » Fri Oct 28, 2011 2:33 pm

Bisqwit wrote:
lidnariq wrote:the color phase doesn't reset with every pixel:

Code: Select all

(hhhhhhll)(llllhhhh)(hhllllll)
\_pixel1_/\_pixel2_/\_pixel3_/
Doesn't that only confirm that it would appear to have a fluctuating brightness? Pixel 1 has the total amplitude of 6*h + 2*l. Pixel 2 has 4*h + 4*l. Pixel 3 has 2*h + 6*l.
An analog TV doesn't see pixel boundaries; it only sees a waveform.

Code: Select all

hhhhhhllllllhhhhhhllllll
It then uses various filters, which can be thought of as a moving weighted average, to separate this waveform into chroma (parts of the signal that are momentarily of the same frequency as the colorburst) and luma (the rest of the signal).

Bisqwit
Posts: 248
Joined: Fri Oct 14, 2011 1:09 am

yay

Post by Bisqwit » Fri Oct 28, 2011 4:02 pm

I think I got quite a good result. It is now much closer to what I see on the tape than it was before.
Thanks tepples and lidnariq.


Source code (caching code redacted for brevity, performance may be bad):

Code: Select all

    SDL_Surface *s;

    const int xres=256*1, yres=240*1;

    void Init()
    {
        SDL_Init(SDL_INIT_VIDEO);
        SDL_InitSubSystem(SDL_INIT_VIDEO);
        s = SDL_SetVideoMode(xres, yres, 32,0);
        signal(SIGINT, SIG_DFL);
    }

    static unsigned framecounter=0;
    static unsigned prev2[3][240][xres]={{{}}}, colorbursts[240], Colorburst=4;
    static u16 prev1[3][240][256]={{{}}}; // NES pixels corresponding to each screen location in each tweak offset
    static bool diffs[240] = {false};     // Whether this scanline has changed from what it was the last time

    void PutPixel(unsigned px,unsigned py, unsigned pixel)
    {
        u16 v = 0x8000^pixel, &p = prev1[Colorburst/4][py][px];
        if(p != v) { p = v; diffs[py] = true; }
    }
    void FlushScanline(unsigned py, unsigned length) /* length is in pixels: 340 or 341. */
    {
        if(py < 240) colorbursts[py] = Colorburst;
        if(py == 239)
        {
            //#pragma omp parallel for schedule(guided)
            for(py=0; py<240; ++py)
            {
                unsigned y1 = (py  )*yres/240;
                unsigned y2 = (py+1)*yres/240;
                unsigned colorburst = colorbursts[py];
                auto& target = prev2[colorburst/4][py];
                auto& line   = prev1[colorburst/4][py];
                if(diffs[py])
                {
                    float sigbuf[256*8], d07=0.f, d15=0.f;
                    float sigi[256*8], sigq[256*8]; // Match I & Q at each position.
                    for(unsigned p=0; p<256*8; ++p)
                    {
                        // Retrieve NTSC signal from PPU
                        int pixel = line[p/8]%512, offset = (colorburst+p) % 12;
                        // Decode the color index.
                        int color = (pixel & 0x0F), level = color<0xE ? (pixel>>4) & 3 : 1;
                        // Voltage levels, relative to synch voltage
                        static const float black=.518f, white=1.962f, attenuation=.746f,
                          levels[8] = {.350f, .518f, .962f,1.550f,  // Signal low
                                      1.094f,1.506f,1.962f,1.962f}; // Signal high
                        auto wave = [](int p, int color) { return (color+8+p)%12 < 6; };
                        // NES NTSC modulator (square wave between two voltage levels):
                        float spot = levels[level + 4*(color <= 12*wave(offset, color))];
                        // De-emphasis bits attenuate a part of the signal:
                        if(((pixel & 0x40) && wave(offset,12))
                        || ((pixel & 0x80) && wave(offset, 4))
                        || ((pixel &0x100) && wave(offset, 8))) spot *= attenuation;
                        // Normalize:
                        float v = (spot - black) / (white-black);
                        // Apply slight signal degradation to it
                        v = v-0.5f;
                        d07 = d07*0.3f + 0.7f*v;
                        d15 = d15*-.5f + 1.5f*v;
                        v = 0.5f + d07*0.7f + d15*0.3f;
                        sigbuf[p] = v;
                        auto cosf = [](int p) { return std::cos(3.141592653 * p / 6); };
                        sigi[p] = v * cosf(p+12+colorburst);
                        sigq[p] = v * cosf(p+21+colorburst);
                    }
                    float gamma = 1.8f;
                    for(unsigned x=0; x<xres; ++x)
                    {
                        float i=0.f, q=0.f, y=0.f;
                        for(int s = x*256*8 / xres, p=0; p<12; ++p, ++s)
                            if(s >= 0 && s < 256*8)
                                { i += sigi[s];
                                  q += sigq[s];
                                  y += sigbuf[s]; }
                        i /= 12.f; q /= 12.f; y /= 12.f;
                        //float amplitude = std::sqrt(i*i + q*q);
                        //y = 0.9f * y + 0.1f * sigbuf[x * 256*8 / xres];
                        //float y = sigbuf[x * 256*8 / xres] - amplitude;
                        auto gammafix = [=](float f) { return f <= 0.f ? 0.f : std::pow(f, 2.2f / gamma); };
                        auto clamp = [](int v) { return v<0 ? 0 : v>255 ? 255 : v; };
                        target[x] = 0x10000 * clamp(255.9f* gammafix(y +  0.946882f*i +  0.623557f*q))
                                  + 0x00100 * clamp(255.9f* gammafix(y + -0.274788f*i + -0.635691f*q))
                                  + 0x00001 * clamp(255.9f* gammafix(y + -1.108545f*i +  1.709007f*q));
                    }
                    diffs[py] = false;
                }
                for(unsigned y=y1; y<y2; ++y)
                {
                    u32* pix = ((u32*) s->pixels) + y*xres;
                    std::memcpy(pix, target, sizeof(target));
                }
            }
            if(++framecounter%1 == 0) SDL_Flip(s);
        }
        Colorburst = (Colorburst + length*8) % 12;
    }
Various title screens rendered in 256x240 and in, umm, 800x240, as tests (vertical scaling would only have increased post length without any information value).
Image Image Image Image
Image
Image
Image
Image


There is one thing that bothers me still in this, though. I am not calculating Y in the manner suggested by lidnariq. Instead, I am simply taking the average of the 12 amplitudes nearest to the current pixel. As a curiosity, here's what I get if I calculate the luma corresponding to the calculated chroma and subtract it from the momentary signal value...
It looks a bit even weirder than it should because of aliasing artifacts.

Image

lidnariq
Posts: 8791
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Post by lidnariq » Fri Oct 28, 2011 9:05 pm

Looks very nice!

So, the way I understand NTSC demodulation works is as follows:

Read the colorburst. This is phase -U. Then either do the "proper NTSC thing" according to the standards or the cheap thing (what the majority of analog televisions did).

The cheap thing: Generate new sine waves at phases U (+ve means bluish) and V (+ve means reddish). Multiply your input composite samples by both sinusoids. Horizontally spatially lowpass both of these demodulated inputs at ~1.5MHz. These are your U and V samples to be used later. Now multiply (remodulate) your lowpassed U and V outputs by the original sinusoids you generated at the top. Subtract this from the input composite signal. Possibly lowpass (less clearly necessary) this at 4MHz to get your "true" Y.

The 'proper NTSC thing': Using -U, generate carriers I and Q at 57° and 147° clockwise. Demodulate I (+ve is orangish) and Q (+ve is pinkish) components the same way as above. Lowpass I and Q at 1.4 MHz and 500kHz respectively instead of the common bandwidth specified for U and V above. You now have I and Q. The rest is the same as above.

As tepples pointed out, nowhere does the TV know where the pixels are, so the edges of pixels are necessarily blurred; calculating the instantaneous luma from each 8-sample subset of pixels is giving you the weird result you have on the Ice Climbers screen there. By treating each 8-sample subset as isolated, you in fact do get the light-dark pattern you were earlier asking us about.

The wikipedia article on YIQ is my source for the above frequencies. I've also taken the image of the colorspace on the YIQ article and combined it with the demonstration output from a Vectorscope showing the standard test bars and combined them with a small overlay to show the NTSC NES hues denoted in small black numbers.
Last edited by lidnariq on Wed Sep 03, 2014 4:57 pm, edited 2 times in total.

User avatar
HardWareMan
Posts: 206
Joined: Mon Jan 01, 2007 11:12 am

Post by HardWareMan » Sat Oct 29, 2011 12:39 am

No one looked at my pictures from the oscilloscope? Let me tell you a terrible secret: PPU makes color subcarrier only for dyed colors. White, black and gray shades are always without subcarrier and look clearly at actual pixel boundaries. This is a terrible simplification goes against the standard of PAL / NTSC, but for decoder ultimately does not matter: there is no subcarrier means no color, simplicity doubles profit. However, the color burst is always present, because it sets the sync decoder.
So, everything simple: you must stop shuffling whole lines - looks terrible. You must make 12 references for color brightness for each subcarrier phase and use it only for colored pixel index. Look at my example (wich a long time ago I made on some forum):
Source:
Image
Colored only:
Image
Colored and not colored are combined together:
Image
Result:
Image
Quite simple, isn't it?

LocalH
Posts: 172
Joined: Thu Mar 02, 2006 12:30 pm

Post by LocalH » Sat Oct 29, 2011 1:47 am

I don't know so much about PAL, but with NTSC, the mere presence of color burst means that the set will try to decode the chroma, even where there is supposed to be none, and luma will bleed over into chroma in certain situations (it's the whole concept behind artifact colors such as seen on the Apple II, TRS-80 CoCo, and IBM CGA). It all depends on the relationship between the pixel clock and the color burst.

I suppose PAL doesn't do this, however, seeing as none of the systems that can do artifact color will generate it on a PAL system, instead just outputting black and white (with occasional light color fringing but nothing that could be called "artifact color").

Bisqwit
Posts: 248
Joined: Fri Oct 14, 2011 1:09 am

Post by Bisqwit » Sat Oct 29, 2011 2:15 am

HardWareMan wrote:No one looked at my pictures from the oscilloscope? Let me tell you a terrible secret: PPU makes color subcarrier only for dyed colors.
I do not understand how that is a secret. That is even described at http://wiki.nesdev.com/w/index.php/NTSC_video .
I did look at the pictures.

For comparison, here is a signal dump of what my most recent version (source code above) produces for scanline 78 of Rockman 2 title screen (right under the "Rockman" text: black, purple solid, white text (begins atop the purple, then continues atop black), black, peach solid, one pixel black, purple solid, black); right above the black line between the peach and the purple in "2".
Sync/colorburst are not shown, because I do not generate them as signals (they are just shortcut-implemented as logic rather than as signal). The dump length is 256*8 samples.

Image
Click to get a magnificated version.
Sure it's not identical down to every detail, but I do not see any categorical differences.

Top: Original, perfectly sharp signal. Bottom: The signal after applying two filters to it in order to get a smoother shape with some high frequency noise. This is the one that gets processed by the simulated "TV".
The ones in the middle are various differently filtered versions, for testing.
The gray bars in the background identify where PPU's pixels begin and end.

Bisqwit
Posts: 248
Joined: Fri Oct 14, 2011 1:09 am

Post by Bisqwit » Sat Oct 29, 2011 5:20 am

I implemented the 1.4 Mhz and 500 kHz lowpass filters to I and Q respectively. Y is filtered at 3.5 MHz. (Actual implementation weighted-averages them at 31, 86 and 12 samples respectively; it's not a perfect solution to lowpass-filtering...)
It also does the 57 and 147 degree offsets to the colorburst, which means that the colors are sightly different to previous demos.
It's getting a bit too large for an inline GIF, so here's a 800x600 AVI, 60.0988 FPS, 18 seconds long, 8 megabytes. Be sure to rightclick and select "save as" and not try to watch it without saving first.
http://bisqwit.iki.fi/kala/snap/nesemu1 ... by600b.avi
I rendered it at 2048x1536 and lanczos-scaled down to 800x600 (because not many a computer can play 2048x1536 videos at 60 fps).

I did not put it to YouTube because YouTube caps the FPS at 30, which is really inconvenient when you are trying to demo something that has relevant details at higher rate than that.

P.S. Initially I accidentally used for Q a ratio 143 rather than 86 for some reason. For curiosity, it's available here. Nice gradients! http://bisqwit.iki.fi/kala/snap/nesemu1 ... 00wide.avi
Last edited by Bisqwit on Sat Oct 29, 2011 6:02 am, edited 1 time in total.

User avatar
Zepper
Formerly Fx3
Posts: 3190
Joined: Fri Nov 12, 2004 4:59 pm
Location: Brazil
Contact:

Post by Zepper » Sat Oct 29, 2011 5:36 am

Just a little question: the NES output come from the RF or composite cable for those pics?

Bisqwit
Posts: 248
Joined: Fri Oct 14, 2011 1:09 am

Post by Bisqwit » Sat Oct 29, 2011 5:49 am

Zepper wrote:Just a little question: the NES output come from the RF or composite cable for those pics?
Is not the difference only in whether it is modulated by the aerial carrier frequency (i.e. RF modulation) and whether the audio is also transmitted along it?
I'm not doing audio side-effects in this simulation, and I also ignore any side-effects caused by the RF modulation.

tepples
Posts: 21755
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples » Sat Oct 29, 2011 6:21 am

Bisqwit wrote:
Zepper wrote:Just a little question: the NES output come from the RF or composite cable for those pics?
Is not the difference only in whether it is modulated by the aerial carrier frequency (i.e. RF modulation) and whether the audio is also transmitted along it?
That, and the fact that RF is supposed to have a low-pass at 4.2 MHz, and the fact that RF tends to have more noise.

User avatar
tokumaru
Posts: 11469
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Post by tokumaru » Sun Oct 30, 2011 10:09 am

Just want to say that I find it really cool that you are going this far to emulate the NES. Most people don't care about faithful video emulation.

Why do flat areas have diagonal stripes though? I don't think my NES does this, or blargg's NTSC filter either.

tepples
Posts: 21755
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples » Sun Oct 30, 2011 10:24 am

tokumaru wrote:Why do flat areas have diagonal stripes though?
Bisq and I discussed this on IRC, and it appears the filter for Y (luma) wasn't rejecting all of the chroma signal. The stopband attenuation around 3.6 MHz needed to be several dB greater. We tried switching to a 12-sample box filter for Y, and the stripes disappeared.

Post Reply