Revisiting the quest for the most accurate NES palette:

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems. See the NESdev wiki for more information.

Moderator: Moderators

User avatar
Dwedit
Posts: 4922
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Revisiting the quest for the most accurate NES palette:

Post by Dwedit »

I don't think you'll find SNES colors that generate the same square waves as NES colors. You might find colors that appear the same, but the waveforms will be different.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Revisiting the quest for the most accurate NES palette:

Post by tepples »

DRW wrote:
tepples wrote:What could be more accurate than the palette that, when fed to the Super NES S-PPU, produces the same voltages?
Would this work?
Does the SNES work with definite RGB values respectively values that can at least be converted to RGB?
Yes. Each palette entry controls the level of each of red, green, and blue channels on a scale from 0 to 31.
If yes, would another console that also works with RGB values produce the same colors when the same voltages are used?
Did you mean "another console" as in another Super NES Control Deck? Each unit with the same motherboard and PPU2 revision should produce the same signal given the same RGB triple. There are three revisions that matter as far as I can tell: 1/1/1 (launch), 2/1/3 (most), and 1CHIP (late and all mini). And even between those three, the variation shouldn't be noticeable.

Or did you mean "another console" as in a different platform entirely, such as a Super NES Control Deck vs. a GameCube with Game Boy Player, whose palette also uses 5-bit-per-channel RGB?
And if you can answer all this with yes, then why didn't you say this earlier, let someone do it and be done with this whole topic
The reason I haven't done so is that I don't own an oscilloscope nor know how to use one nor where to rent one. The reason others haven't done so is that they appear not to be interested.
Or do you have the equipment to measure the actual analog signal (voltage etc.) and then you simply cycle hrough all the colors individually until their analog signals matches with the one on the NES?
Oscilloscopes exist, though I don't happen to own one.
And will there be a perfect match? What if one of the NES colors has, let's say, a color brightness of 83.6253 %, but on the NES, you only find colors with 83.6250 % and 83.6256 %?
Having a brightness between, say, 25/31 and 26/31? That's a possibility. But someone with an oscilloscope should be able to mark that it's between two values. And at that point, bickering about it would become even more sperglord-ish than I'd condone.
Dwedit wrote:I don't think you'll find SNES colors that generate the same square waves as NES colors.
This 3.58 MHz square wave consists of some energy at DC that sets the luma, a 3.58 MHz sine wave that sets the chroma, plus some energy at 10.74 MHz (the third harmonic) and higher. The "color" as a TV receives it is defined by the luma and chroma. So if you make a low-pass filter with a corner frequency anywhere between 4.2 MHz (the standard corner for RF) and 10 MHz, you'll end up finding Super NES colors that generate the same sine waves as NES colors after filtering.
User avatar
DRW
Posts: 2225
Joined: Sat Sep 07, 2013 2:59 pm

Re: Revisiting the quest for the most accurate NES palette:

Post by DRW »

tepples wrote:Or did you mean "another console" as in a different platform entirely, such as a Super NES Control Deck vs. a GameCube with Game Boy Player, whose palette also uses 5-bit-per-channel RGB?
This.
My game "City Trouble":
Gameplay video: https://youtu.be/Eee0yurkIW4
Download (ROM, manual, artworks): http://www.denny-r-walter.de/city.html
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Revisiting the quest for the most accurate NES palette:

Post by tokumaru »

Would the SNES RGB values be of any actual help, though? I mean, there's no guarantee that the same RGB values that an SNES happens to transform into NES-like colors will look anything like NES colors on a modern computer or television. Unless you're making an NES emulator for the SNES, I don't see how knowing these RGB values would help with anything.
User avatar
thefox
Posts: 3134
Joined: Mon Jan 03, 2005 10:36 am
Location: 🇫🇮
Contact:

Re: Revisiting the quest for the most accurate NES palette:

Post by thefox »

DRW wrote:Miyamoto was only the designer for "Super Mario Bros.", not for the NES itself. So, when he says that he took this specific color because it matched what he wanted for the sky, this says nothing about what the designers of the NES had in mind when defining this color. It only means that Miyamoto's TV happened to show the sky color in a puple-like way. But the design choices of this 1985 game still say nothing about the intended specifications of the 1984 console.
I actually think I remember reading an interview where Miyamoto claimed that he took part in choosing the colors for the Famicom. Take his whatever way you like. (I'm not sure how much influence he could've actually had -- the color generation method is such that the hues are pretty much evenly spaced, but maybe he could've had a say in the brightness/saturation of the colors.)
It is an analog signal, consisting of voltages. There is no "most accurate" palette.
Well, I agree, but it has nothing to do with it being an analog signal. It has to do with how people (and TV manufacturers) are interpreting/implementing the standards.
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: fo.aspekt.fi
User avatar
Hojo_Norem
Posts: 137
Joined: Mon Apr 16, 2007 10:07 am
Contact:

Re: Revisiting the quest for the most accurate NES palette:

Post by Hojo_Norem »

TL;DR: Get rid of fixed palettes all together and replace with proper PAL/NTSC emulation with suitable user adjustable controls.

-----
If it hasn't been done already, then the PPU needs to be decapped and the video output circuitry traced and converted to a software algorithm. Next step is to feed the output of this algorithm into a software PAL/NTSC decoder... one with user adjustable realtime brightness/contrast/colour/tint controls.

Colours don't look right? Do what you did with your old TV and just play with the settings until it does.

The thing is, even I know that this is a rather steep task to perform. Hell, take the C64 emulation scene, the correct PAL colour values have been known for years (the pepto palette, derived from the workings of the VIC-II) but people still complain. VICE64 and other emulators implement proper PAL emulation so colour mixing tricks and other GFX techniques can be displayed properly, but people complain about the image not looking 'sharp and pixely'. They say that the colours look too washed out, while ignorant to the fact that the saturation can be turned up in the emulator just like you would turn up the colour knob on your telly.

Which brings me to the argument about the designer's opinions on the colours. Back in the day, most TV sets were adjusted with easy to reach, quick to turn controls. It would be easy for one person to adjust a TV to his or her liking, only for the next person to come along to set it to their preferences. For all we know, Miyamoto could have had the colour dial on his TV all the way to the max and could have thought that everybody else's TVs were dull looking.
Insert witty sig. here...
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Revisiting the quest for the most accurate NES palette:

Post by tepples »

Hojo_Norem wrote:TL;DR: Get rid of fixed palettes all together and replace with proper PAL/NTSC emulation with suitable user adjustable controls.
Decoding the composite signal to a texture doesn't work so well if you're trying to run an emulator on a device whose CPU clock speed is well below 1 GHz, or if you're (say) trying to port an NES game to another retro platform while preserving colors as closely as possible.
If it hasn't been done already, then the PPU needs to be decapped and the video output circuitry traced
Visual 2C02 has done this.
and converted to a software algorithm. Next step is to feed the output of this algorithm into a software PAL/NTSC decoder... one with user adjustable realtime brightness/contrast/colour/tint controls.
That already exists. The problem is that not all TVs are adjusted the way you remember, and not all decoders in all makes and models of TV operate exactly the same. They may differ in their strategy to reduce the appearance of dot crawl or in their nonlinear hue warping to enhance skin tones. How many TV models do we want to emulate?

The point of comparing filtered composite output to that of the Super NES S-PPU is that the S-PPU is also capable of producing RGB at the same pixel rate and nonstandard scanline length as the NES. So if your video processing chain distorts NES video in a certain manner, it ought to be distorting Super NES video in the same way.
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Revisiting the quest for the most accurate NES palette:

Post by tokumaru »

tepples wrote:Decoding the composite signal to a texture doesn't work so well if you're trying to run an emulator on a device whose CPU clock speed is well below 1 GHz, or if you're (say) trying to port an NES game to another retro platform while preserving colors as closely as possible.
If you can't generate/decode the actual composite signal in real-time, can't you at least pre-compute an RGB palette by encoding a dummy video signal containing flat colors and decoding it according to the user's settings? Sure you wouldn't have the other peculiarities of the composite signal, like the color bleed out the dot crawl, but the basic colors would, in theory, be more accurate.
The point of comparing filtered composite output to that of the Super NES S-PPU is that the S-PPU is also capable of producing RGB at the same pixel rate and nonstandard scanline length as the NES. So if your video processing chain distorts NES video in a certain manner, it ought to be distorting Super NES video in the same way.
But once you get those RGB values and use them in another device, the colors will not be coming from an SNES anymore, so they'll go through completely different transformations before they reach the display, won't they?
AWJ
Posts: 433
Joined: Mon Nov 10, 2008 3:09 pm

Re: Revisiting the quest for the most accurate NES palette:

Post by AWJ »

tepples wrote:
DRW wrote:It is an analog signal, consisting of voltages. There is no "most accurate" palette.
What could be more accurate than the palette that, when fed to the Super NES S-PPU, produces the same voltages?
The SNES's video output path is completely unrelated to the NES's, so it's really no help. The SNES PPU outputs analog RGB which is converted to an NTSC or PAL signal by a separate encoder chip. The encoder is an off-the-shelf part, not a Nintendo custom, and different production runs of SNESes used different chips. And on top of that there are various resistors and capacitors between the PPU and the encoder and between the encoder and the multi-out jack, which introduces even more variability due to tolerances, aging, etc. on each of those components.

And even if you decide that the NTSC encoder in your SNES contains special Nintendo pixie dust, you still can't make it output "the same voltages" as a NES. The NES PPU outputs square waves, whereas proper NTSC encoders output sine waves. And many of the NES colors are more saturated than anything an RGB-to-NTSC encoder can output (not coincidentally, those are the colors that vary the most between TVs) You can program the SNES palette to RGB(31, 0, 0) and it's still not as red as the NES color $06.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: Revisiting the quest for the most accurate NES palette:

Post by NewRisingSun »

lidnariq wrote:The only possible remaining variation in colors would be if the output impedance from the NES varies significantly across the 10 different voltages the PPU can emit. And even then, that should really just come down to a little variation in net hue rotation, net brightness, or net saturation.
Here is a morsel of evidence for this phenomenon.

Attached please find find a raw capture of the output of a Sharp Twin Famicom (model AN-505BK), sampled at eight times the color burst frequency (28.6363.. MHz). They were made with an older Hauppage ImpactVCB card, for which custom drivers are available (credit to reengine) that allow getting the raw, undecoded signal, before the card or its drivers mess with it.

This particular game scene has color 0xC ("cyan") in all four levels in sufficiently large horizontal areas to allow measuring the phase at all four levels. If the color burst/color 0x8 is at 180 degrees, then color 0xC is nominally at 300 degrees.

The archive also includes the output of my custom NTSC decoder as a .BMP file. I configured it to use the sync amplitude as a reference for overall signal gain, which makes the entire image brighter while clipping the 3x colors a bit in the RGB colorspace.

The phases of color 0xC that I actually measure (using the unclipped YIQ signal) are:

0x0C: 298.565552
0x1C: 292.832520
0x2C: 288.335663
0x3C: 285.327179

That's quite a considerable a difference of almost 15 degrees between the reference and the brightest version of the color. All four levels were measured within the same horizontal line (147 in the .BMP file).

Since the shift increases with brightness and not with saturation, then assuming I understand what has been written in this thread correctly, it is not slew rate effects (saturation) that matter, but output impedance (level).

At least some game programmers must have had this in mind. For example, the Combat screen in Fire Emblem Gaiden normally uses colors $16 and $26 for the enemy stats panel. If the enemy is almost defeated, the panel goes dark, but instead of going to colors $06 and $16, it goes to colors $05 and $16, exactly what one would do to account for a phase shift between the brightness levels.

This phenomenon should not be observed with a PAL console, as the PAL system is specifically designed to difference out such differential phase shifts. Instead, the 3x colors become more desaturated compared to a NTSC system adjusted to similar phase and baseline saturation. This matches my observation of the PAL NES' picture.
Attachments
Kunio-Sky.zip
(248.19 KiB) Downloaded 370 times
lidnariq
Posts: 11429
Joined: Sun Apr 13, 2008 11:12 am

Re: Revisiting the quest for the most accurate NES palette:

Post by lidnariq »

NewRisingSun wrote:The phases of color 0xC that I actually measure (using the unclipped YIQ signal) are:

0x0C: 298.565552
0x1C: 292.832520
0x2C: 288.335663
0x3C: 285.327179
There's definitely not that much precision. But ... even assuming the fractional part is fictitious...

First I counted the total number of samples between the first end of hsync, and the last end of hsync. From that, I discovered that there was a 57ppm difference between the reference colorburst frequency of the NES and that of the capture card. (Hence 3.999772 below rather than 4)

Using this really simple matlab/octave script:

Code: Select all

% -*- octave -*-
KunioSkyFH = fopen("Kunio-Sky.raw");
KunioSkyData = fread(KunioSkyFH,Inf,"uint8");

Axis1 = KunioSkyData .* cos([1:rows(KunioSkyData)]'*pi/3.999772);
Axis2 = KunioSkyData .* sin([1:rows(KunioSkyData)]'*pi/3.999772);

% 5th order filter at 1.4MHz when sampled at 28.636MHz
% we don't care too much about having a "correct" filter.
% just enough to preserve small-ish horizontal details
[LPNum,LPDen] = butter(5,1.4/28.636);

Axis1LP = filter(LPNum,LPDen,Axis1);
Axis2LP = filter(LPNum,LPDen,Axis2);

plot(atan2(Axis1LP,Axis2LP)*180/pi);
hold on;
plot(180+KunioSkyData);
hold off;
[/size]

I extracted this one scanline:
Kunio-Sky_scanline147_highlighted.jpg
And got this plot of phase (top: samples; bottom: instantaneous chroma angle, note that it is nonsense during black or white pixels)
scanline147.png
If I zoom in (not included), I see ≈12 degrees of difference between the instantaneous angle of color $0C and color $2C. Maybe I made a mistake? You only saw ≈10 degrees.
Since the shift increases with brightness and not with saturation, then assuming I understand what has been written in this thread correctly, it is not slew rate effects (saturation) that matter, but output impedance (level).
It would be lovely if you could test using other hardware. Other revisions of the 2C02, and other analog paths out of the PPU could easily change this exact behavior.
At least some game programmers must have had this in mind. For example, the Combat screen in Fire Emblem Gaiden normally uses colors $16 and $26 for the enemy stats panel. If the enemy is almost defeated, the panel goes dark, but instead of going to colors $06 and $16, it goes to colors $05 and $16, exactly what one would do to account for a phase shift between the brightness levels.
I'm not convinced that's related. I have seen pixel artists talk about intentional hue shifts as things get darker.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: Revisiting the quest for the most accurate NES palette:

Post by NewRisingSun »

lidnariq wrote:There's definitely not that much precision
No, I was just giving you the raw numbers.
lidnariq wrote:It would be lovely if you could test using other hardware. Other revisions of the 2C02, and other analog paths out of the PPU could easily change this exact behavior.
The AN505-BK has a G revision PPU. I have another Twin Famicom with an E revision, and then of course I could capture directly at the output pins of both PPUs.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: Revisiting the quest for the most accurate NES palette:

Post by NewRisingSun »

According to my results, the phase shift seems to be not constant for all the phases within one of the four levels either. I measure color $31 at 319.9 degrees (nominal: 330), or a shift of 10.1 degrees, and color $36 at 113.1 degrees (nominal: 120), or a shift of just 6.9 degrees. I'm hoping that I'm just doing something wrong, because it would really be annoying and unfortunate if that were true. (This is from the same setup as the other captured image. I'll do the alternative methods once I have a flash cartridge that allows me to upload and run a program that displays, and allows me to capture, all colors at once.)
Attachments
PacLand-Start.zip
(442.88 KiB) Downloaded 350 times
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Revisiting the quest for the most accurate NES palette:

Post by tepples »

If the shift is one amount for even phases but a different amount for odd phases, then the duty cycle of the master clock may have something to do with it.
lidnariq
Posts: 11429
Joined: Sun Apr 13, 2008 11:12 am

Re: Revisiting the quest for the most accurate NES palette:

Post by lidnariq »

Do you have a flashcart? If so, it would make things much easier if you just use either Blargg's Full Palette demo or anything else where I can identify specific color by definition/position rather than having to cross-reference it to a picture?

I should probably do this same matlab graph on the previous oscilloscope log of HardWareMan's Dendy.
Post Reply