maybe a nice trick: storing metadata in palette tables

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems.

Moderator: Moderators

Post Reply
User avatar
FrankenGraphics
Formerly WheelInventor
Posts: 2030
Joined: Thu Apr 14, 2016 2:55 am
Location: Gothenburg, Sweden
Contact:

maybe a nice trick: storing metadata in palette tables

Post by FrankenGraphics » Sun Jan 06, 2019 6:14 am

This whole post is contingent on that i've gotten a thing or two about the PPU right and that ocular observation in emulators is worth something, but..

The two most significant bits in stored palette data are never used. If you toggle them on/off, the colour is the same. I'm assuming the PPU is discarding those. You can use this bitspace to include some metadata on how to handle the colours themselves, or some hardware register. Either immediately when loading, or for later if stored in a buffer in RAM. On top of my head, it might make sense to use the sign flag bit (msb) for either:

-toggling of BGR bits
-enable/disable switches for palette buffer animation

you could also rotate/shift bits in/out to produce an alternate palette from the same, but it doesn't seem very practical.

So in my palette data, i might store something like this

Code: Select all

SIGN = %10000000

(...) 

MyPalette:
.db $00, $01, $11, $21 | SIGN
There's no reason you can't store information about emphasis bit toggling or how to enable colour animation in some other scheme, but having it organized in the same place makes sense to me.

For the emphasis scheme especially, I think the 1st byte in the strip should reserved for enabling the other BGR bit readings, thus selecting if its particular BGR setting should be used or not. By manually writing an enable bit into any of the 8 subpalette strips stored in the buffer, you could hypothetically keep all 8 combinations in the buffer at once, even if it's highly unlikely (no bits set is the most useful setting, after all). You can also encode the first byte directly in data if you actually want it to always take mandate, which might be the case sometimes.

Alternately, you could maybe use it to let your engine make assumptions. "If this palette is used, we want to set vertical mirroring", or the like.
http://www.frankengraphics.com - personal NES blog

User avatar
pubby
Posts: 536
Joined: Thu Mar 31, 2016 11:15 am

Re: maybe a nice trick: storing metadata in palette tables

Post by pubby » Sun Jan 06, 2019 11:35 am

It's a fun idea but I'm not really seeing the benefit to doing so.

The alternative would be to store an extra byte per palette and not worry about compression. If you use 10 palettes per game, that'd cost 10 bytes extra. That's a pretty tiny amount.

Keep in mind that whatever code you'd need to decompress this format takes up space too. Probably over 10 bytes.

lidnariq
Posts: 8698
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: maybe a nice trick: storing metadata in palette tables

Post by lidnariq » Sun Jan 06, 2019 12:13 pm

The most obvious thing to me to do with metadata like this is describe how the color should be faded in/out. It's probably only be useful in situations where the used palettes change over the course of a level, though, such as due to dynamic allocation per enemy or at a fixed point in the level.

User avatar
FrankenGraphics
Formerly WheelInventor
Posts: 2030
Joined: Thu Apr 14, 2016 2:55 am
Location: Gothenburg, Sweden
Contact:

Re: maybe a nice trick: storing metadata in palette tables

Post by FrankenGraphics » Sun Jan 06, 2019 12:38 pm

Yeah admittedly it's just that, an idea i had while doing assets-oriented stuff, and am entertaining here. I probably don't see the full picture of engine design that someone with a programmers' background would.

I guess one benefit would be that you might be able to auto-imply some changes to hardware registers based on when palettes are used (for example the colour emphasis) since they're hard-coded into the palette. So you don't really need any decision-making routines for when to as long as the game is with this implication in mind. It can also be less rule-bound than other ways of implying hardware writes. For instance, metroid toggles horizontal and vertical scrolling between room switches, which makes for a very rulebound level design.

I don't think it's any use caring about the 2nd most significant bit since you can't evaluate it directly.
http://www.frankengraphics.com - personal NES blog

User avatar
Bregalad
Posts: 7752
Joined: Fri Nov 12, 2004 2:49 pm
Location: Chexbres, VD, Switzerland

Re: maybe a nice trick: storing metadata in palette tables

Post by Bregalad » Sun Jan 06, 2019 12:56 pm

If you use colour emphasis, there's 3 bits of information but palette entries only have 2 unused bits. So either you'd have to encode it somehow (for example : 0=> No emphasis; 1=> Red emphasis; 2=>Green Emphasis; 3=>Blue emphasis), or you have to split the information among multiple palette entries (there's typically 32 entries but only one colour emphasis for the screen (unless you dedicate the CPU only to that) ).

Actually I do something close to what you say for the game I've been coding since 13 years. I do not use the upper 2 bits of the palette, however I use the unused backgound colours of the BG palette (indexes 4, 8 and 12) to store the starting position of the player in a level. The reason I did it that way is because both datas are linked when a level is loaded, and there was exactly 3 information for the starting position : (X position, Y position and direction), so this fitted the 3 gaps perfectly. Also I do not care to discard the data, so technically garbage palettes will be written to PPU $3f04, $3f08 and $3f0c. This is however discarded on a real NES.

Another thing I did is use the unused bits of sprite attributes in order to compress sprites. I am actually quite proud of what I did there. I was aiming to use both PRG and CHR ROM as efficiently as possible. Imposing some kind of layout to metasprites, such as what Super Mario Bros or Final Fantasy does, is efficient for PRG-ROM but wasteful for CHR-ROM. So I decided to store all metasprites in a very classic format, where something close to the hardware format is used with 4 bytes per hardware sprite, and only X and Y offset are added with screen position to find the final position of the sprite.

Unfortunately this is wasteful for ROM, so I used some compression format allowing sprites defined the "traditional" way to be retro-compatible, so that both new compressed and old sprites would merge nicely. Compressed meta-sprites can scrap information by "guessing" either of the 3 bytes - the X or Y position or tile #. In the unused bits of each attribute sprites I have several options which says "the next sprite is 8 pixels to the left", "the next sprite is 8 pixels to the bottom", or "the next sprite has an index of the current tile + 1", which is situations that appears very often. That way I can remove lots of information in the metasprites definitions, saving a lot of ROM, and still be completely free of defining metasprites however I like, saving CHR-ROM. I also have a bit set on the last sprite in a metasprite, so I don't have to waste an additional byte to say how many sprites there is in a metasprtie.

User avatar
FrankenGraphics
Formerly WheelInventor
Posts: 2030
Joined: Thu Apr 14, 2016 2:55 am
Location: Gothenburg, Sweden
Contact:

Re: maybe a nice trick: storing metadata in palette tables

Post by FrankenGraphics » Sun Jan 06, 2019 2:38 pm

Bregalad wrote:f you use colour emphasis, there's 3 bits of information but palette entries only have 2 unused bits.
Sorry, that wasn't the idea. The second suggestion in your following passage was on the other hand the point. Maybe i should've worked a bit more on the example.

The idea is there's 4 bytes in one subpalette strip. While it is true that there are 2 bits in each that are used, only the MSB is directly available. So:
Check first byte for the sign flag to decide whether to go through and pick up the rest: 3 more sign flags.

In other words -
.db firstByte ; 7th bit - enable/disable checking the rest
.db 2ndByte ; 7th bit - B emphasis
.db 3rdByte ; 7th bit - G emphasis (on ntsc)
.db 4thByte ; 7th bit - R emphasis (on ntsc)

if we're using the emphasis bits as an example again.

I like lidnariqs' notion that these bits could be used to tell something about how to fade in/out since or change over some trigger over the course of some level since it's closely related to the purpose of the contents of the table. But i also think anything that can be somewhat isolated to the circumstance(s) under which the specific palette strip is going to be used could do. More substantioal palette changes are often done during changes of rooms or stages, so that's one area. Which you did with this, bregalad:
however I use the unused backgound colours of the BG palette (indexes 4, 8 and 12) to store the starting position of the player in a level. The reason I did it that way is because both datas are linked when a level is loaded, and there was exactly 3 information for the starting position : (X position, Y position and direction)
I think that's pretty neat! :beer: I assume each level then has a fixed set of subpalettes each.
So I decided to store all metasprites in a very classic format, where something close to the hardware format is used with 4 bytes per hardware sprite, and only X and Y offset are added with screen position to find the final position of the sprite.
From an artists' prespective, this is preferrable to me too, generally. Saving space in the chr tables and the occasional saving on the sprites per scanline bandwidth is generally more important to me than ROM size, since even historical mappers could have "plenty" of ROM. Plus you can do more elaborate animations with an OAM-format animation tables due to being able animating positions and attributes.

Another option to get more bits being unused for more guesses in your compression scheme would be to assume that sprites can't have an offset larger than some power of 2. Maybe a range of 0-63 is enough for the x and y offset data. Shiru's metasprite list generator in NESST are btw using bit 7 in a termination byte. You could have something else, like $ff (since it'd be unreasonable to give a sprite an Y offset like that) increase and check for Z flag. Which'd let you free up bit 7 for something else unless in the case a sprite really would need to be at the very end of the range of the offset.
http://www.frankengraphics.com - personal NES blog

User avatar
rainwarrior
Posts: 7642
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: maybe a nice trick: storing metadata in palette tables

Post by rainwarrior » Sun Jan 06, 2019 3:26 pm

Instead of storing things related to colour, might be easier to store just some unrelated bitfield in there, if you have some ROM bitfield need that is shorter than the length of the palette data.

Unfortunately BIT doesn't have an indexed version, but LDA ..., X will nicely load the high bit into the sign flag. Might even be less code than trying to extract bits from an equivalent packed bitfield.

Post Reply