Nametable compression with 4-way scrolling

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems.

Moderator: Moderators

Post Reply
pwnskar
Posts: 116
Joined: Tue Oct 16, 2018 5:46 am
Location: Gothenburg, Sweden

Nametable compression with 4-way scrolling

Post by pwnskar » Wed Mar 25, 2020 9:24 am

I've managed to implement 4-way scrolling in my never-ending NES project. It's in a sort of working state using metatile compressed nametable data.

Now I'm curious how other people store their maps for 4-way scrolling.

Personally I divide my levels in "screens", where each is 240 bytes. Every 16 bytes represent two nametable rows and each byte is an index to a metatile stored in another list. By storing the data this way I can convert my scroll values to an index in the appropriate "screen" from where I should start fetching metatiles to buffer for next NMI.

It works and all but I find that with my test level of only 4 "screens", I get a compression rate of 50%. I don't know if this is good or bad but either way I'm wondering if there could be a better way to compress the data other than just upping the size of my metatiles from 2x2 to 4x4 tiles.

Would RLE be a realistic choice or just too complex to unpack for this case? I've never done any RLE stuff before so I have no idea what to expect but I guess maybe each "screen" would have to have a lookup table for each row and those rows would then be compressed using RLE. But that sounds like it might be slow to unpack when fetching an entire column for the side of the screen? Especially the farther you go to the right of the screen, as each row would have to be unpacked to the very end..

Anyways, here's some example data from my levels:

Code: Select all

example_level:

	; Header
	@width:		.db 1
	@height:	.db 1
	@metatiles_ptr:		.dw @metatiles

@nametable_0:
	.db	  0,  1,  2,  0,  1,  2,  0,  1,  2,  0,  1,  2,  0,  1,  2,  3
	.db	  0,  1,  2,  0,  1,  2,  0,  1,  2,  0,  1,  2,  0,  1,  2,  3
	.db	  0,  1,  2,  0,  1,  2,  0,  1,  2,  0,  1,  2,  0,  1,  2,  3
	.db	  0,  1,  2,  0,  1,  2,  0,  1,  2,  0,  1,  2,  0,  1,  2,  3
	.db	  0,  1,  2,  0,  1,  2,  0,  1,  2,  0,  1,  2,  0,  1,  2,  3
	.db	  0,  1,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3
	.db	  0,  1,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3,  4,  5,  6,  7
	.db	  0,  1,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3,  8,  9, 10, 11
	.db	  0,  1,  2,  3,  3,  4,  5,  5,  6,  6,  7,  3,  3,  3,  3,  3
	.db	  0,  1,  2,  3,  3,  8,  9, 10, 10, 10, 11,  3,  3,  3,  3,  3
	.db	  0,  1,  2,  0,  1,  2,  0,  1,  2,  3,  3,  3,  3,  3,  3,  3
	.db	  0,  1,  2,  0,  1,  2,  0,  1,  2,  3,  3,  3,  3,  3,  3,  3
	.db	  0,  1,  2,  0,  1,  2,  0,  1,  2,  3,  4,  5,  6,  7,  3,  3
	.db	  0,  1,  2,  0,  1,  2,  0,  1,  2,  3,  8,  9, 10, 11,  3,  3
	.db	 12, 12, 12,  0,  1,  2,  0,  1,  2,  3,  3,  3,  3,  3,  3,  3

@attributes_0:
	.db		$e3, $49, $e3, $49, $e3, $49, $e3, $01
	.db		$49, $e3, $49, $e3, $49, $e3, $49, $23
	.db		$e3, $09, $03, $09, $03, $09, $03, $01
	.db		$49, $23, $00, $00, $00, $00, $aa, $ab
	.db		$e3, $01, $88, $a2, $a8, $22, $00, $00
	.db		$49, $e3, $49, $e3, $01, $00, $00, $00
	.db		$e3, $49, $e3, $49, $23, $aa, $a9, $00
	.db		$04, $03, $09, $03, $01, $00, $00, $00

@nametable_1:
	.db	blablablabla...
	....

@attributes_1:
	.db		blabla....
	.....
	
@metatiles:
	.db		$93, $94, $a3, $a4
	.db		$95, $96, $a5, $a6
	.db		$97, $98, $a7, $a8
	.db		$00, $00, $00, $00
	.db		$01, $02, $10, $11
	.db		$03, $04, $26, $26
	.db		$05, $06, $26, $26
	.db		blablabla and so on.......
		

Should I just stick with what I've got and maybe up the metatile sizes to 4x4? Thanks for reading this far.

Cheers! :beer:

User avatar
tokumaru
Posts: 11561
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Nametable compression with 4-way scrolling

Post by tokumaru » Wed Mar 25, 2020 11:44 am

Compression schemes that don't allow for random access to the data (RLE, LZ, etc.) will add significant complexity to your scrolling engine, in addition to requiring more RAM and CPU time.

TBH, 50% compression sounds terrible for map data, but since each byte represents 4 tiles, each screen should actually be only 25% of a raw nametable. There's of course the overhead of the metatiles themselves, but the more screens you have using the same set of metatiles, the less that overhead will be.

What you can do is add more layers of reusable blocks. Instead of having each screen be built out of 16x16-pixel metatiles, they can be built out of 32x32pixel blocks which in turn are built from 4 16x16-pixel metatiles. I've used that concept to the extreme in my own scrolling engine, where every entity in the level map is built from 4 smaller structures: The level map is an array of 256x256-pixel blocks, which are built from 4 128x128-pixel blocks, which are built from 4 64x64-pixel blocks, which are built from 4 32x32-pixel blocks, which are built from 4 16x16-pixel metatiles, which are built from 4 tiles and have extra attributes (palette index, collision data, etc.). Levels structured like this are a pain to build by hand though, so you'd most likely need to code a converter to generate all the blocks automatically from the raw map data.

User avatar
dougeff
Posts: 2654
Joined: Fri May 08, 2015 7:17 pm
Location: DIGDUG
Contact:

Re: Nametable compression with 4-way scrolling

Post by dougeff » Wed Mar 25, 2020 2:02 pm

If you use the $6000-7fff region as extra RAM, you can unpack the compressed level here, just once at the start of each level.
nesdoug.com -- blog/tutorial on programming for the NES

pwnskar
Posts: 116
Joined: Tue Oct 16, 2018 5:46 am
Location: Gothenburg, Sweden

Re: Nametable compression with 4-way scrolling

Post by pwnskar » Wed Mar 25, 2020 2:26 pm

tokumaru wrote:
Wed Mar 25, 2020 11:44 am
What you can do is add more layers of reusable blocks. Instead of having each screen be built out of 16x16-pixel metatiles, they can be built out of 32x32pixel blocks which in turn are built from 4 16x16-pixel metatiles. I've used that concept to the extreme in my own scrolling engine, where every entity in the level map is built from 4 smaller structures: The level map is an array of 256x256-pixel blocks, which are built from 4 128x128-pixel blocks, which are built from 4 64x64-pixel blocks, which are built from 4 32x32-pixel blocks, which are built from 4 16x16-pixel metatiles, which are built from 4 tiles and have extra attributes (palette index, collision data, etc.).
That sounds clever. I was thinking of storing collision data with the map data as well but have not yet decided if i should just store it as 960 bits for each tile in every screen or if I should mark certain tiles as collidable and store in RAM as the map data is being buffered.
tokumaru wrote:
Wed Mar 25, 2020 11:44 am
Levels structured like this are a pain to build by hand though, so you'd most likely need to code a converter to generate all the blocks automatically from the raw map data.
Yes, I'm using python to compress my raw nametable data from NES Screen Tool. Doing it by hand would get old quick. :)

I'm trying to up the metatile sizes to see if I can shrink the data down even more but it's not showing any huge changes. Though like you said, the size is distributed away from the nametables and to the metatiles, so with bigger maps it would perhaps pay off.

Post Reply