Merging CHR ROM bit planes

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems.

Moderator: Moderators

Post Reply
User avatar
Goose2k
Posts: 104
Joined: Wed Dec 11, 2019 9:38 pm
Contact:

Merging CHR ROM bit planes

Post by Goose2k » Thu Aug 13, 2020 10:03 am

I am trying to display the graphical data from a *.chr file for a tool I am writing.

Nintool.png
Nintool.png (12.04 KiB) Viewed 952 times

Merging the bit planes seems super awkward with my naive approach. Doesn't really matter, since its just a tool, but I was thinking the system must use this format for a reason, and there must be some algorithm which makes this very straight forward. Otherwise, it would seem easier to just store 2 bits together, rather than 64 bits apart. Maybe its a hardware thing?

The bitplanes are stored in 64 bit chunk pairs; basically 1 8x8 sprite at a time. So my thought was to read in both bit planes for a given 8x8 area. Then go bit by bit through each plane (plane a and plane b) and calculate the final value with:

Code: Select all

for each bit:
final_val = a_bit | (b_bit << 1)
I'm wondering if I am missing something and there is a way to do the whole 64 bit block in one operation.

lidnariq
Posts: 9659
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: Merging CHR ROM bit planes

Post by lidnariq » Thu Aug 13, 2020 10:21 am

There's no way the bit swizzling is a significant computational load... I'd recommend just making your code easily understood instead of fast here.

If you really want to make it faster there might be a technique here.

(The reason for the bit ordering is that the NES is planar, and each bitdepth is separate. Many other video game consoles - SNES, Master System, Game Boy, but notably not the Genesis - also use planar storage)

User avatar
tokumaru
Posts: 11858
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Merging CHR ROM bit planes

Post by tokumaru » Thu Aug 13, 2020 10:26 am

The typical approach would be to fetch one byte (plane 0), then the byte 8 positions ahead (plane 1), and loop 8 times, getting 1 bit from each plane to form a 2-bit value and shifting both planes for the next iteration.

Converting from planar format to packed format will require a good amount of bit manipulation, I don't think there's a way around that. Even if the NES used a 2-bit packed format to begin with you'd still need to do a lot o bit manipulation, seeing as you'd still have to "extract" the bit pairs from the packed byte.

calima
Posts: 1186
Joined: Tue Oct 06, 2015 10:16 am

Re: Merging CHR ROM bit planes

Post by calima » Thu Aug 13, 2020 10:27 am

I do seem to recall a bit scatter instruction in one of the recent AVXes. Using it may speed up your conversion by 20-40x, bringing your total load time from 960us to 870us :lol:

User avatar
tokumaru
Posts: 11858
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Merging CHR ROM bit planes

Post by tokumaru » Thu Aug 13, 2020 10:34 am

As far as hardware is concerned, there are no "awkward formats", because all the address and data lines are exposed, so rearranging then is essentially free. In software though, we're bound by how the memory is laid out and by the width of the words the CPU allows us to access/manipulate.

User avatar
Goose2k
Posts: 104
Joined: Wed Dec 11, 2019 9:38 pm
Contact:

Re: Merging CHR ROM bit planes

Post by Goose2k » Thu Aug 13, 2020 10:51 am

Thanks for the info. Good to know I wasn't missing something.
lidnariq wrote:
Thu Aug 13, 2020 10:21 am
There's no way the bit swizzling is a significant computational load... I'd recommend just making your code easily understood instead of fast here.
No concerns for performance at this point. Just felt like I might be missing something. :D
tokumaru wrote:
Thu Aug 13, 2020 10:26 am
The typical approach would be to fetch one byte (plane 0), then the byte 8 positions ahead (plane 1)
That makes sense.

User avatar
Controllerhead
Posts: 148
Joined: Tue Nov 13, 2018 4:58 am
Location: $4016
Contact:

Re: Merging CHR ROM bit planes

Post by Controllerhead » Thu Aug 13, 2020 10:55 am

I have a loop with 5 iterators basically. It spits out a (16 tile/row) left -> right "raster" format of 0,1,2,3; and then i apply palette values and dimensions. I'm too stupid to I kind of remember how this works exactly, but it does. Hope this helps!

Code: Select all

function DATA_loadFileCHR(path)
  DATA_file = io.open(path, "rb")
  DATA_fileString = DATA_file:read("*a")
  DATA_file:close()
  
  i = 0
  iUntil = 8192 -- Byte count / 2
  
  j=0
  jUntil= -1 -- (7 to 0)
  
  k=0 -- Column
  l=0 -- Pixel Row
  m=0 -- Row of tiles
  
  while(i < iUntil) do
    -- It loads a CHR file
    b = DATA_fileString:byte(i+1) -- Each byte; +1 because of stupid Lua
    b2 = DATA_fileString:byte(i+9) -- Each byte; +1 because of stupid Lua
	
    j=7
    while(j > jUntil) do
	get = j + k + l + m
	set = (b % 2) + ((b2 % 2) * 2) -- 0, 1, 2, or 3
	  
	GUI_chrNES[get] = set
	  
	b = b >> 1 -- Next bit
	b2 = b2 >> 1 -- Next bit
	  
	j=j-1
    end

    -- These iterators convert 8x8 to a continuous row of 16 tiles ...somehow
    k = (k + 128) % 1024 -- Column
    if(k==0) then l = l + 8 end -- Pixel Rows
	
    i=i+1 -- Next byte
    if(i%8 == 0) then i=i+8 end -- Goto next byte row
    if(i%256 == 0) then m=m+896 end -- Goto next tile row
  end
  --emu.log("ok loaded CHR")
end
Image
Last edited by Controllerhead on Thu Aug 13, 2020 6:44 pm, edited 5 times in total.
Image

User avatar
Goose2k
Posts: 104
Joined: Wed Dec 11, 2019 9:38 pm
Contact:

Re: Merging CHR ROM bit planes

Post by Goose2k » Thu Aug 13, 2020 4:00 pm

Here's what I ended up with (C#)

Code: Select all

// Test file loaded as a byte array.
byte[] file = System.IO.File.ReadAllBytes("C:\\Users\\Matt\\Documents\\Dev\\NES\\testrom.chr");

// The final format of the data will be a byte array, with each element representing a single pixel.
// It only needs 2bpp, but byte is simpler to work with.
const byte pixels_per_byte_in_source = 4;

// Allocate the final byte array, keeping in mind that in the CHR file, it is using 2 bit planes, so for every byte of
// data we are only defining 4 pixels. It takes 16 bytes to define a row of 8 pixels (2 bits per pixel).
byte[] final_chr = new byte[file.Length * pixels_per_byte_in_source];

// Tracks the current index we will be storing in the final chr rom array.
int final_chr_index = 0;

// Go through the file row by row. Note that we jump by increments of 16, because each row is
// 8 pixels wide, and there are 2 bits per pixel.
// Note that the order of the data is in 8x8 chunks, then moving left to right. 
for (int i = 0; i < file.Length; i += 16)
{
    // Loop through each byte in the row (in pairs).
    for (int j = 0; j < 8; ++j)
    {
        // Row of bit plane 1.
        int ByteA = file[i + j];
        // Row of bit plane 2 (8 bytes later in the file).
        int ByteB = file[i + j + 8];

        // Merge each bit.
        for (int k = 7; k >= 0; --k)
        {
            // Mask starting with the highest (left) bit, and working our way down.
            int mask = (1 << k);

            // Mask out the bit in the first and second bit plane. Then shift the second value left 1 bit, and merge
            // with OR operator. Resulting in, for example:
            // 01 | 10 = 11 (3)
            int final_bit = ((ByteA & mask)>>k) | (((ByteB & mask) >> k) << 1);

            // Store the merged bit planes as a single byte.
            final_chr[final_chr_index] = (byte)final_bit;
            ++final_chr_index;
        }
    }
}

User avatar
Controllerhead
Posts: 148
Joined: Tue Nov 13, 2018 4:58 am
Location: $4016
Contact:

Re: Merging CHR ROM bit planes

Post by Controllerhead » Thu Aug 13, 2020 5:56 pm

Goose2k wrote:
Thu Aug 13, 2020 4:00 pm
// Note that the order of the data is in 8x8 chunks, then moving left to right.
Oh yeah! I remember now. That's what all my iterator wackiness "solves" into rows of 16 tiles :roll:

Your code looks clean!
Image

User avatar
aa-dav
Posts: 102
Joined: Tue Apr 14, 2020 9:45 pm
Location: Russia

Re: Merging CHR ROM bit planes

Post by aa-dav » Fri Aug 14, 2020 11:49 am

Goose2k wrote:
Thu Aug 13, 2020 10:03 am
Otherwise, it would seem easier to just store 2 bits together, rather than 64 bits apart. Maybe its a hardware thing?
None of the things below are applicable to NES CHR data, but bitplanes have several advantages in general:
a) videochip could be designed to work with several RAM chips in parralel and increasing of planes count do not affect reading/writing cycles (you just add yet another RAM chip in parralel). However access by CPU becomes slower. This is more applicable to bitmap screens.
b) 8bit CPUs shift bytes by 1 bit per instruction with drop-out in carry flag. This fact becomes serious trouble then you need to write smooth left/right per-pixel scrolling of continuous pixel data (several bytes in width). if you pack 1 pixel in several bits in the same byte you need to save carry flag between adjacent bytes several times and restore it. So it is really faster to scroll it twice by 1 bit by simple carry-in-carry-out cycle. And this is how you act with bit planes, so they don't mean loss of speed. Packing of bits in 1 byte are something not good for 8 bit sometimes.

Again, all of these didn't apply to NES, but I think idea of bit-planes was fine and easy at the time and they just have nothing against it by force of habit.

Post Reply