To Test My ROM Loader

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

Post Reply
User avatar
HouseCat
Posts: 4
Joined: Thu Aug 25, 2022 10:03 am
Location: Midwest, USA

To Test My ROM Loader

Post by HouseCat »

Greetings to the community of NESdev / emdev! I'm brand new here and to Internet forums and to the world of the 6502!
Should I say more to introduce myself or my project? Shall I get straight to the point?

I just finished writing code to load a NES file. What's a good way to test it? (I need to test the code that loads the test's code. <LOL>)

If I had a bunch of ROM images, I'd write unit tests that load them & dump the header info. Then I'd compare those dumps with known-good ones. However, I don't have a bunch of ROMs, and I'm not interested in SW piracy. The ideal test data would be a *variety* of real, working ROM files that have had their PRG & CHR data wiped/overwritten leaving the rest intact--including the file size. (PRG could be solid EAh, and CHR could be solid 00h.)

Is there a place to find a bunch of blank NES files with valid headers? Or, is there a way to collect some?
Or is there a better approach? What do you think?
.=^ ^=
Hôûse Cat
. . ⎭
Fiskbit
Posts: 890
Joined: Sat Nov 18, 2017 9:15 pm

Re: To Test My ROM Loader

Post by Fiskbit »

Welcome!

There are many free test ROMs available for use in verifying that your emulator matches real hardware. There is a collection on GitHub , and we also link many from the wiki, although I think our wiki list is in need of some maintenance.

Test ROMs don't test every behavior a real game might need, and real games don't use every behavior that test ROMs are testing for, so you may want to take a look at least at some of the tricky-to-emulate games listed on the wiki.

As for where to start, blargg's instr_test_v5 and kevtris' nestest may be worth a look.
User avatar
HouseCat
Posts: 4
Joined: Thu Aug 25, 2022 10:03 am
Location: Midwest, USA

Re: To Test My ROM Loader

Post by HouseCat »

Hi, Fiskbit! Thanks!

Sorry, I guess I didn't explain clearly enough. Please let me clarify my question: I'm not worried about testing my emulation/accuracy yet. I downloaded Nestest to test my CPU, but it's in a .NES file, so I wrote a ROM file loader to read the file. My loader is a bunch of code to read the header and check for PRG size, CHR size, RAM size, mirroring type, signature, title, which ROM type, trainer, which mapper, NES2.0, etc. ... and I want to know, does all *that* work? Is it reading the header correctly? Can it identify all types like PlayerChoice and FDS ROMs? ATM, I'm looking for a way to exercise all that code without touching the CPU. After I test my file loader--only after that, so not yet--I want to put the loader and CPU together and see if it'll run Nestest.
.=^ ^=
Hôûse Cat
. . ⎭
Fiskbit
Posts: 890
Joined: Sat Nov 18, 2017 9:15 pm

Re: To Test My ROM Loader

Post by Fiskbit »

Ah, gotcha. I...don't know if there is any sort of test for this. You may get some mileage out of the NES 2.0 XML database, which contains header information for most known ROMs, but this probably does not cover test ROMs and I'm not sure how much coverage it has for free homebrew games. This is also complicated a bit by many ROMs being iNES, while this database is of NES 2.0 data, though correct iNES and NES 2.0 headers won't typically have contradictory information. Furthermore, many ROMs also have bad headers, so emulators will use the database instead of the ROM header if the ROM is found there because otherwise the ROM would not work.

That all said, what you can do is look up the ROM in the database and, if there's a match, see if the way you decoded the header matches the database, and if it doesn't match, you can check to see if it's a problem with your code or the ROM.

The other thing is that, at least early on in an emulator, you don't need to support most of what the header offers. The vast majority of ROMs can be handled probably just by decoding the PRG size, CHR size, 8-bit mapper number, region, and mirroring, and using some heuristics for particular mappers where necessary. NES 2.0 allows you to emulate stuff without having to make any guesses and adds support for more obscure hardware, but we got by for a long time with just the first 8 bytes of iNES headers.

Maybe someone else here will have better ideas to help you out.
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: To Test My ROM Loader

Post by tepples »

Holy Mapperel binary release sounds like what you want.
User avatar
HouseCat
Posts: 4
Joined: Thu Aug 25, 2022 10:03 am
Location: Midwest, USA

Re: To Test My ROM Loader

Post by HouseCat »

Fiskbit wrote: Tue Nov 22, 2022 11:08 pm Ah, gotcha. I...don't know if there is any sort of test for this. You may get some mileage out of the NES 2.0 XML database, which contains header information for most known ROMs, but this probably does not cover test ROMs and I'm not sure how much coverage it has for free homebrew games. This is also complicated a bit by many ROMs being iNES, while this database is of NES 2.0 data, though correct iNES and NES 2.0 headers won't typically have contradictory information. Furthermore, many ROMs also have bad headers, so emulators will use the database instead of the ROM header if the ROM is found there because otherwise the ROM would not work.
I think you're right about getting a lot of mileage out of the database. Thanks for suggesting it! It would be easy to write a program to simply read all the entries and write out matching dummy ROM files to use for testing. I'll need to make it more efficient than that--I need one of each variety, not all of them--and I want to parameterize it so I can target specific features... I have a path forward now. <beaming smile> I think I'll call this my "Dummy ROM Generator."

I'm not worried about invalid headers--not beyond rejected anything that fails simple checks. I figure I'll leave the task of correcting bad headers to other software.

The remaining problem is, if I write code that reads the database and writes a header to a file and then run a unit test that uses my loader to read it from file and compare it to the database, I could easily make the same mistake in both places. Just because they agree with each other does not mean they are correct. They could both be wrong. In other words, I need to test my Generator before I can trust it to test my ROM loader. Otherwise, I need to test the loader before I can trust it to test my Generator. It's kind of a bootstrap problem.

I guess the obvious solution is to try loading the dummies into some existing emulators. If those emulators can be trusted, and the ROMs load up just fine, then my Generator can be trusted too.
Fiskbit wrote: Tue Nov 22, 2022 11:08 pm The other thing is that, at least early on in an emulator, you don't need to support most of what the header offers. The vast majority of ROMs can be handled probably just by decoding the PRG size, CHR size, 8-bit mapper number, region, and mirroring, and using some heuristics for particular mappers where necessary. NES 2.0 allows you to emulate stuff without having to make any guesses and adds support for more obscure hardware, but we got by for a long time with just the first 8 bytes of iNES headers.
Good point. If I can handle NES 2.0, and I'm fine requiring it, then I don't have to worry about the older formats. But, I don't want to require 2.0, and I already wrote code to handle older formats, and I want to test it...

Haha! Good point! Now I'm laughing at myself! My NES implementation won't be too elaborate. I got carried away making my ROM loader far more ambitious than the original project! <still chuckling> ...I still want to test it...

--House Cat
User avatar
HouseCat
Posts: 4
Joined: Thu Aug 25, 2022 10:03 am
Location: Midwest, USA

Re: To Test My ROM Loader

Post by HouseCat »

tepples wrote: Fri Nov 25, 2022 12:38 pm Holy Mapperel binary release sounds like what you want.
That looks like it's designed for testing cartridge PCB assembly which is not what I'm after. However, I think it'll be very useful for testing mapper implementations in the future. Thanks, Tepples!

Also, if it does output NES files with different headers, then I can use them for testing my ROM loader. It's limited, but it helps. Thanks again!

--House Cat
Joe
Posts: 649
Joined: Mon Apr 01, 2013 11:17 pm

Re: To Test My ROM Loader

Post by Joe »

HouseCat wrote: Tue Nov 29, 2022 2:20 pmAlso, if it does output NES files with different headers, then I can use them for testing my ROM loader.
You can find the NES files inside holy-mapperel-bin-0.02.7z in the "assets" section of that page. The file names indicate the header contents.
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: To Test My ROM Loader

Post by tepples »

The build process of Holy Mapperel generates ROMs based on a list of header specifications in make_roms.py.
Post Reply