A note to all NES-emulator authors: ROM header wishlist

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

User avatar
MottZilla
Posts: 2837
Joined: Wed Dec 06, 2006 8:18 pm

Re: A note to all NES-emulator authors..

Post by MottZilla »

byuu wrote: So I've pretty much come to the conclusion that there is no solution to this problem.

Support iNES 1.0, support iNES 2.0, support as many external board markups as you can, support an internal database, cross your fingers and hope for the best.
This pretty much sums it all up. Everyone just needs to live with it.
User avatar
blargg
Posts: 3715
Joined: Mon Sep 27, 2004 8:33 am
Location: Central Texas, USA
Contact:

Re: A note to all NES-emulator authors..

Post by blargg »

You'll never have an end-all solution that covers everything. The best you can do is have a dual approach: self-contained format(s) for describing new games, and a database of known games. The former allows a determined person to make a game work. The latter accounts for all the existing files out there to make them work without having to convert them to a "correct" format.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: A note to all NES-emulator authors..

Post by tepples »

etabeta wrote:the problem is that either we establish some sort of 'central authority' which assigns in a meaningful way the mappers and keeps doing this until the firms in Taiwan and HK quit producing new carts
We've been trying to start a mapper registry. First there's the grid to document what we have, and then there's a "planes" proposal to segregate new dumps from the Far East into NES 2.0 mappers 512-767.
just look at the 5-6 known cases of mapper #s which have been assigned to 2 incompatible boards...
One of them is #34, but that's easy to solve: CHR RAM or 8 KiB CHR ROM = BNROM, and anything bigger = that NINA board. Others are disambiguated with iNES submappers.
byuu wrote:Maybe everyone assumes 8KB PRG is enough for Galaxian, and then a 4KB PRG ROM is discovered later on.
I asked kevtris about this and he said that in these cases, an overdump shall be considered a "good dump" in the iNES or NES 2.0 format. For example, Slappin' (from the 2011 compo) has a 4 KiB CHR ROM repeated twice, and Galaxian has an 8 KiB PRG ROM repeated twice.
zzo38
Posts: 1096
Joined: Mon Feb 07, 2011 12:46 pm

Re: A note to all NES-emulator authors..

Post by zzo38 »

oRBIT2002 wrote:I have a wish for everyone that's coding a NES-emulator...
I know some emulators uses checksums for detecting certain ROMs and to automatically adjust the experience. STOP doing this...
I agree completely. Implement NES 2.0, and optionally implement .NES.INI too (the ROM images I have written relased so far, and that I will do so in the future, all include such a file; if different emulators use their own format, a program can be written for conversion in both directions).
(Free Hero Mesh - FOSS puzzle game engine)
etabeta
Posts: 109
Joined: Wed Nov 29, 2006 10:11 am
Location: Trieste, Italy

Re: A note to all NES-emulator authors..

Post by etabeta »

byuu wrote:But now you have another problem. All the Javascript emulators are going to want to use JSON. MESS is going to want to use its own proprietary XML format. Nestopia has its own XML format. I use BML myself.
Luckily, in most cases, a simple script (possibly even online) could be written to convert from one format to the other ;)

tepples wrote:
just look at the 5-6 known cases of mapper #s which have been assigned to 2 incompatible boards...
One of them is #34, but that's easy to solve: CHR RAM or 8 KiB CHR ROM = BNROM, and anything bigger = that NINA board. Others are disambiguated with iNES submappers.
yeah, but I was using those as an example of what happens when there is no centralized effort to keep consistency as it has happened in the past... and sometimes still happens, with a few chinese dumpers (not all of them, luckily) marking as plain mapper 4 any waixing mapper variant they dump and release, so that it does not work in most emus
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Re: A note to all NES-emulator authors..

Post by Near »

> and optionally implement .NES.INI too

Ooooh, another new format. And one that doesn't support trees. Don't forget to also include a MESS internal database entry, a Nestopia XML file, and a higan manifest.bml as well. Is anyone using JSON or YAML yet? Gotta be somebody out there.

> Luckily, in most cases, a simple script (possibly even online) could be written to convert from one format to the other

Tough to say. MESS' SNES DB does not capture any ROM/RAM address mapping/mirroring. You would not be able to convert your entries to mine. Similarly, I don't capture the writing on top of chips, so I couldn't turn my entries into yours (well, they'd probably work anyway, since that info is not used by emulators.)

But ... you make an interesting point. Just as a hypothetical, what if we had a community effort to form a massive textual database (in some common extensible format, it doesn't matter which), that each emulator author could derive from? Pull out the info they need, and format it into their preferred markup language with their preferred variable names.

The idea would be that absolutely anything and everything would go into it. If anyone finds certain info relevant, we could add it. Peoples' derivations could discard what they consider unimportant. Not only could you generate XML/JSON/YAML/BML, you could also generate iNES 1.0/UNIF/iNES 2.0 headers from the information. Even iNES 2.0 fans should consider it a boon, as it'd be a really fast way to propagate any fixes to bad iNES 2.0 headers.

It would be trivial to include unlicensed games as well, by having a flag to differentiate them from licensed games. Any volunteers that try and run some homebrew which doesn't work, could quickly submit a new board definition, and all emulators participating would benefit.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: A note to all NES-emulator authors..

Post by tepples »

byuu wrote:It would be trivial to include unlicensed games as well, by having a flag to differentiate them from licensed games.
Distinguishing among original licensed games, original unlicensed games, and original homebrew games could be done with a flag called something like CIC Type. It'd take values such as "3193" (for licensed North American games), "Rabbit" (for Tengen games), "CIClone" (for the majority of homebrew), "passthrough" (for Game Genie, HES games, and that Noah's Ark game for Super NES), or some designator for a CIC stun circuit. But then that still doesn't solve how to designate ROM hacks of licensed games, which are usually built on donors.
User avatar
James
Posts: 431
Joined: Sat Jan 22, 2005 8:51 am
Location: Chicago, IL
Contact:

Re: A note to all NES-emulator authors..

Post by James »

byuu wrote:form a massive textual database...you could also generate iNES 1.0/UNIF/iNES 2.0 headers
If there's a centralized database, just have a unique ID for every board/tv format/PPU/whatever combination, then use the reserved bits in the iNES 1.0 header to indicate that there's a database entry. e.g.:
  • - byte 11 - some unique value to indicate that the rom is described in the database. Or maybe use 4 bits for that purpose and 4 bits as a checksum/CRC to reduce the chance of some DiskDude! like corruption?
    - bytes 12-14 - database id. Lower 16 bits would be primary board type (i.e., existing iNES mapper assignment), upper 8 bits would define variant/different configurations.
get nemulator
http://nemulator.com
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: A note to all NES-emulator authors..

Post by lidnariq »

That's basically what NES2.0 is, except that NES2.0 has less goofy indirection.
User avatar
James
Posts: 431
Joined: Sat Jan 22, 2005 8:51 am
Location: Chicago, IL
Contact:

Re: A note to all NES-emulator authors..

Post by James »

lidnariq wrote:That's basically what NES2.0 is
The mapper variant part, yes. But NES 2.0 attempts to further define the ROM in the header, and there will invariably be something that doesn't fit into the format. The idea here is to reduce the board definition to a simple ID, where anything can be defined in the database. Is it the same as using NES 2.0 and ignoring everything except the mapper variant field? Yeah, I suppose it is.
lidnariq wrote:except that NES2.0 has less goofy indirection.
How so?
get nemulator
http://nemulator.com
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: A note to all NES-emulator authors..

Post by lidnariq »

James wrote:there will invariably be something that doesn't fit into the format.
Hence why the remaining bits are explicitly marked as 0 in the header for future use, if we figure out what needs doing. It is possible we'll run out, but we've got a significant amount of headroom with no indication we'll need to use it.
lidnariq wrote:except that NES2.0 has less goofy indirection.
How so?
If the concept that you want to contain is, say, "This a Vs. System game that uses the RP2C04-03 without any additional mapping hardware", in NES2.0 you set the bits that exactly correspond. Providing a SQL unique column index, or whatever you want to think of it, instead says "Please look up UID 26987", which in turn tells you that "it's a RP2C04-03 without any additional hardware". You could sort the UIDs such that they're actually a bitmap, but then that's basically NES2.0.

In any case, there's enough information in Nestopia's database (and thus also in the export from nescartdb) to supply everything but the NES2.0 submapper field for a header-fixing tool. It's just no-one's done it, either due to not wanting to touch it with a ten foot pole or not having it occur to them.
User avatar
James
Posts: 431
Joined: Sat Jan 22, 2005 8:51 am
Location: Chicago, IL
Contact:

Re: A note to all NES-emulator authors..

Post by James »

lidnariq wrote:
James wrote:there will invariably be something that doesn't fit into the format.
Hence why the remaining bits are explicitly marked as 0 in the header for future use, if we figure out what needs doing. It is possible we'll run out, but we've got a significant amount of headroom with no indication we'll need to use it.
So would you dedicate some of that space to defining different VRC4 revisions? How about something like whether or not expansion audio is used and how it's mixed on the cart?
lidnariq wrote:It's just no-one's done it, either due to not wanting to touch it with a ten foot pole or not having it occur to them.
That and because iNES is good enough for the vast majority of cases/users.
Last edited by James on Fri Jun 21, 2013 11:47 am, edited 1 time in total.
get nemulator
http://nemulator.com
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Re: A note to all NES-emulator authors..

Post by Near »

IDs are flawed and unnecessary, and you'll have different people defining their own conflicting IDs, just like with mappers.

Map database entries to SHA256 hashes. Barring a future intentional exploit that allows SHA256 hash collisions (which we don't have yet, and if there ever is one, we can use stronger hashes), there are more possible ROM combinations than there are atoms in the universe.

And now we don't need a header at all, so we can leave it with DISKDUDE or whatever other people insist on keeping there.

> Hence why the remaining bits are explicitly marked as 0 in the header for future use, if we figure out what needs doing.

I want to know what controllers a game supports, and which ones are the preferred input methods, if applicable. Where can I put that into the iNES 2.0 header? I'd also like to note what the default overscan clipping should be for each game, to avoid showing scrolling artifacts on the edge of the screen. Is there space left for that?
User avatar
James
Posts: 431
Joined: Sat Jan 22, 2005 8:51 am
Location: Chicago, IL
Contact:

Re: A note to all NES-emulator authors..

Post by James »

byuu wrote:Map database entries to SHA256 hashes.
What do you do with homebrew and hacks? Fallback to iNES/other header and hope for the best?
get nemulator
http://nemulator.com
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: A note to all NES-emulator authors..

Post by tepples »

@byuu
The NES 2.0 file represents the function of everything inside the Game Pak shell. Things like controllers and overscan go in the manual. (Yes, a few NES era manuals mentioned overscan. I can provide references if you want.) So what you need is a machine-readable version of things commonly found in the manual.

In most cases, if you stick to the overscan amount that the scaled mode of PocketNES implements (8 left and right, 16 top, 11 bottom), you should end up with 240x213 pixels of clean video. Nesticle used a heuristic that if the game sets the clip bit to cut off the left 8 pixels, the emulator would also cut off the right 8.

EDIT: A scan of the Jeopardy! 25th Anniversary Edition manual shows controller information on page 4 and overscan at the bottom of page 18: "This game has been programmed to utilize the full TV screen. Since some older model TV sets have rounded screens, a portion of the image may be blocked out." The transcription of the Super Mario Bros./Duck Hunt manual has similar language about overscan.

@James
Homebrew and hacks can fall back to NES 2.0. If they use something not describable in NES 2.0, they can come with a database entry in the same format as the entries associated with hash values.
Post Reply