NES 2.0 submappers

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: NES 2.0 submappers

Post by lidnariq »

rainwarrior wrote:So for mapper 3 the question should be the opposite. Are there any mapper 3 games that require no bus conflicts? If yes, we should propose the submapper for this. If not, we don't need it yet.
Yes, there are both mapper 3 things that require the presence and absence. That's why I put CNROM in the list on the page.

And yes, emulators are beginning to move towards "discrete logic mappers enforce bus conflicts by default", even in iNES1 images.
Union Bond [...] the ROM that needs it I don't think we should worry about it.
Looks like Titanic 1912 and/or Darkseed.
023: 15 - the mapper articles claim that this is already compatible with the existing PRG-RAM implementation, i.e. it's not a compatibility issue.
It is incorrect to provide PRG RAM to a game whose header says it does not have PRG RAM.

For the compatibility argument, there are only seven original games that require NES2.0's PRG RAM field. They are:
* Low G Man and/or StarTropics, because the former is the only game that requires the absence of RAM (but it uses the MMC3's PRG RAM disable) and the entire problem here is due to the MMC3/MMC6 PRG RAM protection differences
* The games that used SXROM and SOROM. (FF1+2, Best Play Pro Yakyuu, Genghis Khan, Nobunaga's Ambition, Romance of the Three Kingdoms)

Every other game could be safely emulated with 8 KiB of PRG RAM, except:
* Those that use MMC5, which needs somewhere between 32 and 64 depending on just how klever you're being.
* Those that use Bandai's LZ93D50 with I²C EEPROMs, where they need 128, 256, or 128+256 bytes of EEPROM, but this division is already accurately handled by the three assigned iNES1 mappers.

These seven games are a small enough list that marking this set of games could easily have fit in the submapper field. Since it was NOT put in the submapper field, I can only conclude that the intent of the field was to accurately represent the amount of nonvolatile storage on the cartridge. And, therefore, I have to conclude that a "compatibility" implementation is against the spirit of NES2.0, at the very least for the PRG RAM size field.

Basically, my opinion is that this emulation container should handle both the "simple emulators" use case, as well as people making reproductions, as well as people making ROM hacks. And that means we can't just implement the lowest common denominator, but must enforce extra restrictions to keep people from assuming things that happen to be false.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 submappers

Post by rainwarrior »

lidnariq wrote:
023: 15 - the mapper articles claim that this is already compatible with the existing PRG-RAM implementation, i.e. it's not a compatibility issue.
It is incorrect to provide PRG RAM to a game whose header says it does not have PRG RAM.
Why is it incorrect? That implementation flawlessly supports the ROM. It really doesn't matter whether it came on a VRC2 or VRC4 board, or whether it was a glob top or DIP, whether the PCB was red or green. It's still a completely valid setup to run that ROM. You're asking for a unique mapper to add support for a game that already has support.

I appreciate the desire for emulation tools that can really verify hardware accuracy. I've said before and I'll say it again, I want an emulator like that too. An emulator that I can go into the options and say "run this as if it was board X with jumper Y etc." and have it flawlessly validate whether it would run on that board. That would be a cool tool.

The thing is, only a very small group of people need that. Millions of people play NES emulators, and almost all of them have the primary concern of "will it play my games?" The question of "what would happen if I put this ROM in my Genghis Khan cartridge" is an incredibly niche concern, and requires a few metric shit-tons of meticulous implementation and testing to provide. Almost nobody would even think of trying to implement all of this crap, and if we put all of it in the NES 2.0 spec, it will be very poorly adopted. Quite frankly I think you're destroying the NES 2.0 format by trying to put this burden on it.

I would really love someone to build the meticulous hardware-oriented emulator tool. They don't need an NES 2.0 specification to do it, though. They can get to work on that right now. It really has nothing at all to do with NES 2.0... why do you want this format to do all that?

It's very clear that kevtris' intent was to make a format that was a practical extension to the existing format to get those last handful of problem games running. It wasn't supposed to do any more than that, and it really shouldn't if we expect it to be adopted. Every unnecessary submapper that is added makes more work for the implementers, and retards proliferation of the format. Creating a schism by making a parallel spec with the same name that's incompatible with whatever kevtris is up to just makes the situation worse. It's... ugh.

This is very frustrating. Like, I know what you want, but it's extremely impractical to expect this specification to solve that problem. Specifying the hardware details doesn't make the emulator that implements them appear. If you want that emulator, just build it. Build the shit out of it. I want to see that emulator. Arguing about the NES 2.0 specification doesn't make that magical tool any easier to implement. It just makes the NES 2.0 specification harder to implement, and more obscure than it already is.
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: NES 2.0 submappers

Post by lidnariq »

rainwarrior wrote:Why is it incorrect? That implementation flawlessly supports the ROM.
The exact same argument holds for StarTropics in an emulator that doesn't implement the MMC3/6 protection registers.

The thing is, only a very small group of people need that. Millions of people play NES emulators, and almost all of them have the primary concern of "will it play my games?"
And the people who just want to play their games can safely just use Nesticle. Nesticle is phenomenally inaccurate, but it is wonderfully compatible.

The first 16 mappers gets 80% of the NES and Famicom library. Old crappy iNES files, only mappers 0 through 15, no ability to mark Vs. System or Playchoice. The full iNES1 definition gets us up to 95%. Compatibility is basically a solved problem...

But an emulator cannot be wholly separated from a development tool. People DO test with the emulator that they play their games with. And so the emulator must implement the same restrictions as the hardware... because otherwise these people aren't developing software for the NES/FC, but instead are developing software for Nesticle.

And part of "implementing the same restrictions as the hardware" requires knowing what those restrictions are. And knowing what those restrictions are requires encoding it somewhere. And that somewhere can only be either a hash lookup table, or metadata that is bundled with the image.
Almost nobody would even think of trying to implement all of this crap, and if we put all of it in the NES 2.0 spec, it will be very poorly adopted. Quite frankly I think you're destroying the NES 2.0 format by trying to put this burden on it.
That's a heavy accusation and pretty blatantly unjustifiable. There's plenty of instances of people who have implemented NES2.0, submappers or no, and it's not incrementally a greater load than implementing the 200+ mappers already present.

On the other hand, deciding that the submappers page was horrible and needed to be gutted? Yeah, suddenly changing things like that pretty clearly is a problem.
It's very clear that kevtris' intent was to make a format that was a practical extension to the existing format to get those last handful of problem games running. It wasn't supposed to do any more than that, and it really shouldn't if we expect it to be adopted.
Then spare a sentence to conjecture why you think he added the PRG-RAM size field. Explain to me how my logic is flawed, or what concept I'm missing.

Or if Kevtris wants to answer, I'd happily accept his explanation. Even better if he additionally explains his allocation of submappers 19.1, 34.*, and 78.2.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 submappers

Post by rainwarrior »

lidnariq wrote:The exact same argument holds for StarTropics in an emulator that doesn't implement the MMC3/6 protection registers.
Yes, I think that is a pretty good solution, actually, but it was inconsistently adopted. The submapper is a good solution to the difference of opinion, I think. A practical way to fix the issue.
And the people who just want to play their games can safely just use Nesticle.
Could you treat this discussion with more respect, please?
On the other hand, deciding that the submappers page was horrible and needed to be gutted? Yeah, suddenly changing things like that pretty clearly is a problem.
I didn't "gut" the page. Most of the information is exactly the same as it was before. I moved ones that seemed to be unimplementable, or in some "draft" state to the proposals page. All that info is still there on the proposals page too! Nothing at all has been removed, just transferred to another place so that we can keep "work in progress" stuff bubbling somewhere else until it's stable and ready to implement.

The transfer of VRC2/4 to proposals was made in good faith that it wasn't finished yet (and it looks very unfinished in the state you moved it back to the page).

Most of my work on the page was simply to add description of the allocations that exist, add games lists to explain why those allocations exist, and uniform formatting for organization.
Then spare a sentence to conjecture why you think he added the PRG-RAM size field. Explain to me how my logic is flawed, or what concept I'm missing.
It's not a big mystery why PRG RAM was added to the format. It seemed like a field with practical use so he added it. This isn't some hyper-elegant attempt to minimally specify every existing game with not a wasted bit anywhere. He thought being able to know if a game has no WRAM, 8k WRAM, or other WRAM size would help being able to emulate them correctly.

Its intent wasn't for mapper selection at all, which kevtris stated clearly a little while back, even though it's obviously dependent on mapper. WRAM sizes, especially when it's battery backed, can be helpful for emulators and tools that need to manage save files, for example. I think it's useful additional information, rather than inferring it from mapper or other heuristic info.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: NES 2.0 submappers

Post by tepples »

rainwarrior wrote:It's not a big mystery why PRG RAM was added to the format. It seemed like a field with practical use so he added it. This isn't some hyper-elegant attempt to minimally specify every existing game with not a wasted bit anywhere. He thought being able to know if a game has no WRAM, 8k WRAM, or other WRAM size would help being able to emulate them correctly.
Especially now that FME-7 is known to support large PRG RAM and tpw_rules has been working on a program that makes use of it.
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: NES 2.0 submappers

Post by lidnariq »

rainwarrior wrote:
And the people who just want to play their games can safely just use Nesticle.
Could you treat this discussion with more respect, please?
As far as I can tell, you're arguing that compatibility is one of the foremost objectives here. Compatibility requires imprecision. Precision is necessary for accuracy.

The entire reason for the wiki's Program compatibility article is to demonstrate this: these programs were made by testing against emulators that were more compatible (they implemented a superset of the NES that had the various gotchas elided) and therefore less accurate.

Yes, it's unfair to exaggerate and use Nesticle here, but it's only a straw man in the sense that it's an exaggeration of FCEUX's inaccuracies. And I'm serious when I say that for just playing games, it really was adequately accurate, because SMB1 was a fairly grueling test of raster effects, $2007 delay, sprite 0 hit, and so on. But no-one would say that Nesticle was accurate in any absolute sense—especially not for development.

An emulator that allows writing to CHR/NT during rendering produces homebrew that assumes that it can write to CHR/NT during rendering.
An emulator that doesn't enforce bus conflicts produces homebrew that assumes the absence of bus conflicts.
An emulator that doesn't emulate DPCM glitches produces homebrew that assumes the absence of DPCM glitches.
An emulator that always provides 8 KiB of RAM mapped from $6000-$7FFF produces homebrew and translations that assume it has that RAM.
An emulator that doesn't implement MMC3's protection register AND always provides 8 KiB of RAM produces conspicuously wrong music in Low G Man.
An emulator that conflates VRC4 variants produces translations that randomly pick a different set of mirrors relative to the game.
&c.
[PRG RAM is] a field with practical use so he added it. This isn't some hyper-elegant attempt to minimally specify every existing game with not a wasted bit anywhere. He thought being able to know if a game has no WRAM, 8k WRAM, or other WRAM size would help being able to emulate them correctly.
Is this about emulated them correctly, which can only mean accurately, or is this about emulating them compatibly?

If the objective is compatibility, denoting that piece of information is superfluous. No Famicom game relies on or detects mirrors for the few games released with a 2 KiB PRG RAM, nor does any MMC5 game. The only place this matters at all is for the different ABIs for Generic MMC1 vs SNROM vs SXROM/SOROM/SUROM.

As far as "helping", a lookup table of
Mapper 1, submapper 2: 16384
Mapper 1, submapper 4: 32768
is really not significantly different than
Mapper 1, PRGRAM=0x77 or 0x80 or 0x08: 16384
Mapper 1, PRGRAM=0x90 or 0x09: 32768

On the other hand, if the objective is accuracy, it's necessary to encode the actual set of restrictions that comes from the exact hardware and to precisely enforce them. And stating that VRC2 on PCB 350926 has 1 bit of "RAM", or that VRC4 PCBs 352396 and 351406 has 2 KiB of RAM that can be addressed at $6000 and $6800 but not at $7000 or $7800 is precise.
tepples wrote:Especially now that FME-7 is known to support large PRG RAM
That's an ex-post-facto justification, so not really relevant to my argument...
FrankWDoom
Posts: 260
Joined: Mon Jan 23, 2012 11:27 pm

Re: NES 2.0 submappers

Post by FrankWDoom »

Sounds like you guys are getting a bit off track with emulator accuracy vs compatibility. The iNES header represents the configuration of a particular hardware setup in cart the accompanying program runs on. In this regard I believe it should be as explicit as possible, i.e. as many submappers defined as necessary to eliminate ambiguity and deduction. How well any given emulator honors the stated config is another issue, but ideally iNES should make it absolutely clear what the config is.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 submappers

Post by rainwarrior »

FrankWDoom wrote:Sounds like you guys are getting a bit off track with emulator accuracy vs compatibility. The iNES header represents the configuration of a particular hardware setup in cart the accompanying program runs on. In this regard I believe it should be as explicit as possible, i.e. as many submappers defined as necessary to eliminate ambiguity and deduction. How well any given emulator honors the stated config is another issue, but ideally iNES should make it absolutely clear what the config is.
I might need to clarify what I mean. I'm using this specific definition of compatibility and accuracy:
  • Compatibility means "necessary to run existing games correctly."
  • Accurate means "try to represent the specific original hardware as closely as practical".
Tepples wrote a relevant definition on the wiki, where compatibility means "correct output with known ROMs", and accuracy means "correct output with unknown ROMs".

I think compatibility is a much clearer goal. The test for whether a submapper is needed is easy: for the game in question, will any of the existing ones already produce the correct output? The second part of this is that it demonstrates why someone would want to implement it, i.e. if you do this your emulator will be able to run a specific game. The third part of this is that the game provides a test ROM for correctness. If you can verify that your emulator produces correct output for the game, you know it's correctly implemented.

Accuracy on the other hand I don't think is a practical goal for this format. If you create a submapper for VRC2c, you can't test it with any existing games. Ganbare Goemon Gaiden is going to look the same with or without the IRQ. This bit of accuracy has no practical test yet. You could easily make a mistake when implementing it if you don't have a test ROM; especially if you're implementing 100 little features at once like this and not testing any of them.

And hey, if you want to make the test ROM for a proposed submapper, do it. Do it first, though, don't just put it in the spec and then expect somebody else to do it for you. This is basically my entire suggestion for arbitration. If there's no ROM that can demonstrate the difference yet, don't add it to the spec.

So, the "why should I implement this?" question for accuracy is, "in case homebrew wants to use it in the future, I want to help them with development" which is a nice thought but it's impractical to think you can cover every way that a new programmer can fail simply with a file format specificatoin, or perhaps "in case somebody finds an undiscovered VRC2c game that relies on the non-existence of IRQ". I think this is getting rather obscure and unnecessary. Compatibility as a goal keeps the spec compact and gives a clear guideline for what to implement, and what you're supposed to get for implementing it. Accuracy is more of a lifelong pursuit with no clear boundaries.


I love all the great hardware/accuracy descriptions that the wiki has (and it continually grows); this stuff is wonderful. An emulator with an option to turn on bus conflicts, PPU rendering address conflicts, enforcement of other weird rules that aren't needed for compatibility would be AWESOME. I just don't think this is a good goal for NES 2.0 specifically. iNES certainly wasn't aiming for that, and kevtris' goal was to extend iNES; he said as much a few posts back, it was explicitly compatibility (plus a few arbitrary extra things he wanted).

The stuff currently in the article (with the exceptions of VRC2/VRC4, and SEROM/SHROM) all have test cases, and nothing re-assigns any of kevtris' assignments now that Tepples re-instated MMC6. The stuff that kevtris allocated that doesn't have any test cases yet should be left out of the spec until they get used in the future someday. It's not like we can't add things in the future as they happen. Same deal with all that proposal stuff. If there's no test ROM, leave it out. For stuff that's not in kevtris' spec, I see no problem with us allocating new things that he didn't think of, just so long as we don't reassign his original ones. There's plenty of room here. Same deal with the WRAM fields; leave things in a way that's compatible with kevtris' version; it's really not hard to accomodate this. There's no need for a format schism.


TLDR version, this is really what I'm getting at:

1. A test case, like a game that won't run without it, or even a test ROM should be a minimum standard requirement before allocating a submapper.
2. Don't redesign kevtris' format or reassign older allocations.
FrankWDoom
Posts: 260
Joined: Mon Jan 23, 2012 11:27 pm

Re: NES 2.0 submappers

Post by FrankWDoom »

I get what you're saying now. I supposed I'd mostly support compatibility.

On VRC2c, I think I disagree with you. We have a hard example, and that should be accounted for somewhere. If an emulator chooses to support that by using the generic VRC core rather than VRC2c strict, that's the author's decision. If I were personally writing an emulator my goal would be strictness, and I'd want to know the actual cart config is VRC2c. It seems like a bad idea to omit that info when it could be clearly provided.

Otherwise I would agree on declining new assignments without a test rom or physical cart.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 submappers

Post by rainwarrior »

What's the example for VRC2c? Ganbare Goemon Gaiden is reported to be compatible with the VRC4 default. Is there another game that breaks if you don't emulate it as strict VRC2c?
Last edited by rainwarrior on Mon Aug 10, 2015 11:38 am, edited 2 times in total.
FrankWDoom
Posts: 260
Joined: Mon Jan 23, 2012 11:27 pm

Re: NES 2.0 submappers

Post by FrankWDoom »

There isn't one, but since it's a known quantity I think it should be identified as such.

After reviewing your an my posts, I'll have to correct myself and say I lean towards accuracy as preferred, but maintaining flexibility. We don't have to limit ourselves to only known hardware cases, but anything known should be at least specified as such. And again if an emulator author chooses to support that with a generic implementation so long as there's no case to force the strict implementation, I can't argue with that.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 submappers

Post by rainwarrior »

Also I think it's bad that we've overspecified the submappers for VRC2/VRC4. 45 assignments have been made for only 8 existing variations, and the VRC2-only specification only has one possible assignment (in 2 cases already used up). I think it would be a lot better to just assign submappers 0,1,2,3 to the existing cases instead of devising a scheme to accomodate fictitious ones.

Since VRC2/VRC4 wasn't in kevtris' original set and there are no current test cases, I would presume this is still open for debate, but if we want to stick with it I wouldn't fight it. Just saying I think it's a bad idea.
FrankWDoom
Posts: 260
Joined: Mon Jan 23, 2012 11:27 pm

Re: NES 2.0 submappers

Post by FrankWDoom »

Are you referring to deriving the submapper from the address line numbers? If so I'd agree that's overly complicated, and handing out submapper assignments in order for known cases would be preferable. Maybe leave a few blank spots between vrc4/2 groups, e.g. vrc4 starts at 1 and vrc2 start at 10 or so, as a sort of logical grouping. That's mostly for human readers.
FrankWDoom
Posts: 260
Joined: Mon Jan 23, 2012 11:27 pm

Re: NES 2.0 submappers

Post by FrankWDoom »

This is an aside but it looks like VRC4 and VRC2 are labeling the address lines in opposing fashion. In particular VRC4 seems to have the line numbers flipped if we're looking at the VRC4 page for reference.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 submappers

Post by rainwarrior »

FrankWDoom wrote:There isn't one, but since it's a known quantity I think it should be identified as such. ... anything known should be at least specified as such.
Yes, I've been trying to make it very explicit what each submapper is required for. I think I've done this for each of the submappers currently in there and I'd like to see this standard applied in every one that gets added.

What I'm really after is that the spec on the wiki should be in an implementable state, and part of that means already proven and testable. Every allocation is a request that many emulators should implement it. I don't think it's too much to ask that a person make and test ONE working implementation of it before they request many implementers to go at it. (It's also important for consistency and correctness.)

Like, I don't really want to quibble about ideology for whether this or that variation needs a specification. That's thoroughly exhausting (and seems rather fruitless so far). If there's a ROM for it, go ahead I guess, but just run it by people first so we make sure it's not already covered, or improperly described, etc. and keep continuity with earlier versions of the spec.
Post Reply