NES 2.0 Additions Proposal

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

Post Reply
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

VEG wrote:This field is already supported by the emulators which know about NES 2.0. (...) It is not adding a new field, it is using an existing one.
"Additional field" refers to the added bits to byte 12, not the entire byte or NES 2.0 in general. Also, the Dendy bit as I proposed has already been implemented by Mesen and NintendulatorNRS, and ROMs have been distributed setting bits using that proposal. So for me, that is the "legacy behavior" to consider.
VEG wrote:The thing that an emulator which doesn't support Dendy mode could use the fallback behavior is just a nice addition to the backward compatibility.
Again, accommodating emulators who don't want to add Dendy mode is not a legitimate goal for a header specification.
VEG wrote:If the ROM has specific code to work properly on all known systems (it detects current system and makes some adjustments), it is not a useless information
I still fail to see the value. I don't care if a ROM adjusts itself or not --- I just want to play the game or demo properly, and that's it, and if I do want the emulator to replicate the console I had or have, I'll use the emulator's "region override" option, but then I will accept that the game may run with flaws, or not at all.
VEG wrote:A lot of people played NTSC games on Dendy in their childhood. And sometimes they prefer to play these games with the same timings even nowadays, and they don't care that the games weren't designed for Dendy.
I read that as "I want to play the games like my childhood console, except where it's not possible, but only then I will accept something else". Seems possible, but still highly contrived, too contrived to warrant additional byte 12 bits.
VEG
Posts: 53
Joined: Mon Nov 11, 2013 2:55 pm
Location: Minsk, Belarus

Re: NES 2.0 Additions Proposal

Post by VEG »

NewRisingSun wrote:Also, the Dendy bit as I proposed has already been implemented by Mesen and NintendulatorNRS
The issue that your proposition is poorly compatible with old code. For example, if I use an old version of Mesen with a ROM which uses 00000011 in this byte, it would always fallback to PAL, and it is wrong, because it should be NTSC. With current implementation I always have to specify NTSC in the header (even if I need Dendy), just because I'm quite sure that all emulators understand it, and it is questionable with your proposition. I want to be sure that my ROM will work as expected using any old emulator (people update emulators not so frequently), so I just won't use the new NES 2.0 feature because it may cause issues. For my project, Dendy is the most preferred system, and PAL is the least preferred system. That's why I specify NTSC, because it is in the middle.
NewRisingSun wrote:, and ROMs have been distributed setting bits using that proposal. So for me, that is the "legacy behavior" to consider.
It is unlikely, if you take into account compatibility issues of current approach, and the fact that it wasn't included into the specification. If there are really such ROMs, they had too little of time for spreading around the world.
Last edited by VEG on Sun Nov 04, 2018 5:41 am, edited 2 times in total.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

NewRisingSun wrote:if I use an old version of Mesen with a ROM which uses 00000011 in this byte, it would always fallback to PAL, and it is wrong, because it should be NTSC.
Says who? It is true that it would be falling back to PAL rather than NTSC, but there is no reason to think that one is more correct than the other. That the Dendy is closer to the NTSC NES in terms of timing than the PAL NES as an abstract matter is irrelevant; what matters is whether a game that requires Dendy timing is more likely to run correctly with NTSC or with PAL timing, and I can show you several games that need Dendy timing but also run under PAL but not NTSC. (In fact, I know of no game that requires Dendy timing that does work with NTSC better than PAL.) Your proposal falls back to NTSC, mine to PAL; but that does not make yours correct and mine wrong. So there is no compatibility advantage to your proposal once the incorrect assumption that the correct fallback for Dendy is NTSC is out of the way.

This lack of compatibility advantage, together with its questionable applications, the fact that it is difficult to understand what exactly "preferred" means, that it uses more precious header bits for little relevant information, and (and I agree that this should come last and least) that it goes against what has already been implemented, I continue to oppose this extended byte 12 proposal.
VEG
Posts: 53
Joined: Mon Nov 11, 2013 2:55 pm
Location: Minsk, Belarus

Re: NES 2.0 Additions Proposal

Post by VEG »

NewRisingSun wrote:"Additional field" refers to the added bits to byte 12, not the entire byte or NES 2.0 in general.
BTW, "preferred" system is not a new thing in my proposition. Current specification (without your changes) specifies meaning of two bits, and it has information about preferred system:

Code: Select all

00 - NTSC is supported and preferred.
01 - PAL is supported and preferred.
10 - Both supported, NTSC is preferred.
11 - Both supported, PAL is preferred.
The specification describes it in other words, but effectively it means what I have described above. As an example, Mesen had used the last bit to detect if PAL system is preferred for years, and breaking it is not a good thing to do. Your proposal just force people to avoid these new features of NES 2.0 which may cause unexpected behavior.

NES 2.0 is all about backward compatibility. That's why you should always take into account how an old emulator will behave when it encounters a new file with the new features used.
NewRisingSun wrote:Says who? It is true that it would be falling back to PAL rather than NTSC, but there is no reason to think that one is more correct than the other.
My proposition adds an ability to choose a fallback system. So, an author of a ROM would have an ability to choose.
NewRisingSun wrote:That the Dendy is closer to the NTSC NES in terms of timing than the PAL NES as an abstract matter is irrelevant;
It is very relevant. The probability that a Dendy ROM will work properly on an NTSC system is higher than on a PAL system, just because Dendy was designed to be as much compatible with NTSC games as possible with 50Hz framerate.
NewRisingSun wrote:Your proposal falls back to NTSC, mine to PAL; but that does not make yours correct and mine wrong. So there is no compatibility advantage to your proposal once the incorrect assumption that the correct fallback for Dendy is NTSC is out of the way.
My proposal allows you to specify a fallback system. So, in case of my proposal it will be NTSC or PAL, not just NTSC.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

VEG wrote:The probability that a Dendy ROM will work properly on an NTSC system is higher than on a PAL system, just because Dendy was designed to be as much compatible with NTSC games as possible with 50Hz framerate.
As I said before --- a purely abstract argument; I offer actual games as counter-examples (帝国风暴 - Napoleon's War version 980340, 西天取经 II - Journey to the West, Dark Seed - 黑暗之蛊, 櫻桃小丸子, Education Computer 26-in-1, Arabic Study Cartridge (32-in-1), 2002 World Cup P.K., to name a few).
VEG wrote:Mesen had used the last bit to detect if PAL system is preferred for years, and breaking it is not a good thing to do.
Only if one agrees that the preferred fallback, in the absence of a "fallback system", is NTSC, which I do not.
VEG wrote:My proposition adds an ability to choose a fallback system. So, an author of a ROM would have an ability to choose.
I am not asking for a fallback system at all. An emulator should support Dendy if Dendy is required, and if it cannot, it is going to emulate the game incorrectly anyway, which is another reason why I do not buy the breaking-compatibility argument. So it is a waste of three bits to want to minimize the incorrectness of the emulation. And the tiny number of ROMs that adjust themselves to all three systems (apparently your demo and a few games from Ei-How Yang) do not need a fallback system for compatibility anyway, because they will run nicely, if not optimally, as NTSC or PAL.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

Oh, I missed another thing:
VEG wrote:The specification describes it in other words, but effectively it means what I have described above.
The actual specification says nothing about preferences. It only says "B: When set, indicates this ROM works on both PAL and NTSC machines." It does not state that in the presence of the B bit (bit 1), the P bit (bit 0) would indicate a preference. That may be how earlier versions of Mesen interpreted it, but is not in the specification, and is not universal --- stock Nintendulator, for example, just ignores the P bit in the presence of the B bit, leaving whatever region was set before intact:

Code: Select all

	if ((RI.INES_Version == 2) && !(RI.INES2_TVMode & 0x02))
		SetRegion((RI.INES2_TVMode & 0x01) ? REGION_PAL : REGION_NTSC);
It is with this interpretation in mind that I wrote my proposal.
VEG
Posts: 53
Joined: Mon Nov 11, 2013 2:55 pm
Location: Minsk, Belarus

Re: NES 2.0 Additions Proposal

Post by VEG »

NewRisingSun wrote:And the tiny number of ROMs that adjust themselves to all three systems (apparently your demo and a few games from Ei-How Yang)
A lot of homebrews do similar adjustments. I took the idea of the adjustment of speed from Shiru, who made a lot of NES games, and his neslib (which makes this adjustment by default) is used by other authors of NES games.
NewRisingSun wrote:I am not asking for a fallback system at all. An emulator should support Dendy if Dendy is required, and if it cannot, it is going to emulate the game incorrectly anyway.
Dendy is preferred, but NTSC could and should be used as a fallback. Don't try to diminish this use case. Be constructive.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

Based on that last response, it seems that there is nothing more to add from you (and me). I continue to oppose it --- the fact that your compatibility-breaking argument turns out to be possibly true only for certain versions of certain emulators is an additional reason. And unless there is overwhelming support for it from others, I will not add it to NintendulatorNRS, for all the reasons already given.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

What does need to be pointed out in the specification however is that the VT03+ famiclones have no such thing as "PAL" (2C07) timing: NTSC and PAL versions are the same chips which are just configured for a particular television system through solder pads (one of the pins is hilariously labeled "XPORN"), and by connecting the appropriate quartz oscillator. The "PAL" VT03+ configuration is equivalent to the Dendy-like timing.

All games for these consoles however are clearly developed for 60 Hz, as evidenced by the fact that the television-system-independent handheld consoles on which they also appear choose 60 Hz timing, so the proper region for these games will be NTSC anyway.
VEG
Posts: 53
Joined: Mon Nov 11, 2013 2:55 pm
Location: Minsk, Belarus

Re: NES 2.0 Additions Proposal

Post by VEG »

According to VTxx hardware. Just FYI, there are a lot of famiclones which use something labeled VT02S. I have two of them in my collection:
Image Image
Image Image
They are configured to have Dendy timings by default, but there are jumpers for switching it to NTSC. Actually, these jumpers is not a new thing. Dendy Junior II from 90s also had similar jumpers, and UM6561T (NES-on-a-chip) was able to switch between Dendy and usual NTSC timings.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

We had a discussion on the bootleg games Discord on whether it was possible to change the setting without exchanging the quartz oscillator. I would expect that one could get a usable, though not 100% standards-conforming signal, when using the oscillator it already has, such as the 26.6 MHz XTAL seen in your picture. According to the VT02 application note, there are four different crystals, and only one of them would be correct for the four different television systems supported. (I am not sure about the specifics of the S model of the VT02 though.)
VEG
Posts: 53
Joined: Mon Nov 11, 2013 2:55 pm
Location: Minsk, Belarus

Re: NES 2.0 Additions Proposal

Post by VEG »

Also I'd like to discuss this part:

Code: Select all

        Byte 9: Upper bits of ROM size
        7       0
        ---------
        CCCC PPPP

        C: 4 more CHR ROM size bits
        P: 4 more PRG ROM size bits

        If P/C has the value $F, header bytes $4/$5 changes meaning from number of 16 KiB/8 KiB PRG/CHR banks to the following floating-point-like format:

        7       0
        ---------
        EEEE EEMM
        |||| ||++- Multiplier, actual value is MM*2+1 (1,3,5,7)
        ++++ ++--- Exponent (2^E), 0-63

        The actual PRG- or CHR-ROM size therefore becomes 2^E * (MM*2+1). This allows for ROM sizes larger than the current maximum of 64 MiB without too much padding.
I believe that this change of meaning of $F should be removed completely. There are a lot of reasons:
1. It breaks compatibility with old specification which allowed $F.
2. So, it reduces the maximum effective size which is possible to specify if you want to keep compatibility with the de facto NES 2.0 specification. It reduces it twice.
3. It makes a complicated and tangled format even more complicated and tangled.
4. It is very approximate, you can't specify precise size.
5. None of the old emulators understand it. It is effectively the same as creation of an absolutely new header format. It makes no sense to imitate the structure of the iNES header in this case.
6. Actually, it is even worse. Old emulators will try to load it because it tries to imitate iNES format, and a file which uses the proposed change would look like a usual iNES file for them.

To handle the case when the ROM is so huge that the de facto NES 2.0 can't handle it, just increase the header size twice and put DWORDs for every value in question. To prevent old emulators from using this files with increased headers, and to distinguish files with default header size and with increased header size, just use NES 0x1B instead of NES 0x1A as a signature. It will prevent old software from loading this file with extended header size, because it would not be able to load it anyway.

Conclusion: we have 16 bytes in the default iNES header. It makes sense to add new fields which preserve compatibility with iNES, otherwise it does make no sense to keep ourselves in the window of 16 bytes. So, adding a field which describes which input device is preferred is a nice idea. If I specify this field, newer emulators would take advantage of it, older emulators would just work as before, everyone is happy. But changing format in a way which literally breaks everything makes no sense if you try to keep compatibility with iNES. Just increase the header size, make it intentionally incompatible by changing of the signature, and the problem is solved. Moreover, it will not introduce additional problems, because old software will understand that it don't know how to handle such files right from the beginning of reading a file, because of the signature.
Last edited by VEG on Sun Nov 04, 2018 3:33 pm, edited 3 times in total.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

Since I merely adopted this part based on rainwarrior's proposal, I will let him handle this (if he chooses to). :)
Last edited by NewRisingSun on Sun Nov 04, 2018 4:18 pm, edited 5 times in total.
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 Additions Proposal

Post by rainwarrior »

I proposed it only for the sake of argument, really. Personally I don't even care about ROMs large enough for this field change proposal to be relevant, so please don't consider me a person who should be standing behind that idea, but that said I don't think any of those 6 points were valid.
VEG wrote:1. It breaks compatibility with old specification which allowed $F without reasonable advantages.
2. So, it reduces the maximum effective size which is possible to specify if you want to keep compatibility with de facto NES 2.0 specification. It reduces it twice, without any good reason for it.
3. It makes a complicated and tangled format even more complicated and tangled, without good reasons for it.
4. It can't specify precise size.
5. None of the old emulators understand it. It is effectively the same as creation of an absolutely new header format. It makes no sense to imitate the structure of the iNES header in this case.
6. Actually, it is even worse, because it tries to imitate iNES format, old emulators will try to load it.
1. The old specification's $F was useless, as there are no ROMs of that specified size, and impossible to have a ROM of that size. ROM chips do not come in sizes like that.
2. No the exponent vastly increases the range of possible sizes. Please review what it actually does.
3. The practical reason was that the new size barrier the first iNES 2 draft created has already been broken by at least one homebrew.
4. Yes it can, there's no imprecision here. Again please review what it actually does.
5. No old emulator can run a ROM that's bigger than the format could specify, yes that's true. Why would they? There is no such thing as backward compatibility for that.
6. You can't make an old emulator forward compatible with a ROM that requires new features. That's not a reasonable goal.
Last edited by rainwarrior on Sun Nov 04, 2018 4:18 pm, edited 2 times in total.
VEG
Posts: 53
Joined: Mon Nov 11, 2013 2:55 pm
Location: Minsk, Belarus

Re: NES 2.0 Additions Proposal

Post by VEG »

rainwarrior wrote:2. No the exponent vastly increases the range of possible sizes. Please review what it actually does.
I mentioned "if you want to keep compatibility with de facto NES 2.0 specification". It means that when different emulators (old and new ones) treat $F in different ways, it makes no sense to use such a value, if you care about compatibility of your ROM.
rainwarrior wrote:4. Yes it can, there's no imprecision here. Again please review what it actually does.
OK, accept it. Probably, this idea is a good one for a new header format.
rainwarrior wrote:3. The practical reason was that the new size barrier the first iNES 2 draft created has already been broken by at least one homebrew.
I meant that there is no any good reason to imitate iNES in this case. The result is effectively a new format.
rainwarrior wrote:5. No old emulator can run a ROM that's bigger than the format could specify, yes that's true. Why would they? There is no such thing as backward compatibility for that.
6. You can't make an old emulator forward compatible with a ROM that requires new features. That's not a reasonable goal.
That's why it makes no sense to imitate iNES in this case. If a file which uses a new feature requires a new emulator, we should reliably prevent old emulators from trying to load these files (by changing the signature), and we can use any new format of the header in this case, without trying to look like an iNES file. We are not limited to these 16 bytes if we introduce something which is not compatible with old emulators at all.

I just want to say that if we introduce such changes, we could just create a new header with slightly different signature (like 0x1B instead of 0x1A) and with all the required fields without ugly hacks like a mapper number stored in different nibbles of 3 different bytes. So, this NES 0x1B even could use proposed by rainwarrior scheme of specifying of the size, but without storing 0xF in the byte 9 or somewhere else. What do you think? I don't see any good reason for sticking to the signature of iNES and trying to imitate this format in such cases.
Post Reply