NES 2.0 Additions Proposal

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

No. "Multicart" signals that different expansion port devices are required depending on which game was selected. Ideally, the expansion port device automatically changes (endogenously) as the PRG-ROM mapped into CPU address space changes through (exogenous) user action, though how to account for that is up to the emulator.

Since the expansion-port-using games on common multicarts usually are the Zapper games, the spec recommends emulating an expansion port Zapper plus two standard controllers as the simplest way to account for that, though more involved approaches are possible, such as checking for the port-reading signatures of the handful of common multicart games, which is what I do in NintendulatorNRS, and turns out to be very ROM-hack-robust.

Allpads, as I understand it, is the opposite --- the PRG-ROM mapped into CPU address space is always the same, but takes different code paths (endogenously) as a result of (exogenous) changes in the connected expansion port device.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: NES 2.0 Additions Proposal

Post by tepples »

NewRisingSun wrote:No. "Multicart" signals that different expansion port devices are required depending on which game was selected.
From a certain point of view, if you pretend each test screen is a "game", and a game is "selected" by plugging in the correct controller and pressing its primary button... But I admit it's a stretch and a bit backward in a way from the usual use of the term.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 Additions Proposal

Post by rainwarrior »

NewRisingSun wrote:On the other hand, defining Standard Controllers as anything other than $00, and defining $00 as "no information" would mean that the majority of released NES 2.0 ROMs would become underspecified with one stroke.
iNES 2.0 has been around for quite a few years now and there are quite a lot of ROMs out there using it. I don't even know where to begin enumerating these. The spec for controllers is only 9 days old. I think you're vastly underestimating what might be invalidated by changing the meaning of $00 (which was until that point specified as reserved). You're making the assumption that everything people have done with this header for years needs to specify standard controllers. Allpads is not at all the only example, just the very clearest one to demonstrate the need.

They're not "underspecified", they are specified with exactly as much information as the author gave them, and it is simply wrong to assume that anyone meant specifically standard controllers before it was part of the spec.

I know you have some idea that you want to be able to say there is one canonical header for every single ROM out there, but there's a difference between being able to suggest one "best" header and insisting that every possible piece of data always has to be specified. This is an optional field, and it has to be because of its historical use. All of the later additions like this have to be. iNES 2.0 has legacy already.

I strongly oppose insisting that old ROMs get new meaning imposed on them by the change to this field. Reserved $00 meant reserved, not "you have to come and change this later when someone feels like changing the format".
NewRisingSun wrote:The same goes for the TV System byte In fact, I cannot think of any other NES 2.0 field where value $00 really means "no information", and in my stated interest of avoiding ambiguity, I don't really want to introduce one. (That's why I now take back the word "unspecified" that is not on the wiki, but was part of the draft earlier in this thread.)
Byte 15 was defined as "reserved as $00" since the beginning of iNES 2.0 as a proposal and until 9 days ago. Every single other byte of iNES 2.0 had a defined value for $00 since its inception, and/or subsequent additions to it haven't changed the meaning. TV system is not comparable, as it was in iNES 2.0 since its beginning in 2006.
NewRisingSun wrote:"Standard controllers" cannot be $01 unless everything else moves down one number. Some recent ROM images already use that field, and I do not want to invalidate or replace them.
I don't really care what number you want to use for it, as long as it isn't $00. I said $01 just to make an example (though I think it'll be a little goofy it ends up as some random value because of this). How many recent ROM images have started using it in the last 9 days? Did you go and create a whole catalogue immediately after revising the spec? If you want to reorganize it or not I don't really care, all I really care about is the meaning of $00 here.

The suggestion that any other value could specify standard controllers is a concession for your goal to be able to recommend a fully specified header for everything under the sun. I've said before, I don't mind that goal, and I'm sympathetic to it, but I think it's unreasonable to insist that anything less than this is an invalid header. You can absolutely say that "this header that specifies $01 in byte 15 for game X that is known to require controller Y is better than specifying $00 in that byte", but you cannot retroactively invalidate or change the meaning of $00 for it. Headers that do not specify controllers are valid. The meaning has to stay "unspecified".

I do actually love that you added this field to the spec. I thought for a long time this was needed, and it's a great option to have, but $00 needs to be backward compatible with the prior definition.
Last edited by rainwarrior on Fri Jan 11, 2019 12:38 pm, edited 1 time in total.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 Additions Proposal

Post by rainwarrior »

Sour wrote:
rainwarrior wrote:After seeing how Mesen has implemented, I have a request for Byte 15:

Add a value for "do not override user settings" or something of that nature. This is needed for stuff like tepples' "allpads" or other situations where it's deliberately wanted for the user to plug in what they like.
FYI, you can disable byte 15's behavior by disabling the "Automatically configure controllers when loading a game" option in the input options. This disables using both the game database and byte 15 to select controllers.
Yes I did find the off switch for it, but I'd much rather be able to have this feature on without it disrupting ROMs that have been a casualty of this meaning change. I don't think that's possible with the definition NewRisingSun made for $00.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

rainwarrior wrote:I think you're vastly underestimating what might be invalidated by changing the meaning of $00
Then tell me. Which games were released in NES 2.0 format that do require something other than standard controllers? I am willing to be persuaded by the number of games that will be invalidated, not by the vague theoretical possibility ("might").

Also, I question the use of "invalidated" --- before the field was defined, emulators would not choose the required input device for them (some even reverted to standard controllers for every newly-loaded ROM), so they would have to do it themselves. Now, it chooses the wrong one for them (standard controllers) so they have to do it themselves, after loading the ROM file, that is. The difference is that with $00 defined as standard controller, these ROMs would suffer the indignity of being "wrong", whereas if $00 is defined as "unspecified", they would be a more dignified "suboptimal"; but in both cases, the canonical header would be a different one. So I am not seeing any actual damage being done by giving value $00 a defined meaning, and do see actual damage by defining a format such that there are "optimal" and "suboptimal" headers, as opposed to simply correct and incorrect ones.
rainwarrior wrote:They're not "underspecified", they are specified with exactly as much information as the author gave them
They were not underspecified according to the previous definition, but would be underspecified (or "suboptimal" if you prefer that term) according to the updated one. your proposed one.
Last edited by NewRisingSun on Fri Jan 11, 2019 1:00 pm, edited 2 times in total.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 Additions Proposal

Post by rainwarrior »

NewRisingSun wrote:
rainwarrior wrote:I think you're vastly underestimating what might be invalidated by changing the meaning of $00
Then tell me. Which games were released in NES 2.0 format that do require something other than standard controllers? I am willing to be persuaded by the number of games that will be invalidated, not by the vague theoretical possibility ("might").
Well you must understand that this is an extremely difficult question to answer. The iNES 2 spec has been floating around for more than a decade, though maybe it's only got traction in the last 5 years. Offhand I can think of a few homebrew to check but I think it would literally take me months to research this adequately.

What ROMs have been created in the last 9 days that have $00 in there that you think are going to be invalidated, on the other hand should be extremely easy to answer, relatively speaking, shouldn't it?

What I'm asking for is to proceed with caution and not assume you can change meanings that have existed for a long time already. Someone who was using the iNES 2.0 spec from a month ago should be able to remain confident that they haven't had their work broken by a future change. Byte 15 was "reserved" and that does in fact mean exactly that they were not specifying anything in this byte.
NewRisingSun wrote:Also, I question the use of "invalidated" --- before the field was defined, emulators would not choose the required input device for them, so they would do it themselves. Now, it chooses the wrong one for them (standard controllers) so they have to do it themselves. The difference is that with $00 defined as standard controller, these old ROMs would just be "wrong", whereas if $00 is defined as "unspecified", they would be "suboptimal"; in both cases, the canonical header would be a different one. So I am not seeing any actual damage being done by giving value $00 a defined meaning, and do see actual damage by defining a format such that there are "optimal" and "suboptimal" headers, as opposed to simply correct and incorrect ones.
I am seeing damage done by it, which is why it came up. I think Mesen's interpretation and implementation of it is exactly what the current spec you wrote is calling for: $00 is telling the emulator that standard controllers has been specified, and it should override user settings, exactly like all the other values of this field should do with their respective control sets.

The value of $00 must not insist that the author was specifying for standard controllers, "no specification" is the only possible meaning that has backward compatibility with older versions of the spec.
Last edited by rainwarrior on Fri Jan 11, 2019 1:00 pm, edited 1 time in total.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

missed one question:
rainwarrior wrote:How many recent ROM images have started using it in the last 9 days? Did you go and create a whole catalogue immediately after revising the spec?
There have been ROM images released using it since Mesen added it based on the proposal. That was about six months ago or so.
rainwarrior wrote:$00 is telling the emulator that standard controllers has been specified, and it should override user settings,
Only the previous settings. I explicitly called in the spec for accepting user-defined settings after the ROM has been loaded. So again, I question the use of terms like "having their work broken". I may need to re-check what Mesen actually does, but if it refuses to allow the user to choose something other than what the header calls for, then I would agree with you that this is not what is supposed to happen.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

rainwarrior wrote:What I'm asking for is to proceed with caution and not assume you can change meanings that have existed for a long time already.
And what I am asking for is weighing the actual consequences, not the theoretical consequences, of either approach.
rainwarrior wrote:How many recent ROM images have started using it in the last 9 days? Did you go and create a whole catalogue immediately after revising the spec?
The current release of Project Plug'n Play, which mostly consisted of reheadered ROMs that were released previously, has 569 ROMs .7z files, each of which containing one or several ROM files, and is dated January 2nd. There are also others going back six months, as mentioned, but you might say they don't count having been released before the spec was finalized.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 Additions Proposal

Post by rainwarrior »

NewRisingSun wrote:missed one question:
rainwarrior wrote:How many recent ROM images have started using it in the last 9 days? Did you go and create a whole catalogue immediately after revising the spec?
There have been ROM images released using it since Mesen added it based on the proposal. That was about six months ago or so.
So what are these ROMs? Were they published somewhere? Are they well known an accessible? Updateable? Changeable? Is this something you could automatically just replace $00 with $42 or whatever value you want to use to say "standard controllers specified"? (Presumably all other values are fine to keep, I'm only talking about $00.)

What you're try to weigh this against is a huge unknown, a spec we've been living with for many years... I could only begin to list stuff, but let's start with allpads and thwaite.

And I think we're weighing the new ROMs you're referring to being "underspecified" against some unknown number of homebrew from the past several years being specified wrongly. The underspecified thing I think is a lot less of a problem than assuming a ROM specifies something it doesn't. Giving the user the wrong set of controls is exactly what this byte should be design to avoid.
NewRisingSun wrote:
rainwarrior wrote:$00 is telling the emulator that standard controllers has been specified, and it should override user settings,
Only the previous settings. I explicitly called in the spec for accepting user-defined settings after the ROM has been loaded.
That's not what it means. Yes of course an emulator should have an option to override any header information that is undesired, TV system, controls, etc. that really doesn't even need to be mentioned in the header spec, IMO.

The problem is if the ROM says "X is specified", the automated selection is allowed to apply. It should never apply if something is unspecified.

...and that's exactly what Mesen is doing. It's applying it because it thinks $00 is specified... which it isn't. A user should be able to have this option on without it breaking ROMs that were made before the change.


It's also not just about ROMs that exist, this can affect any homebrewer that is working from an older version of the spec. People should be realistically be able to make valid iNES 2 ROMs with what the wiki said 1 months ago. I have published a bunch of example code or open source stuff that says "reserved" in the comments for this byte, etc. we can't insist that everyone is suddenly up to date with the current spec always.

I'm sure tepples' has some open source code lying around on websites, for example. If someone uses thwaite's source code as an example to build on, there are 4 different versions of that source out there, available in different places, many of which tepples could not reasonably update, all of them currently "wrongly" specifying standard controllers by this change, it would seem?

I don't know who published ROMs using a new spec 6 months ahead of when you made it public on the wiki, but when moving forward the lag in information propagation will be huge.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

rainwarrior wrote:So what are these ROMs?
Answered above.
rainwarrior wrote:What you're try to weigh this against is a huge unknown. (..) I could only begin to list stuff, but let's start with allpads and thwaite.
Well, list some more, because so far, the huge unknown consists of two ROMs, one of which is not a game.

What really irks me right now is that the proposal and Mesen's interpretation of it have been out there for half a year. Some people have contributed to it (including you, though not to that field), not without feeling the need to express how much they actually don't care but are just sayin' and stuff, and now that it has been finalized the way it was proposed, now that a sizable number of ROMs (see my previous post) have been (re-)released, now somebody suddenly has concerns that the most common field value is unacceptable because it breaks two ROMs and a "huge unknown", except that it does not actually break them but just requires the same user intervention it previously did. What do you expect me to say? :P
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: NES 2.0 Additions Proposal

Post by lidnariq »

Every single ROM I have ever released has had NES2.0 headers and has left that byte as its "reserved" value. Automatically switching to "standard controllers" is wrong for at least 3/4 of them. Are most of them not games? Yes. Does that matter? No.

Demanding that you be enumerated every single thing that it breaks is unfair to everyone else. How can we track it down? How can anyone? That's the entire point of leaving the standard as upwards compatible.

On the other hand, you know exactly what ROMs you've released with a 0 in there meaning "override the emulator's settings with standard controllers".
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

lidnariq wrote:Demanding that you be enumerated every single thing that it breaks is unfair to everyone else.
I did not demand that. I asked for something more specific than a "huge unknown". Mostly, I "demanded" that you should have said something in the past six months when the proposal was out there, instead of going out of your way to declare how much you don't care.

And I already described how the "breaking" aspect is pretty much of a nothingburger.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 Additions Proposal

Post by rainwarrior »

NewRisingSun wrote:
rainwarrior wrote:So what are these ROMs?
Answered above.
That wasn't just a single question:
rainwarrior wrote:Were they published somewhere? Are they well known an accessible? Updateable? Changeable? Is this something you could automatically just replace $00 with $42 or whatever value you want to use to say "standard controllers specified"? (Presumably all other values are fine to keep, I'm only talking about $00.)
Are the ROMs you're referring to are a romset that is updated periodically?

The point of this field is so that when you start up a ROM, the emulator will be able to provide you with the right control set automatically.

This change you made means anything made before the spec that wanted something other than standard controls will get the wrong control set.

However, for any of these ROMs that were released based on the new spec, all it means is the ones that were specifying standard controls would now fall back to not specifying, which is a very mild consequence IMO and not even "wrong" controls but just whatever the underlying emulator default is (probably user settings).
NewRisingSun wrote:What really irks me right now is that the proposal and Mesen's interpretation of it has been out there for half a year.
Well, I'm sorry I didn't catch it sooner. I noticed the problem just now, 9 days after you published the spec. I reported it as soon as I noticed it. I'd actually been having the problem with Mesen for a few months, I think, but I didn't realize that this was a consequence of that thing until today. I had no idea Mesen had implemented it proactively. Mesen is not the only emulator I use, either, so it took some luck that I just happened to be trying to write a game with mouse controls lately... which I started writing before this spec was published... and am now realizing that this problem with Mesen resetting its controls is more or less directly mandated by the spec you published.
NewRisingSun wrote:
rainwarrior wrote:What you're try to weigh this against is a huge unknown. (..) I could only begin to list stuff, but let's start with allpads and thwaite.
Well, list some more, because so far, the huge unknown consists of two ROMs, one of which is not a game.
So... I can't speak for all the people out there working from older versions of the spec. There are a lot of people making homebrew NES games right now. Many of them will be in the dark about this change for a good long time. I would look at a bunch of the games on Action 53, there are at least 10 mouse and zapper games in there, some of which will have used iNES 2, some probably not. Many of them are open source releases, and have been public for many years, disseminating widely. That source code itself is more collateral, because it will teach new devs what to do as well, and that's all based on the older spec.

These are in a thousand different places in many forms. A lot of them have several versions of the ROMs in forum threads, versions of the source on various github pages, websites, etc. it's actually more or less impossible to update these retroactively.


...and ALL I'm asking for is for you to change the meaning for just $00 back to what it was, for backward compatibility with all previous versions of the iNES 2 spec.

Yes, it downgrades that ROM release you were talking about, I guess, but it's a very, very small downgrade to fix a compatibility problem this redefinition of $00 created.
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: NES 2.0 Additions Proposal

Post by NewRisingSun »

All right. That field has given me more trouble than anything else in there, and the prevailing opinion had been that it should not be there anyway, because description of what's inside the cartridge shells and such, so I am now disavowing it and removing it. Should anybody care to want it back, he can always choose an older version from the wiki and change it in a way that is satisfactory to him and the huge unknown. I shall have no opinion on it.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: NES 2.0 Additions Proposal

Post by rainwarrior »

I like the field. I think it's a very good addition. I only dislike the meaning you chose for $00.
NewRisingSun wrote:the "breaking" aspect is pretty much of a nothingburger.
I don't want to make this an argument of sides where we're trying to argue one pile of ROMs to update is bigger than the other. That's not what I'm trying to do here, I genuinely feel that $00 specifying standard controllers hurts the spec.

In my view even if this were just about thwaite, the trade looks like this to me:

A. A bunch of well known games need to specify that they use the standard controllers 6 months ago.
B. Anyone trying to play an older ROM of thwaite will have to futz with their input settings every time the ROM is loaded.

That is not a good trade. Being able to specify standard controllers is nice metadata, but solves almost no problems for the user, it's what the emulator would be doing by default in every case anyway. Really the important use cases for this field were things that needed non-standard controls.

On the other side, it creates a setup problem for the user who wants to play thwaite... and it's really hard for an emulator author to get around this with UI, it's inherently asked for by specifying the controllers, the only option left is to abandon (i.e. turn off) the feature, which defeats the purpose of having it in the first place.


So... that's what I'm getting at. My goal isn't to make you do more work on some ROMs, it's to have a spec that doesn't create new problems for the users. The byte 15 proposal is very good to have, and I really did like it going in. The only problem I have is the meaning of $00, the rest of it is fine. You can still have the metadata to specify standard controllers too! ...but it shouldn't be $00.
Post Reply