SOROM and older emulators

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: SOROM and older emulators

Post by rainwarrior »

As far as I can tell, the MMC1 implementation just needs a special case for the 16k boards. I think there are only 3 extant cases for battery backed WRAM on MMC1?

1. 8k

2. 16k (SOROM): save only the second half.

3. 32k (SXROM): save all 32k

Since WRAM size is absent from iNES 1, you need to get the data in either through the database, or through iNES 2's WRAM field.

It sounds like if there's a database entry specifying bigger WRAM, it saves the whole specified WRAM? In which case adding 16k to the entry should make saves work, but Koitsu doesn't like it that it's storing stuff that's not actually saved by the battery. The 32k database entry makes no sense at all; I mean it would work, but it's pointless.


The iNES 2 header is already being parsed, I think, (see NstCartridgeInes.cpp line 737) so it's really just a matter of having the save routine pay attention to it. (Having an iNES 2 solution also means romhacks and homebrew can work, which is a big plus, IMO.)

I don't think the database has the a separate field for battery backed like iNES 2, but knowing that WRAM is 16k is enough to disambiguate the special case for MMC1. I don't think there's such thing as another MMC1 16k board that wants to save the other half, or the whole 16k, and at this point the discovery of such a thing should deserve a new mapper assignment.

i.e. just add a Save function to Mmc1, that calls Board::Save unless it'a a 16k case. IMO you should also update the database to say 16k though, handle both the database and iNES 2.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: SOROM and older emulators

Post by koitsu »

This is going to take me some time to work out, mentally. I've read your post 4 times now and there is just a lot to process/juggle, compounded by lack of code familiarity on my part. This is even more compounded by the fact that these Koei games, even when shoved through present-day GoodNES, do not have NES 2.0 headers -- they have original NES.

Okay, actually now I've gone over it 6 times, and have re-written my post 3 times over now. :/

Thus, unless I'm missing something here (which is extremely likely!), I'm not sure how to cleanly implement this. The problem is not just with saving, it's also with loading. For saving, this is the only thing I can think of (again: certain I'm missing something) that would work given the situation in my first paragraph:

* Update NstDatabase.xml to force PRG-RAM of 16KB for these individual ROMs
* Modify NstBoardMmc1.cpp to detect if game PRG-RAM is 16KB. If it is, figure out magic to only write the 2nd 8KB PRG-RAM page to disk (I say "figure out magic" because I haven't looked at this code or how any of these functions/methods are implemented in detail); if it isn't, just call Board::Save (which will allow 8KB PRG-RAM and 32KB SXROM games to work).

What worries me also about the 2nd part is the "keying off of 16KB PRG-RAM" (setup.prgRam). I don't know if this is truly safe or not for all games.

For loading, the situation becomes unknown/precarious because I don't know what's going to happen if NstDatabase.xml has a PRG-RAM force size of 16KB for a game; it may attempt something like reading 16KB from the .sav, which could/might fail with I/O errors due to reading past EOF, or it might work but stick the 8KB in the first page rather than the 2nd.

I believe NstDatabase.xml trumps whatever is in the NES ROM header, hence the logic/code in NstCartridgeInes.cpp would get overridden, which would matter for loading.

I'm left thinking maybe review of the NstDatabase.xml capabilities is in order, possibly needing extended capabilities in that, e.g. for these particular Koei games, load/save 8KB PRG-RAM from the 2nd page. I make it sound so easy, yet I know damn well it isn't going to be. This makes me wonder: how does FCEUX deal with this?

I think you get where I'm going with this... I hope. Sorry if I'm not making sense in advance.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: SOROM and older emulators

Post by koitsu »

I also just came across this -- specifically item 4: http://wiki.nesdev.com/w/index.php/MMC1 ... mbiguation -- which seems to imply 16KB .sav files (with redundant data in them) are perfectly OK, even though it talks about "loading from an 8KB .sav" (I assume this refers to "old .sav files before this type of heuristic is put into place"?):
If the battery bit is present, only banks which are written to are saved to the disk when the game is quit. When loading a game with the battery bit set, if a 8KB .sav file is present, it is repeated equally across all banks. This will lead to data being saved when it wasn't supposed to for SOROM games, but 8KB of hard disk space isn't a problem, ...
At this point I'm utterly baffled.
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: SOROM and older emulators

Post by rainwarrior »

koitsu wrote:This is even more compounded by the fact that these Koei games, even when shoved through present-day GoodNES, do not have NES 2.0 headers -- they have original NES.
Almost nothing has iNES 2 headers. That's why the database exists (and why it's worth maintaining).
koitsu wrote:The problem is not just with saving, it's also with loading.
Yes, also override Load in a more or less identical way.
koitsu wrote:* Update NstDatabase.xml to force PRG-RAM of 16KB for these individual ROMs
Yes, do this. This is actually more important than supporting iNES 2, because it will affect more people.
koitsu wrote:I believe NstDatabase.xml trumps whatever is in the NES ROM header
An iNES 2 should always trump the database. If it doesn't, you haven't implemented iNES 2 properly. (The database, in turn, trumps iNES 1, which was incapable of describing 16k WRAM.) I can't say whether Nestopia UE implements iNES 2 properly, though. Follow the fields read from the header and see where they go. Possibly nowhere at the moment?
koitsu wrote:What worries me also about the 2nd part is the "keying off of 16KB PRG-RAM" (setup.prgRam). I don't know if this is truly safe or not for all games.
An Mmc1::Save only applies to MMC1 boards, and there's only one type of board with 16k WRAM. (SOROM) It should have no consequences outside of MMC1, and within MMC1 it should be safe.

As for iNES 2, just make sure the values read from the WRAM field get summed and put into setup.prgRam, or wherever the database happens to stick it. Mmc1:Save doesn't need to know whether the info came from the database or iNES 2, it just needs to know the WRAM size.
koitsu wrote:For loading, the situation becomes unknown/precarious because I don't know what's going to happen if NstDatabase.xml has a PRG-RAM force size of 16KB for a game; it may attempt something like reading 16KB from the .sav, which could/might fail with I/O errors due to reading past EOF, or it might work but stick the 8KB in the first page rather than the 2nd.
Board::Save is two lines long. It just retrieves a pointer to WRAM, and its size, and saves it to a file (if battery backing is detected). Board::Load is similar. I don't think there's much risk of mysterious problems here. They should both just do the same test.
koitsu wrote:I'm left thinking maybe review of the NstDatabase.xml capabilities is in order, possibly needing extended capabilities in that, e.g. for these particular Koei games, load/save 8KB PRG-RAM from the 2nd page. I make it sound so easy, yet I know damn well it isn't going to be.
Why would you need to extend the database capabilities? It can already specify 16k WRAM. You don't need anything else here.
koitsu wrote:This makes me wonder: how does FCEUX deal with this?
It has an internal database and automatically assigns the WRAM size (look for DetectMMC1WRAMSize and GenMMC1Init in boards/mmc1.cpp).
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: SOROM and older emulators

Post by rainwarrior »

koitsu wrote:I also just came across this -- specifically item 4: http://wiki.nesdev.com/w/index.php/MMC1 ... mbiguation -- which seems to imply 16KB .sav files (with redundant data in them) are perfectly OK, even though it talks about "loading from an 8KB .sav" (I assume this refers to "old .sav files before this type of heuristic is put into place"?):
If the battery bit is present, only banks which are written to are saved to the disk when the game is quit. When loading a game with the battery bit set, if a 8KB .sav file is present, it is repeated equally across all banks. This will lead to data being saved when it wasn't supposed to for SOROM games, but 8KB of hard disk space isn't a problem, ...
At this point I'm utterly baffled.
Saving and restoring the RAM that's not battery backed is harmless. It's not "accurate", but it's harmless. If you remember the other thread about initializing RAM, it's the same deal, there's no "accurate" WRAM initialization either, so it doesn't matter if you fill it with saved data, or 0s, or 1s, or random values.

The only problem with saving 16k is that you're saving 8k of usless information.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: SOROM and older emulators

Post by koitsu »

The rabbit hole goes deeper, I'm afraid.

NestopiaUE already thinks/knows the WRAM is 16KB. My second post actually shows this. That made me wonder how this all worked. I made the mistake of looking at the code first -- I should have looked at NstDatabase.xml first -- so I'll post what I "dug up" about that.

Looking through board/NstBoardMmc1.cpp showed me there's some routine called GetWram(), which is just a convenience routine for GetSavableWram() + GetNonSavableWram(). Those method names made me realise the emulator seems to have an internal delineation between what's battery-backed and what isn't.

There's a whole ton of crazy in boards/NstBoard.cpp that I can't quite grasp. The GetSavableWram() and GetNonSavableWram() routines are probably the most generic things I've seen in a while, with no real indication of what any of the magic numbers mean. id seems to be... board ID? I'm not even sure. It almost looks like C++ templating is used here.

I decided to try another test: load up Genghis Khan, play the game for a season or so, choose Other -> Save, then Soft Reset, then tried to Load Data -- it worked. A Hard Reset (akin to a power-cycle) did not. This obviously isn't testing .sav loading/saving (well, Hard Reset might be, I'm unsure), but it does seem to mean that emulation of SOROM with 16KB SRAM/WRAM/PRG-RAM is working in some regard.

I was therefore left thinking that modifying NstDatabase.xml for these games, using <wram size="16k" /> wouldn't fix anything, because it seems to already know that... so what gives? That lead me to look in the actual database itself. Lo and behold:

Code: Select all

    <game>
        <cartridge system="NES-NTSC" dump="ok" crc="2225C20F" sha1="209911D7BD15ABB7BEF2E35A473DF725B6738CD7">
            <board type="NES-SOROM" mapper="1">
                <prg size="256k" />
                <vram size="8k" />
                <wram size="8k" />
                <wram size="8k" battery="1" />
                <chip type="MMC1B2" />
            </board>
        </cartridge>
    </game>
Well that's quite interesting: here we can see the segmentation of the 16KB PRG-RAM into the battery-backed and non-battery-backed regions. To verify that this is in fact what is causing the emulator to get the correct sizes/etc., I disabled use of the internal database in Nestopia (Options -> Database -> uncheck Internal) and loaded the ROM.

Results are here -- note the Board and WRAM size being wrong:

Code: Select all

File:         Genghis Khan (U) [!].nes
Directory:    D:\Console\NES Games\USA\
Soft-patched: No
CRC:          2225C20F
SHA-1:        209911D7BD15ABB7BEF2E35A473DF725B6738CD7
System:       NES-NTSC
Board:        SNROM, Mapper 1
PRG-ROM:      256k
V-RAM:        8k
W-RAM:        8k
Solder Pad:   H:0 V:1
Battery:      Yes
File:         Genghis Khan (U) [!].sav
Directory:    D:\Console\NES\Nestopia\save\
Dump:         Unknown
That left me wondering where the real problem was. It was almost like the combination of two <wram> definitions doesn't work right somehow... except the 8KB Nestopia .sav DOES NOT work in FCEUX... so could it be that the wrong 8KB page is being written to disk? Sure. But there's also the evidence that using the .sav from FCEUX in Nestopia doesn't work either, so loading is busted too.

As a final "screw it, I'll just try this" test, I decided to change the NstDatabase.xml for this game to say <wram size="16k" battery="1" />, deleted any existing .sav, and go through the same procedure. Results (for both re-opening the ROM, as well as Hard Reset) -- nada: Load Data still fails. But... guess what size the .sav file was? 8KBytes. So even "crummy cheap workarounds" to induce a larger .sav file won't work.

Sorry if this seemed pointless or rambly, but this is an example of how convoluted this whole ordeal is (to me anyway). I'm therefore in agreement: there is definitely something very broken about the saving/loading for SOROM, and fixing that up in NstBoardMmc1.cpp through overrides/etc. I think is the only way to get this to work.
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: SOROM and older emulators

Post by rainwarrior »

Oh, if Board::GetWram already divides it like that then you probably don't need to add anything special in Mmc1 at all.

Like it sounds more like everything was set up with the correct thing in mind, but somewhere in the chain the data is getting dropped.

Adding a special case isn't going to help if the information isn't getting there. I guess following "id" around would be what you really need to do. With a debugger you could probably do it a lot faster, as opposed to trying to figuring out by reading the source. (Of course you'd need a debuggable build for that...)
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: SOROM and older emulators

Post by koitsu »

Yeah, agreed. This is complicated by the fact that the rdanbrook/nestopia repo doesn't include the Win32 GUI code/etc., so quite honestly I have no idea how to build NestopiaUE for Windows (with the standard GUI etc.) given what's there. :\
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: SOROM and older emulators

Post by rainwarrior »

I'm surprised you haven't filed a "windows build files missing" issue. ;)

I'd actually have built it and taken a quick look by now if I could. The only windows build files I could find in there make a DLL for libretro.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: SOROM and older emulators

Post by koitsu »

Yeah, rdanbrook semi-recently "revamped" it all so that it would be easier ported to other platforms; I believe he moved it to use SDL. There's no GUI any more, except on *IX, where there's some GTK support I think. Windows, OS X, etc. all have nothing, AFAIK. The Win32 binaries for the SDL/libretro combo he calls "winalpha" here, while the original Win32 one we're all used to (DirectX + GUI) is just called "win32": https://sourceforge.net/projects/nestopiaue/files/1.47/

Note that in that directory (nor in earlier versions on his Sourceforge site) there is no Win32 source. AFAIK, that's in violation of GPL (Nestopia is GPLv2). But because he's a good guy, I don't wanna push him on it. But I really would like to see the current 1.47 Win32 code on github or somewhere like that, since it includes the Visual Studio projects and all the GUI bits.

I think what he's doing is (manually) backporting fixes from GitHub into the original 1.40 Nestopia Win32 source, then build binaries of that on rare occasion. We haven't had a Win32 build since January 2016.
Post Reply