RSNS Savestate proposal

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

Post Reply
User avatar
Disch
Posts: 1848
Joined: Wed Nov 10, 2004 6:47 pm

RSNS Savestate proposal

Post by Disch »

Talk in that other thread got me psyched up about this.. and it does seem like a really good idea.

I typed up a priliminary outline:

http://www.geocities.com/disch_/rsns.txt

I was actually hoping I could get some feedback before starting on controllers and mappers. Or maybe even get someone else to do controllers, since I lack knowledge in any area other than standard gamepads.

I was going to dip into mapper stuff after I got your guys impression on the layout so far. I think I get a little too wordy at times, but I figure it's better to be clear than misunderstood.

Anyway, comments/ideas/critisism/additions not only welcome, but desired. Be brutal if you need, I really want to see something like this polished up.

We might have to get in touch with Xod somehow and see if we can get his support on this. I sure hope Q is willing to come on board as well. I'll still try and get bbitmaster to work it into the next FCEUXD -- once we have something more complete, that is.

One thing about mappers -- do you guys think it'd be a better idea to go with a generic MAPR block like SNSS did? Or should we do specific blocks for specific mappers (like an MMC1 block, or VRC6 block, etc). The former prevents an insanely large amount of block types, but the latter allows for easier version revisions of individual mappers as new info arises. What do you guys think?
User avatar
Zepper
Formerly Fx3
Posts: 3262
Joined: Fri Nov 12, 2004 4:59 pm
Location: Brazil
Contact:

Post by Zepper »

I already have my own format for savestates, within tags. It's easily loadable in other emus. Data never change if you can TAG the blocks. ;)
Nessie
Posts: 133
Joined: Mon Sep 20, 2004 11:13 am
Location: Sweden
Contact:

Post by Nessie »

Fx3: Yeah, I also use blocks and tags. However, I think agreeing on a common format is better than having everyone implement RockNES' format.

Great, Disch. :)
A few suggestions:
- Is it really necessary to acommodate emus that don't emulate the MMC6 properly? If anyone were to implement the RSNS, perhaps they could fix their MMC6 emulation too.
- "romname.rs#", would that allow romname.rs10 too? Cause I wouldn't want to restrict the no of saves to 10.
- Would be nice with a block that's reserved for custom data. Some emus may want to save extra data such as statistics, screenshots, timestamps, etc - stuff that isn't essential for emulation.
User avatar
Zepper
Formerly Fx3
Posts: 3262
Joined: Fri Nov 12, 2004 4:59 pm
Location: Brazil
Contact:

Post by Zepper »

^_^;;
Last edited by Zepper on Thu Jul 21, 2005 4:30 pm, edited 1 time in total.
User avatar
Disch
Posts: 1848
Joined: Wed Nov 10, 2004 6:47 pm

Post by Disch »

- Is it really necessary to acommodate emus that don't emulate the MMC6 properly? If anyone were to implement the RSNS, perhaps they could fix their MMC6 emulation too.
I know it seems sloppy, but the problem is due to iNES there's no way to tell whether a game is MMC3 or MMC6 short of CRC checking for a game ID (game specific hacks like that REALLY rub me the wrong way). For emus which don't CRC check, they'll try and load an MMC3 savestate, but emus which do will load a MMC6 savestate.

And personally -- I would argue that CRC checking to bypass the header is the bigger problem here. Rather than working on ways to make bad headers work, I feel we should be working on making headers right -- and in fact should be forcing bad headers to operate as they dictate (ie: bad header = broken ROM).

But that's beside the point -- the method proposed in the state format may not be ideal, but it's a realistic preventative measure, and it only costs 4k of space. I was even thinking of adding another value to specify the desired offset for that RAM (so that the first 4k wouldn't even have to be saved).
- "romname.rs#", would that allow romname.rs10 too? Cause I wouldn't want to restrict the no of saves to 10.
I was keeping SNSS's idea to allow for crippled 8.3 file naming convention. Although I don't really see the harm in doing something like .rs10 -- since to load that in an emulator which only has 10 savestate slots you'd have to rename it anyway to change the slot number.
- Would be nice with a block that's reserved for custom data. Some emus may want to save extra data such as statistics, screenshots, timestamps, etc - stuff that isn't essential for emulation.
I agree completely. If nothing else this would allow emulators to create a sort of custom format that would at least be mostly supported in other emulators without extra effort on their part. The personalized blocks could be ignored by emulators which don't recognize them.

It couldn't be one custom block for every emu though -- each emulator should have to have their own block ID (or one emu's custom block would conflict with another emu's). Perhaps we could say custom blocks have to stick with all lowercase letters or something. Of course custom blocks would have to be optional in whichever emu they're supported (otherwise states from other emulators wouldn't work).


Other things that were suggested:

- split up MAIN into CPU/PPU/APU sections so that OOP designed emus have an easier time.

- rather than forcing the state to be done before NMI/IRQ -- we could just keep a flag to mention if the NMI has been executed or not.

- the 'BG Color index' may have been ill advised. I guess you're just supposed to draw whatever the PPU address is pointing at (if PPU is between $3F00-$3FFF) -- or $3F00 (if PPU address is elsewhere).
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples »

Nessie wrote:- "romname.rs#", would that allow romname.rs10 too? Cause I wouldn't want to restrict the no of saves to 10.
romname.1.rsns, romname.2.rsns, romname.3.rsns, romname.11.rsns
- Would be nice with a block that's reserved for custom data. Some emus may want to save extra data such as statistics, screenshots, timestamps, etc - stuff that isn't essential for emulation.
Might want to have a one or more 'BXML' blocks for XML data in WBXML encoding. Screenshots could be 'PNG ' blocks containing a PNG file.
There cannot be two different blocks with the same identifier string in a file.
So does that rule out support for keyframes or for multiple screenshots?
The size of this block should always be $1157
Shouldn't block sizes be a multiple of four bytes to make parsing easier on 32-bit machines?
bit 1 -> Odd frame
It might be better for future-proofing to store a 32-bit timestamp in frames since power-up. This could be used to simulate the PPU freezing that occurs when $2003-$2007 is accessed before two frames have elapsed, or to simulate the overheating that made sprites flicker in prototypes of The Three Stooges according to Andrew Davie's posts to nesdev@yahoogroups, EDIT: or to store the time of a given keyframe when there are multiple keyframes in one save state.
DMC information is stored in this block as opposed to the ADIO block because the DMC can affect CPU flow through the game (by both IRQ and/or stolen cycles), whereas nothing in the ADIO block is critical (aside from preventing audio distortion).
Don't the length counters affect CPU$4015, and don't some games run their timing off CPU$4015? Where in your spec is the current state of the length counters saved?
Last edited by tepples on Thu Jul 21, 2005 10:39 am, edited 1 time in total.
User avatar
Disch
Posts: 1848
Joined: Wed Nov 10, 2004 6:47 pm

Post by Disch »

tepples wrote:romname.1.rsns, romname.2.rsns, romname.3.rsns, romname.11.rsns
Do you guys think ignoring 8.3 restrictions is wise? I could go either way, but I leaned towards being compatible when typing up the blueprints.

Personally, I never had a need for more than 10 (hell, I don't think I ever used more than 4) slots. Nor do I plan on adding more than 10 to my emu. And considering 10 slots is the 'norm' anything beyond that will have to be renamed when moved to another emu anyway.
Might want to have a one or more 'BXML' blocks for XML data in WBXML encoding. Screenshots could be 'PNG ' blocks containing a PNG file.
I can see the desire for such info like screenshots and other things which might be common. Although since I had never had a use for such things (I only am really concerned with the state itself) -- perhaps someone else here would be willing to write up blocks covering these areas. I don't think I'd be the person for this particular job.
So does that rule out support for keyframes or for multiple screenshots?
I suppose that part could be removed. At the time I was only thinking in standard savestate terms.
Shouldn't block sizes be a multiple of four bytes to make parsing easier on 32-bit machines?
I don't see how it would make it any easier. I was never a fan of padding to the nearest 4 bytes in file formats -- they always just complicated things for me (I remember the frustration brought up with this when working with BMPs long ago).

I'm a fan of the no padding -- and personally I'd prefer to keep it the way it is.
It might be better for future-proofing to store a 32-bit timestamp in frames since power-up.
That is a good idea.
Where in your spec is the current state of the length counters saved?
DOH I knew I forgot something.

Remembered the Linear counter but forgot the length counters -- aye. I'll have to squeeze those in.

I'm also thinking of moving the DMC stuff with the other audio stuff and making all of it manditory. Things like the duty cycle counters and tri-step generator wouldn't really matter so much, but other things like the length counters and DMC could affect program flow.
User avatar
Quietust
Posts: 1920
Joined: Sun Sep 19, 2004 10:59 pm
Contact:

Post by Quietust »

Disch wrote:
tepples wrote:romname.1.rsns, romname.2.rsns, romname.3.rsns, romname.11.rsns
Do you guys think ignoring 8.3 restrictions is wise? I could go either way, but I leaned towards being compatible when typing up the blueprints.
To hell with 8.3 restrictions. The only operating systems that rely on 8.3 (MS-DOS being the only one that actually has NES emulators written for it) are no longer in common use today, so it's pointless to restrict today's more advanced environments.
Quietust, QMT Productions
P.S. If you don't get this note, let me know and I'll write you another.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples »

Disch wrote:
tepples wrote:romname.1.rsns, romname.2.rsns, romname.3.rsns, romname.11.rsns
Do you guys think ignoring 8.3 restrictions is wise?
GNU/Linux has long file names, *BSD has long file names, Mac OS has had 31-character file names since 1.0 (1984) and long file names since 10.0, and Windows has had long file names since 1995. One of the most popular file name suffixes lately (.torrent) has seven characters.
I could go either way, but I leaned towards being compatible when typing up the blueprints.
rs0 through rs9 would take 10 slots in the Windows file name suffix registry.
I don't see how [making chunk sizes a multiple of 4 bytes] would make it any easier.
Virtually every 32-bit architecture other than x86 needs 32-bit memory accesses to be aligned to a 4-byte boundary. Aligning all multi-byte integers on their "natural" size (all 16-bit integers aligned to 2 bytes and all 32-bit integers aligned to 4 bytes) would allow for in-place parsing using itohs() or itohl() (Intel to host short/long, analogy to ntohs() and ntohl() used in TCP/IP). Being able to parse a binary file in memory without having to allocate more memory is especially handy in low-memory situations, such as emulators on handheld devices (e.g. PocketNES for Pocket PC or, worse yet, PocketNES for GBA) or emulators that use keyframes (such as anything implementing Sands of Time or emulators that can act as an AVI codec).
User avatar
blargg
Posts: 3715
Joined: Mon Sep 27, 2004 8:33 am
Location: Central Texas, USA
Contact:

Post by blargg »

I like the distinction between audio state that can affect long-term emulation and that which can't. As tepples noted, the length counters can also affect CPU operation since they affect what is read out of $4015.

The square phase ranges from 0 to 7, not 0 to 15 (I have just re-verified that the square timer itself is reloaded with raw_period * 2 + 2, as documented on the NesDev Wiki).

The APU frame counter operation is significantly more complex than is currently documented. I haven't finished the second round of APU reverse-engineering I started many months ago (it got put on hold). I should prioritize on aspects of the APU that affect the CPU. To give you an idea of what I mean, here is a small bit of its exact operation:
Cycles since write to $4017 on even internal APU clock (on odd clock, everything occurs one clock later):

Code: Select all

29830
        Perform CPU read/write
        Set frame interrupt flag if $4017 = $00
29831
        Perform CPU read
        Perform CPU write to $4008
        Clock linear
        Clock length
        Perform other CPU writes except length counter reload (ignored)
        Set frame interrupt flag if $4017 = $00
29832
        Perform CPU read/write
        Set frame interrupt flag if $4017 = $00
29833
As shown, the frame irq flag is set three times (so that if you read it at 29831, it will be set again and still set at 29832 and beyond). Also, a write to a length counter reload register at 29831 is completely ignored; the length counter is never reloaded.
Ignore the following minor quibbles if you like :)

With the header having a count of the blocks in the file, it requires rewriting after writing blocks (or knowing in advance how many blocks there will be). If you don't like treating EOF as the end of blocks, you could require an explicit "end of file" block at the end.

For some reason I think it's a good idea to put large arrays in separate blocks, for example low memory in its own block with nothing else. Also regarding arrays, I don't like their size being determined by the size of the block. Why not store the size of the array just before the elements?

I don't like the excessive bit packing in the audio state. I think it would be simpler if it were more relaxed and grouped by channel. Something like

Code: Select all

Square 0
	Envelope Reset Flag
	Envelope Divider Period Counter
	Envelope Volume Counter
	Sweep Reset Flag
	Sweep Divider Period Counter
	Tone Divider Counter
	Duty Cycle step (0-15)

Square 1 (same)

Triangle
	Linear Counter Halt Flag
	Linear Counter
	Tone Divider Counter
	Tri-stp (0-31)

Noise
	Envelope Reset Flag
	Envelope Divider Period Counter
	Envelope Volume Counter
	Tone Divider Counter
	RNG Shift Register
Regarding the noise rng never being zero, a correct emulator would never have zero here, and it's trivial to check this when loading (and use 1 instead).
User avatar
Zepper
Formerly Fx3
Posts: 3262
Joined: Fri Nov 12, 2004 4:59 pm
Location: Brazil
Contact:

Post by Zepper »

blargg, you said that the square phase goes from 0 to 7... and you wrote 0 to 15... :)
Post Reply