ppu loading rom or using ram

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

User avatar
Disch
Posts: 1848
Joined: Wed Nov 10, 2004 6:47 pm

Post by Disch »

PPU cycles * 5 is what my emu does, that way both NTSC and PAL CPU cycles will always land on an even cycle. But like I said I don't know if that's "friendly" enough.

Other things worth noting:

- little endian. The human readability factor is kind of a lame reason to go with big endian.. especially considering that there's no padding so the file is already going to be kind of a mess to view in a hex editor. Plus binary files don't need to be human readable. Little endian is the logical way to go for NES savestates.

- Saving 4 nametables seems like kind of a waste since 99.99% of the games out there only have 2. Games that have more can have the extra nametables in the mapper section -- or should at least make the number of nametables in the state variable.

- As Q initially mentioned, specifying the swapped in PRG/CHR banks might not be the way to go. It makes more sense and would be safer to have swap stuff in the mapper section (the contents of mapper registers should suffice -- and they'll probably have to be saved anyway for other reasons like wram mapping, mirroring, and other stuff).

- I don't see the point for a byte specifying whether or not SRAM is writable -- that can be determined from the mapper registers. Plus it doesn't help in determining if SRAM is readable -- or even if it's swapped in. And it creates conflicts with mappers like FME-07 which put PRG at $6000-7FFF

- Mapper section needs a serious overhaul. Every mapper (save NROM/mapper 0) should save something. The contents of all mapper registers should be saved, along with other things needed (IRQ counters, reload values, the MMC1 shift register and relating info, etc)

- Nothing that can affect emulation should have to be assumed when you load a savestate (other than when in the frame the state has occured). The savestate should dictate everything needed to playback exactly as it would have had emulation resumed from the time the state was saved.

I don't know what precautions could be made to work with the Sands-of-time effect. I don't see what kind of special info you'd need in a savestate for that.

I'm really digging this idea. I'd like to start up preliminary format specs for this tomorrow (unless of course Baisoku will do it) -- although I'm skeptical as to how many emus will actually support it. Not very many of the biggies seem to be in active development -- our best bets (or our only bets) seem to be Nintendulator and Nintencer (I think Xod would be willing to support it, especially considering Nintencer is still pretty young). Maybe FCEU as well since all these new FCEUXD builds are coming out (and I just chatted with bbitmaster, he says he might be willing to come on board with the next FCEUXD release). Though, still... it may not be worth the work to develop this format if only three major emus support it. What do you guys think?
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples »

Disch wrote:- Saving 4 nametables seems like kind of a waste since 99.99% of the games out there only have 2. Games that have more can have the extra nametables in the mapper section -- or should at least make the number of nametables in the state variable.
If nametable VRAM contents go in the mapper section, then so do pattern table VRAM contents and CPU$6000 SRAM contents.
- Nothing that can affect emulation should have to be assumed when you load a savestate (other than when in the frame the state has occured).
And even that can be simplified by specifying that all save states take effect at the moment of vblank.
I don't know what precautions could be made to work with the Sands-of-time effect. I don't see what kind of special info you'd need in a savestate for that.
You'd need a format similar to that of a movie, storing the last few seconds of gameplay. The emulator would expand it to whatever kind of cached state it needs on a Load State command. But it would still be nice to have keyframes, especially if someone wants to make an emulator that acts as a VFW or DirectShow decoder.
Maybe FCEU as well since all these new FCEUXD builds are coming out (and I just chatted with bbitmaster, he says he might be willing to come on board with the next FCEUXD release). Though, still... it may not be worth the work to develop this format if only three major emus support it. What do you guys think?
We'd need to standardize the movie format as well, right? If save states and movies are portable, then it would become possible to compare the pixel-for-pixel and sample-for-sample output of two emulators (modulo phase differences in the audio, which we can chalk up to different TVs).

Eventually, one goal I can see is to have the emulator itself as a module (e.g. a .dll or .so file) that takes a UNIF file, an initial state, and streams from the input devices, and produces video and audio streams as output. This refactoring, analogous to what has happened in PS1 emulators, would let one use, say, the FCEUXD frontend with the Nintendulator backend.
User avatar
Disch
Posts: 1848
Joined: Wed Nov 10, 2004 6:47 pm

Post by Disch »

tepples wrote:If nametable VRAM contents go in the mapper section, then so do pattern table VRAM contents and CPU$6000 SRAM contents.
Well perhaps they should all be in their own respective blocks.

PPU block (has the 2 native nametables)
Cartridge RAM block ($6000 area)
Cartridge Nametable block (for games with 4-screen mirroring -- and possibly for MMC5's ExRAM)
CHR RAM block

This way, the rare occurence of 3+ nametables is accounted for, but doesn't unnecesarily bloat states which don't need it. Plus the far more common cartridge RAM can still be a variable size without having to keep extra nametables restricted to a fixed size. Plus CHR RAM can also have a variable size if needed (i'm sure there's some game out there somewhere that has more than 8k CHR-RAM)
And even that can be simplified by specifying that all save states take effect at the moment of vblank.
I think Quietust's idea is best -- the start of the dead scanline right after rendering, but before VBlank. That way you don't have to worry about pending NMIs, and you also avoid the issues of being so close to rendering time. But yeah, that's one thing that I think should be assumed on state load (since it's just too much work for no gain to have the savestate allowed to be saved anywhere in the frame)

EDIT -- although I just thought of a problem. If the game writes to $4014 just before the start of that dead scanline, that would be a major problem since the CPU would be stalled through the whole scanline and into VBlank. Granted this is a rare occurance, but it might happen in games like Castelian which shut the screen off early to squeeze in more drawing time.

How about putting it a bit into VBlank? Far enough in to stay away from the triggered NMI, but near enough to the start so that there's no problem on $4014 stalls. Maybe something like 1 scanline into VBlank. Although I don't much fancy the idea of splitting VBlank... it might be the best option. Or maybe it would be simpler to just do it at the start of VBlank and just deal with NMIs. What do you guys think?
You'd need a format similar to that of a movie, storing the last few seconds of gameplay.
So the state is actually somewhat of a short movie file? I'm not entirely sure I'd agree with this idea, since it'd considerably slow down state loading -- not to mention it would force all emus to have a movie player... making it much harder to support the format.

Besides... would this even be necessary for the rewind feature? I mean its absence would only mean you can't rewind immediately after loading a state.

Perhaps we could document it as an optional block for emus which support the feature. It by no means should be a required block, imo. But then of course that leads to sideeffect of savestates restoring to different times in different emus.
We'd need to standardize the movie format as well, right?
That's a very good idea, which I actually hadn't thought of. I'm not really as interested in the movie format -- at least not yet. Though it is definatly something we should keep in mind when making the savestate format, since they're undoubtedly connected.


One thing about a movie format I will say -- I think it's better to drive movies by joypad strobes rather than by frames (since it would be theoretically possible to strobe a joypad twice in one frame). The games have to strobe the joypad to fetch keypresses anyway -- so that seems like the logical time to fetch them from the movie file as well (as well as record them)
Marty
Posts: 40
Joined: Fri Nov 12, 2004 5:02 am

Post by Marty »

I rewrote my own movie code recently and took a different approach. I decided to scrap the controllers and instead just feed on the $4016 and $4017 ports only. The gain is precise accuracy and that any device can be hooked up and switched in real-time without affecting and/or complicating the movie at all. DIP switch configuration for VS. System games gets preserved as well.

With this scheme the only free bit available is $4016.7 which I let act as a CTRL bit with the next byte(s) telling how many frames ahead the current frame input block data should repeat itself - similiar to RLE. $4016 and $4017 are maintained seperately and while $4017 can't afford any bits it gets its own CTRL byte.

The downside to all this is bigger files since often times unnused data will be passed in but it can be kept at a minimum by compressing the different streams in which I let Zlib do for me. The compression ratio is very good since the input data is well suited for the dictionary-based LZx algorithm Zlib uses.

For reference, here are the file sizes for five minutes recording with:

SMB1 - 1725 bytes, Oeka Kids (drawing) - 26329 bytes
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples »

Disch wrote:Plus CHR RAM can also have a variable size if needed (i'm sure there's some game out there somewhere that has more than 8k CHR-RAM)
Videomation and any homebrew games that use the same board (CPROM).
If the game writes to $4014 just before the start of that dead scanline, that would be a major problem since the CPU would be stalled through the whole scanline and into VBlank. Granted this is a rare occurance, but it might happen in games like Castelian which shut the screen off early to squeeze in more drawing time.
And my tetramino game, which shuts off the screen early for the same reason.
How about putting it a bit into VBlank? Far enough in to stay away from the triggered NMI, but near enough to the start so that there's no problem on $4014 stalls.
A $4014 copy can happen at any time during vblank or at any other time when the screen is turned off. In fact, DMC DMA can happen at any time, and whatever time you choose might happen to be a cycle in the middle of a DMA.
Maybe something like 1 scanline into VBlank. Although I don't much fancy the idea of splitting VBlank... it might be the best option. Or maybe it would be simpler to just do it at the start of VBlank and just deal with NMIs. What do you guys think?
That might be the best option.
So the state is actually somewhat of a short movie file? I'm not entirely sure I'd agree with this idea, since it'd considerably slow down state loading -- not to mention it would force all emus to have a movie player... making it much harder to support the format.
It'd be optional. Emulators that don't support the input log block would just always start the emulation at the beginning.
Besides... would this even be necessary for the rewind feature? I mean its absence would only mean you can't rewind immediately after loading a state.
Unless you provide a "keyframe" method to store states that happen along the way, an emulator would start from the beginning. The "keyframe" method would also be useful in an emulator that acts as an AVI filter.
But then of course that leads to sideeffect of savestates restoring to different times in different emus.
Unless it is RECOMMENDED that the emu have options for starting at the beginning of any keyframe or at the end of movie.
The games have to strobe the joypad to fetch keypresses anyway -- so that seems like the logical time to fetch them from the movie file as well (as well as record them)
How would the Zapper fit in to this system? It doesn't use strobing because it has one button and one photosensor, on different data bits.
User avatar
blargg
Posts: 3715
Joined: Mon Sep 27, 2004 8:33 am
Location: Central Texas, USA
Contact:

Post by blargg »

Just some things that come to mind.

- Use a tagged, chunked format (like IFF) to allow extension without requiring all emulators to be updated. Having a chunk for every single value might be an overkill, so a standard chunk could be a variable-length set of 4-character tags and 32-bit integer pairs, which would suffice for most registers.

- If possible, use a completely tagged format so that fields can be accessed in code via a 4-character tag rather than using structures. I could help design and implement this.

- Store the last value written to registers. In most cases this is sufficient to restore hardware state, and easiest to implement in any emulator. If the last written value isn't sufficient, also store any extra internal hardware state.

For example, applied to the APU frame counter ($4017) the state file needs to store the last value written, internal frame interrupt flag, how long until the next frame, and the frame number.

- When storing hardware state, store only what is necessary in a format closest to how it is in hardware. This won't favor any particular emulator and will be the most stable format.

- Avoid storing redundant data, unless really inconvenient. Redundancy allows inconsistency.

- The sands-of-time feature only needs a save state in the past along with external input (joypad) from then to the present (the same requirement as a normal movie). Periodic save states along the way will reduce initial setup delay, but aren't absolutely necessary.

Basic support (being able to rewind n seconds and no further) can be handled by a mini-movie from n seconds ago to present. Full support (being able to rewind back to the beginning) requires a full movie from power-up, and would benefit greatly from periodic save states along the way. These periodic "key frames" would speed up normal movie seeking too.

- To test an implementation, modify the emulator to continuously save, reset emulator, then restore the state every frame.

- Having a common emulator format would help validate emulators. Each could be run on the same code for the same amount of time and the save states could be compared.
User avatar
Disch
Posts: 1848
Joined: Wed Nov 10, 2004 6:47 pm

Post by Disch »

Well given my lack of experience with non-standard NES pads (like the zapper) and just movies in general, I'll keep my nose out of the movie area for the most part. Just remember that the goals we're going for are not only to be efficient and to cover as many possibilities as possible, but also to be easy to impliment in an emulator. After all if it's a royal pain in the ass to use the proposed standard format, people are just going to make their own.

As for the designated state time -- I'm not especially thrilled about having the emu split their VBlank for state loading/saving, since that's likely to be run in a chunk in pretty much every emu. Although scanline 240 might be considered part of that chunk, so putting the state at the start of VBlank might end up splitting emulation at an awkward time anyway.

Either way it looks like there's 4 options:

- Start of scanline 240 (have to account for pending NMIs)
- Start of VBlank (have to account for pending NMIs)
- 1 Scanline into VBlank (don't have to worry about NMIs, but a little awkward)
- Start of VBlank (if NMIs disabled) or immediately after NMI (if NMIs enabled)

During or just before rendering is out of the question -- since this raises a whole slew of other problems.

The 3rd and 4th options will avoid the complications of state loading right before NMI --- however they may end up being more complicated than just working around pending NMIs (4th especially).

Maybe we should just stick with scanline 240... and if it spills into VBlank so be it. We'll just have to account for that. That will be a rare occurrance anyway, and shouldn't be too difficult to work around.

For reference/clarification -- when VBlank is started there's latency on when an NMI actually occurs, right? Is it a certain number of cycles? Or do you just run 1 CPU instruction before tripping an NMI? My emu just ran a single instruction and that seemed to work.


@ Blargg:

Yes, I agree about the tagged chunk/block format. SNSS was set up that way and I just assumed we were going to do something similar (at least that's what I had in mind).
Post Reply