the "right" timing for savestates?

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

User avatar
Zepper
Formerly Fx3
Posts: 3190
Joined: Fri Nov 12, 2004 4:59 pm
Location: Brazil
Contact:

the "right" timing for savestates?

Post by Zepper » Mon Jan 14, 2019 1:37 pm

I wonder... Eugene has found an obscure bug in my emulator regarding savestates during a screen transition. The game is Battletoads & Double Dragon.

The bug occurs when a savestate is saved right during a screen transition (in this case, when you go from a sublevel to another, like 2-2 to 2-3, and the screen fades to black). Loading the state before the new level "reappears", the emulator can crash (invalid opcode) or display glitched graphics. The result depends of the timing.

Is there a right time for saving/loading a state? I mean, in mid-instruction or only after the entire instruction be completed..? What if an NMI or IRQ is pending? Should I do the NMI/IRQ, then poll the keys? I need some advice.
Last edited by Zepper on Mon Jan 14, 2019 6:17 pm, edited 1 time in total.

lidnariq
Posts: 8692
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: the "right" time for savestates?

Post by lidnariq » Mon Jan 14, 2019 1:42 pm

If you're getting a crash, there has to be some state that is necessary to restore that you're not restoring. There's almost nothing you can safely leave at a default value when restoring a save state.

You don't have to resolve the NMI/IRQ before saving; you just have to make sure you're saving all the state necessary.

User avatar
Zepper
Formerly Fx3
Posts: 3190
Joined: Fri Nov 12, 2004 4:59 pm
Location: Brazil
Contact:

Re: the "right" time for savestates?

Post by Zepper » Mon Jan 14, 2019 1:48 pm

The problem is very specific. If I save a state when the game is loading the next level, then restore it, the bug appears.

User avatar
tokumaru
Posts: 11438
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: the "right" time for savestates?

Post by tokumaru » Mon Jan 14, 2019 2:01 pm

Why not trace log this particular moment once when saving the state and then again after loading so you can compare the two and see where things go wrong? That might give you some insight on which specific bit of state is not being restored properly.

User avatar
zeroone
Posts: 929
Joined: Mon Dec 29, 2014 1:46 pm
Location: New York, NY
Contact:

Re: the "right" time for savestates?

Post by zeroone » Mon Jan 14, 2019 2:05 pm

It could be any number of issues:
  1. Failure to persist the full state of the emulator.
  2. Failure to pause the emulation during state capture causing a rolling-shutter-like bug.
  3. Failure to resume properly after loading the state.
  4. Just some stupid coding bug in save state logic.
Resuming from a save state is actually a bit challenging. If the emulator were a text-book state machine, then the state could be easily persisted and restored. But, more often than not, little bits of state end up all over the place and the execution thread needs to jump to exactly where it left off.

User avatar
Dwedit
Posts: 4233
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: the "right" time for savestates?

Post by Dwedit » Mon Jan 14, 2019 3:08 pm

I was working on something like this for RetroArch, a savestate validation system.

What I'm doing:
Run two different instances of the emulator, give them the exact same input.
Each frame, do this instead:

Instance A: Run Frame, Save State
Instance B: Run Frame, Save State, Run Frame (optional), Load State

Then you compare the contents of the savestates. If they match, it's all good. If they don't, there's an error.
If they don't match, you can run the emulator in Trace logging mode and see where they differ.

For RetroArch, I was also capturing the sound and video output and comparing that as well.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!

User avatar
koitsu
Posts: 4214
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: the "right" time for savestates?

Post by koitsu » Mon Jan 14, 2019 4:40 pm

Sounds like either a bug in the save state restoration code, *or* (and this is more likely I'd think) there's something you're not storing in a save state to begin with that's necessary to ensure reliable restoration.

Usually it's just MMIO contents, register contents, RAM, and some general ROM state (PRG bank, etc.). But I bet there's some internal PPU state that's not being saved/restored. 100% speculative on my part.

So... what exactly do you store in your emulators' save state file? Maybe by disclosing that we can tell you what might be missing that needs to be saved.

(Same question applies to any/all emulator authors. Maybe Sour can explain what he saves in Mesen's save states and you two can compare.)

User avatar
Zepper
Formerly Fx3
Posts: 3190
Joined: Fri Nov 12, 2004 4:59 pm
Location: Brazil
Contact:

Re: the "right" time for savestates?

Post by Zepper » Mon Jan 14, 2019 6:16 pm

None of you got the point, so I changed the topic slightly.
I'm asking if it's OK to save during mid-instruction, or if I should wait the current instruction to be completed, then poll keyboard (for saving/loading).

lidnariq
Posts: 8692
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: the "right" timing for savestates?

Post by lidnariq » Mon Jan 14, 2019 7:29 pm

We did answer that.

We said: Are you saving all the state for whatever you choose?

User avatar
koitsu
Posts: 4214
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: the "right" timing for savestates?

Post by koitsu » Mon Jan 14, 2019 7:50 pm

Okay, I'll give my answer, given the subject change + edit applied to the initial post (substantially more clear to me, thanks!):

My advice would be to only save state after the running 6502 instruction has executed.

Can anyone think of a case where waiting for a 6502 instruction to finish (before doing the actual save state) would result a bad UX (user experience)? Example: user press keyboard shortcut to save state (or GUI option via mouse), followed by a long delay (greater than, say, 0.5 seconds) before the user got actual confirmation that the state was saved to the file? I can't think of one off the top of my head.

User avatar
Quietust
Posts: 1487
Joined: Sun Sep 19, 2004 10:59 pm
Contact:

Re: the "right" timing for savestates?

Post by Quietust » Mon Jan 14, 2019 8:00 pm

If you want to allow saving state at any point in time, then you need to save all of the data necessary to fully represent that point in time - e.g. if you're saving mid-scanline, you need the entire state of the sprite state machine to ensure that sprite 0 hit and/or sprite overflow will occur correctly.

If, on the other hand, you enforce an arbitrary rule that e.g. savestates can only be captured during VBLANK, then you can safely discard a lot of that data because it'll get reinitialized from scratch before it would need to be used. My own emulator requires savestates to be saved during scanline 240 (the post-render scanline before NMI), which made things a whole lot simpler, though I suppose it also opens the possibility of a malicious program that causes the emulator to hang if you try to save its state (e.g. by initiating a Sprite DMA at scanline 239).

For what it's worth, I can't think of any reason why you would even want to allow pausing emulation mid-instruction, let alone creating a savestate.
Last edited by Quietust on Mon Jan 14, 2019 8:23 pm, edited 1 time in total.
Quietust, QMT Productions
P.S. If you don't get this note, let me know and I'll write you another.

User avatar
Zepper
Formerly Fx3
Posts: 3190
Joined: Fri Nov 12, 2004 4:59 pm
Location: Brazil
Contact:

Re: the "right" timing for savestates?

Post by Zepper » Mon Jan 14, 2019 8:20 pm

Quietust wrote:For what it's worth, I can't think of any reason why you would even want to allow pausing emulation mid-instruction, let alone creating a savestate.
In my code, there's a function to poll keys right before the VBlank (line 240, cycle 0). Such function is called inside ppu_clock() which is called at every CPU read/write (1 CPU cycle = 3 PPU cycles). So, if polling keys is inside ppu_clock() it may be called in mid-instruction or even during an NMI/IRQ.

So, this way, I suppose it's NOT correct at all. So, waiting for the current instruction to finish (+ any pending interruptions) seems to be the right timing for it.
koitsu wrote:My advice would be to only save state after the running 6502 instruction has executed.

ap9
Posts: 39
Joined: Sat Jun 01, 2013 11:55 am
Location: Maine, U.S.A.
Contact:

Re: the "right" timing for savestates?

Post by ap9 » Mon Jan 14, 2019 10:46 pm

Sometimes timing can be sensitive that if you're off by a cycle or subcycle, if can ruin consistency or crash the game. I know that resetting Super Mario Bros 3 at the wrong time can cause it to crash.

Are you sure the save time is exactly the same as the restore time? It doesn't matter if it's in the middle of an instruction if even the instruction state is saved, restored at the same point in process.

My suggestion is to just keep it simple and consistent where and when the state save/load is made.

User avatar
Zepper
Formerly Fx3
Posts: 3190
Joined: Fri Nov 12, 2004 4:59 pm
Location: Brazil
Contact:

Re: the "right" timing for savestates?

Post by Zepper » Tue Jan 15, 2019 5:36 am

ap9 wrote:Sometimes timing can be sensitive that if you're off by a cycle or subcycle, if can ruin consistency or crash the game. I know that resetting Super Mario Bros 3 at the wrong time can cause it to crash.
Good point.
ap9 wrote:Are you sure the save time is exactly the same as the restore time? It doesn't matter if it's in the middle of an instruction if even the instruction state is saved, restored at the same point in process.
It's true only for the PPU timing, but not for the CPU. I believe to have fixed such issue, as koitsu pointed out.

WedNESday
Posts: 1236
Joined: Thu Sep 15, 2005 9:23 am
Location: Berlin, Germany
Contact:

Re: the "right" timing for savestates?

Post by WedNESday » Thu Jan 17, 2019 6:50 am

From what I've read, it does seem like a variable not being restored properly.

I've never personally implemented savestates for WedNESday, so I've no experience with them. But shouldn't an emulator wait for the entire frame to finish before saving the savestate?

Post Reply