Defining a cross-emulator save state format
Moderator: Moderators
Defining a cross-emulator save state format
In this post, mkwong proposed designing a save state format that is portable across different NES emulators. Let me describe some snags that an effort to design such a format spec would encounter:
How are you going to define a representation for the internal state of hundreds of mappers? That'd be a lot of work.
As we come to understand the NES better, we may discover more state that we need to store. Either it'll have to be a name-value pair format (except for bulk RAM contents) or there'll need to be some sort of version number for the Control Deck's state and the mapper's state.
How are you going to define a representation for the internal state of hundreds of mappers? That'd be a lot of work.
As we come to understand the NES better, we may discover more state that we need to store. Either it'll have to be a name-value pair format (except for bulk RAM contents) or there'll need to be some sort of version number for the Control Deck's state and the mapper's state.
Re: Defining a cross-emulator save state format
Cross-emulator save state formats have been tried in the past. They don't really work.
Quietust, QMT Productions
P.S. If you don't get this note, let me know and I'll write you another.
P.S. If you don't get this note, let me know and I'll write you another.
Re: Defining a cross-emulator save state format
A Q post from 2005 which I wholeheartedly agree with.
Re: Defining a cross-emulator save state format
Yes, that is definitely a lot of work. But as all authors of emulators with save state functionality have done it already, why not spare newcomers from reinventing the wheel?tepples wrote:How are you going to define a representation for the internal state of hundreds of mappers? That'd be a lot of work.
Re: Defining a cross-emulator save state format
Well, if nothing else, attempt to export and import savestates from the "popular" open-source emulators. Maybe FCEUX, Nestopia, Nintendulator, etc...
You might not have complete information needed for another emulator, but some things can be extrapolated.
For example, if you have the current CPU or PPU memory map, you can fill in what the values for mapper latches should be.
If you know the PPU raster position that the save state would happen at, you can ensure that the save state is made at that time.
Even if you can't get a 100% accurate state conversion that would survive TAS-like validation, you might still get something that can let you proceed to play the game without a crash.
You might not have complete information needed for another emulator, but some things can be extrapolated.
For example, if you have the current CPU or PPU memory map, you can fill in what the values for mapper latches should be.
If you know the PPU raster position that the save state would happen at, you can ensure that the save state is made at that time.
Even if you can't get a 100% accurate state conversion that would survive TAS-like validation, you might still get something that can let you proceed to play the game without a crash.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
- rainwarrior
- Posts: 8732
- Joined: Sun Jan 22, 2012 12:03 pm
- Location: Canada
- Contact:
Re: Defining a cross-emulator save state format
Sparing them from reinventing the wheel? Why exactly are they making a new emulator for a system that has hundreds?mkwong98 wrote:Yes, that is definitely a lot of work. But as all authors of emulators with save state functionality have done it already, why not spare newcomers from reinventing the wheel?tepples wrote:How are you going to define a representation for the internal state of hundreds of mappers? That'd be a lot of work.
I think the idea of a savestate converter tool might be fun.Dwedit wrote:Well, if nothing else, attempt to export and import savestates from the "popular" open-source emulators. Maybe FCEUX, Nestopia, Nintendulator, etc...
You could do this as its own separate utility without trying to burden any emulator authors with it. It might also become a nice point of reference for the actual savestate binary formats used by various emulators, since that stuff tends not to be well documented at all.
(On the other hand, the benefits of having such a thing are probably so small that nobody would ever want to undertake a project like this.)
Re: Defining a cross-emulator save state format
My idea is similar to yours. A minimum amount of info required for a save state is defined. Beyond that, values are defined with a default value.Dwedit wrote:You might not have complete information needed for another emulator, but some things can be extrapolated.
For example, if you have the current CPU or PPU memory map, you can fill in what the values for mapper latches should be.
If you know the PPU raster position that the save state would happen at, you can ensure that the save state is made at that time.
Even if you can't get a 100% accurate state conversion that would survive TAS-like validation, you might still get something that can let you proceed to play the game without a crash.
Because they want to.rainwarrior wrote:Sparing them from reinventing the wheel? Why exactly are they making a new emulator for a system that has hundreds?
Anyway we can't ignore the possibility that there are people out there with new ideas and ability to create a good emulator. Mesen is a very good example of a relatively new emulator when there are already hundreds.
If we have a save state converter tool, the internal state of the tool itself can be used as the base of a cross-emulator save state format. Then we can change this tool into a lib for emulators to use and we are done!rainwarrior wrote:I think the idea of a savestate converter tool might be fun.
You could do this as its own separate utility without trying to burden any emulator authors with it. It might also become a nice point of reference for the actual savestate binary formats used by various emulators, since that stuff tends not to be well documented at all.
(On the other hand, the benefits of having such a thing are probably so small that nobody would ever want to undertake a project like this.)
Re: Defining a cross-emulator save state format
The basic NES memory can't be different, along the CPU state. The problem is all about mappers, I suppose.
Re: Defining a cross-emulator save state format
Different emulators use different models of PPU state, which may cause part of the problem even apart from mappers. That's why I recommended requiring portable save states to be taken in the post-render line, as it's when the PPU state is least in flux.
Re: Defining a cross-emulator save state format
Emulators can be used for at least four different purposes:tepples wrote:Different emulators use different models of PPU state, which may cause part of the problem even apart from mappers. That's why I recommended requiring portable save states to be taken in the post-render line, as it's when the PPU state is least in flux.
- Running as efficiently as possible a game that's known to use only a certain set of hardware features and programming techniques.
- Validating that a game only relies upon a particular set of hardware features and programming techniques.
- Running a game that uses an unknown set of hardware features and programming techniques.
- Experimenting with new programming techniques.
When running programs which don't exploit too many corner cases, an APU/PPU emulator which is written in an interpreted language but skips time-consuming corner cases could probably be faster than a more "accurate" emulator written in native code. Among other things, if the interpreted language included "sync" instructions to indicate when an action by the main CPU might have side effects that are observable on the PPU or vice versa, and if such side effects are rare, the emulator could reduce the rate at which it has to switch between processing CPU-side logic and PPU-side logic, thus improving cache performance.
If save-game files could include emulator *logic* in a portable format, then they could be processed by any emulator that can interpret the bytecode language.
Re: Defining a cross-emulator save state format
About the savestate, is there a problem of setting APU (and possibly) PPU timings to zero instead of saving cycle counters? Good about saving on scanline -1. I believe it's no so hard to create a standard savestate format with tagged blocks.
Re: Defining a cross-emulator save state format
The DMC timer can be used to generate raster-synchronized interrupts without requiring a special mapper, but losing the APU state would totally break things.Zepper wrote:About the savestate, is there a problem of setting APU (and possibly) PPU timings to zero instead of saving cycle counters? Good about saving on scanline -1. I believe it's no so hard to create a standard savestate format with tagged blocks.
- rainwarrior
- Posts: 8732
- Joined: Sun Jan 22, 2012 12:03 pm
- Location: Canada
- Contact:
Re: Defining a cross-emulator save state format
In addition to the DPCM IRQ...
The APU frame counter can also be polled or used as an interrupt.
The APU status bits will change e.g. when a DPCM sample reaches its end.
So there's a few APU things that a game might rely on. Probably a lot of games wouldn't have any discernable effect of losing APU state at a save, but I'm sure there are notable ones that would be affected.
There are some expansion audio devices that are effectively "write only" though. Those at least might potentially have no non-audio consequences. ...and of course if you care about audio consequences, this whole idea isn't even on the table anyway.
The APU frame counter can also be polled or used as an interrupt.
The APU status bits will change e.g. when a DPCM sample reaches its end.
So there's a few APU things that a game might rely on. Probably a lot of games wouldn't have any discernable effect of losing APU state at a save, but I'm sure there are notable ones that would be affected.
There are some expansion audio devices that are effectively "write only" though. Those at least might potentially have no non-audio consequences. ...and of course if you care about audio consequences, this whole idea isn't even on the table anyway.