Mesen - NES Emulator

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Mesen - NES Emulator

Post by Sour »

For me personally at least, the main benefit of keeping a power-on movie as stateless is that the movie can be expected to work even if I have to break save state compatibility for any reason (and in fact, if they didn't, I'd have to re-record my 240+ test rom movies every time I broke save state compatibility!). If I force all movies to have a save state, it would make breaking save state compatibility even more of a big deal than it may already be. FYI, all movies from power on have no save state, including ones with save RAM - instead the save RAM's data is included into the movie file. The only exception here is the history viewer because I can't know what the save RAM's original state was (well, I mean, I could probably just keep a backup of it)

Here's what I have for the watch at the moment:
watch.png
Letter with no number = always assume 1 byte values
Technically, anything above 4 bytes won't work because the evaluator returns a 4-byte signed int to the UI as its result. I imagine NES games don't often handle values over 32 bits anyway (and increasing it to 64-bits would conflict with some implementation details of the expressions that are hidden away by the 32-bit limit)

Decimal display is the same as before: 4-byte signed (so it's the same as prefixing S4 everywhere)
Hex is equivalent to H1
Binary is equivalent to B1

I think these are sane defaults?
Hex/Binary/Unsigned are automatically extended to fit.
Not sure how to best handle signed values if they are out of bounds. e.g What should "S1:$4FFF" output?
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Mesen - NES Emulator

Post by rainwarrior »

Sour wrote:For me personally at least, the main benefit of keeping a power-on movie as stateless is that the movie can be expected to work even if I have to break save state compatibility for any reason
Ah, well that's a practical reason.

I don't know how you handle your savestates, but if that's off the table, I'd at least suggest that a movie contain info about the version of Mesen, and emulation settings that should affect things like mapper/RAM init. Like it'd be frustrating to have a movie that you can't get working and have no idea what combination of settings or what version of the emulator might be capable of running it otherwise. (I haven't asked about random mapper init, but it seems like it's in the same boat?)

IIRC FCEUX does a named data fields thing in FM2 so that it can add new information fields that old versions can ignore... I believe Nestopia savestates are similar, with information parcelled into packets with a 3 character name.


The types for watch look good! Does B/B1 always pad with zeroes to 8 bits? I notice you have a 9 bit value in the display there.

As for 64 bit types, I've never needed them for NES. I've definitely used 32 bit though.

I dunno what S1:$4FFF should do, maybe just give an error result? "invalid range" or something?
Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Mesen - NES Emulator

Post by Sour »

rainwarrior wrote:I'd at least suggest that a movie contain info about the version of Mesen, and emulation settings that should affect things like mapper/RAM init.
It does, Mesen movies are just zip containers pretty similar to Bizhawk's, it contains a "GameSettings.txt" file with (what should be) everything that can potentially affect emulation outcome, e.g:

Code: Select all

MesenVersion 0.9.7
MovieFormatVersion 1
GameFile Super Mario Bros. + Duck Hunt (U) [!].nes
SHA1 663DDEF71CC808C5382A3426EF6D03B5499A51C8
Region NTSC
ConsoleType NES
Controller1 StandardController
Controller2 Zapper
CpuClockRate 100
ExtraScanlinesBeforeNmi 0
ExtraScanlinesAfterNmi 0
InputPollScanline 241
DisablePpu2004Reads false
DisablePaletteRead false
DisableOamAddrBug false
UseNes101Hvc101Behavior false
EnableOamDecay false
DisablePpuReset false
ZapperDetectionRadius 0
RamPowerOnState 4294967295
Here, RamPowerOnState is actually set to "-1" in the code for random. Normally this would be either 0 or 255.
Does B/B1 always pad with zeroes to 8 bits? I notice you have a 9 bit value in the display there.
At the moment, yes, and it'll extend 1 bit at a time as needed. Is that what you'd expect?
I dunno what S1:$4FFF should do, maybe just give an error result? "invalid range" or something?
That's pretty much what I was thinking, too.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Mesen - NES Emulator

Post by rainwarrior »

Sour wrote:Mesen movies are just zip containers pretty similar to Bizhawk's, it contains a "GameSettings.txt" file with (what should be) everything that can potentially affect emulation outcome
Ah! That's neat. That's a good way to make it human readable/accessible too... as long as they know where to look. :)
Sour wrote:
Does B/B1 always pad with zeroes to 8 bits? I notice you have a 9 bit value in the display there.
At the moment, yes, and it'll extend 1 bit at a time as needed. Is that what you'd expect?
Yeah I think that makes sense. The thing I was more concerned about was whether 16 or 8 bits would be padded, because with binary and hex I think it's better that the output has a constant number of digits so you can read them in columns. But for a value that's 1 bit larger than expected, I dunno, just extending it a bit at a time is probably fine, I don't think that will come up very often anyway.

Though maybe 16 bits is a lot of binary digits to stare at... maybe a separator would be useful after every 4 or 8... (e.g. C++14 allows a quote as separator)

Then again, maybe a monospace font would help too, w.r.t. stationary columns.


Anyhow, that's getting into not very important details maybe. Just being able to break this window out for more flexible display and more vertical space is fantastic already. Thanks for that!
Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Mesen - NES Emulator

Post by Sour »

After I started updating the documentation for this, I ended up thinking a ", [format]" suffix would be consistent with the array display format that's already available, so I changed it to that instead. I think it improves readability, too.

I added a right-click menu to change (or clear) the format (which can be used on multiple entries at once), here's what it looks like at the moment:
watch.png
(It can now be used to control how arrays are displayed too)
rainwarrior wrote:Though maybe 16 bits is a lot of binary digits to stare at... maybe a separator would be useful after every 4 or 8...
Adding a comma every 4 bits would probably make sense (makes it easier to convert to/from hex that way, too)

Edit: This was committed and will be available on appveyor in a few minutes.
M_Tee
Posts: 430
Joined: Sat Mar 30, 2013 12:24 am
Contact:

Re: Mesen - NES Emulator

Post by M_Tee »

Conversation from the Grunio thread:
tokumaru wrote:
M_Tee wrote:The row of garbage pixels at the top is intentional, and is in all of the games we've made so far. We render our picture starting one row of pixels lower. I'll have to let Łukasz chime in on the specifics, but from what I recall, it makes sprite drawing more convenient in some way.
It's because the PPU displays sprites one scanline lower than the actual value set for their Y coordinate, so if you also lower the background by 1 scanline, you get a perfect match between sprite and background coordinates, sparing you the trouble of compensating for that 1 scanline in collision checks and the like. You can probably make it look less glitchy if you set the scroll to 239 instead of 255 though, because you'd at least get valid NT data instead of AT data interpreted as if it were NT data.
It'll definitely be covered up by overscan (even on a PAL system from what I've been told).
I was under the impression that all scanlines were visible in PAL.
I'm actually a little disappointed that Mesen, by default, displays the PAL overscan rows even when region is set to NTSC (as it makes it difficult to suggest the emulator for casual players who would be more reluctant to adjust the setting).
Heh, I'm the exact opposite! As a developer, I like to see the entire picture by default, and I hate that FCEUX crops 8 scanlines from the top and bottom by default!
lidnariq wrote:
tokumaru wrote:I was under the impression that all scanlines were visible in PAL.
PAL (both 2C07 and UA6538) displays 252x239 pixels. Left and right two pixels, and top scanline (where sprites couldn't appear) are all black instead.
Sour wrote:I wish I had a simple solution here - would be convenient if the header could be used to specify stuff like this (if only for homebrew's sake), but I imagine a lot of people would be opposed to changing this.
Would a usable compromise be to display 224 scanlines normally, but all 240 when any of the various Debug windows are open?
I had originally posted in that thread, but am moving it here:
I understand the desire to see all for some reasons while developing, but there's also a time in development when you need to start seeing what the player will see.

Another downside with displaying everything by default is that it will further confuse new developers about screen-safe area, a problem which already plagues developers even with NTSC area cropped. See the development histories of Super Tilt Bros. and The Mad Wizard for games which had to correct layout to compensate for screen-safe areas.

I actually think having a semi-opaque, check-screen-safe-area overlay available for developers to preview with would be very useful, that not only shows what for sure is and isn't displayed on PAL/NTSC, but also has the crosshairs (from the Nintendo internal screen-planning sheets) or something derived from Tepples' screensafe chart to indicate what *might* be hidden on an NTSC. Having a clearly labeled menu in the dropdown would attract new devs to try it and see what's happening.

Anyway, setting default to 8 or not, tied to region-setting wouldn't be a bad idea. It just seems that if neither PAL nor NTSC screens render that top row, there's no benefit in displaying it, especially by default. It really does seem like displaying area outside of what the hardware displays is a developer's feature, and developers are going to be more willing to dig through settings to see what they want/need, but when I'm trying to suggest the emulator to new or casual players, (especially those that might be willing to try the game, but not deep into emulation), it's not likely they're going to to be willing to or even know to change settings to see what the devs intended.

I like lidnariq's suggestion to have it as a debug feature. Maybe even a shortcut key to toggle between full area display and whatever the user has overscan set to, to quickly check when desired.
Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Mesen - NES Emulator

Post by Sour »

M_Tee wrote:I actually think having a semi-opaque, check-screen-safe-area overlay available for developers to preview with would be very useful, that not only shows what for sure is and isn't displayed on PAL/NTSC, but also has the crosshairs (from the Nintendo internal screen-planning sheets) or something derived from Tepples' screensafe chart to indicate what *might* be hidden on an NTSC. Having a clearly labeled menu in the dropdown would attract new devs to try it and see what's happening.
This is already available as a built-in Lua script (based on tepples' image on the wiki), but it's definitely not "easy" to discover - I wouldn't be surprised if I'm the only one who knows about it.
Maybe a shortcut key to switch between "overscan on", "overscan off" and "show safe regions overlay" would be useful for devs?
Forcing the whole screen to be shown automatically when debug tools are opened isn't really ideal (a user might have a reason to want to use the debug tools with the overscan active, etc.).

That being said, both Nestopia UE and FCEUX default to cutting the top and bottom 8 rows (though puNES shows everything by default), so maybe I should just revert to using that as a default as well - it's easier for 50 developers to set it to 0 than it is for hundreds of users to set it to a sane default.
Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Mesen - NES Emulator

Post by Sour »

rainwarrior wrote:[history viewer feedback]
I've fixed it so all movies will have a save state included when using randomized RAM.

Also added:
-The ability to export a portion of a segment (by specifying the start/end in hh:mm:ss)-
-A fairly basic "scale" option to resize the screen
-The ability to export a save state for the current position (e.g to avoid having to resume gameplay and then saving the state from the main window)

Still leaves a few things (which might have to wait until after the next proper release):
-Showing current/total frames while playing a movie (on the main window, or in the history viewer)
-Moving back and forth in the history viewer 1 frame at a time
-Loading up an existing movie into the history viewer to be able to browse through it without watching the whole thing
M_Tee
Posts: 430
Joined: Sat Mar 30, 2013 12:24 am
Contact:

Re: Mesen - NES Emulator

Post by M_Tee »

Sour wrote:This is already available as a built-in Lua script (based on tepples' image on the wiki), but it's definitely not "easy" to discover - I wouldn't be surprised if I'm the only one who knows about it.
Maybe a shortcut key to switch between "overscan on", "overscan off" and "show safe regions overlay" would be useful for devs?
Forcing the whole screen to be shown automatically when debug tools are opened isn't really ideal (a user might have a reason to want to use the debug tools with the overscan active, etc.).

That being said, both Nestopia UE and FCEUX default to cutting the top and bottom 8 rows (though puNES shows everything by default), so maybe I should just revert to using that as a default as well - it's easier for 50 developers to set it to 0 than it is for hundreds of users to set it to a sane default.
That sounds really good.

I'm eager to dig through Mesen and see what more it has to offer. You've chosen a good default palette and included solid alternatives. It's also great how the full palette is previewed when being changed, and I'm loving other features such as suggesting to continue previous game session when opening.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Mesen - NES Emulator

Post by Fiskbit »

One minor issue I noticed tonight is that stepping backward out of a function causes the next step backward to mark the instruction as the start of a function. There's also no way to remove this mark that I can find.

Thanks a ton for the PreviousOpPC option for conditions; that's been a big help for me already. Along these lines, I've also got a request for something similar for memory and maybe registers that I think could be useful. When reverse engineering, I sometimes find myself wondering if a load or store ever winds up mattering (perhaps to know whether it's unused and can be removed, to save bytes or cycles). For example, in Zelda, the collision detection code loads a value into Y that I currently suspect is never used outside the function, and stores horizontal and vertical position deltas into a couple temporaries that I didn't think were used. Auditing all loads from those temporaries throughout the ROM leads me to believe that the horizontal one isn't used, but I did find a vertical one that took quite some time to confirm from reverse engineering. I'd find it very useful if I could set a condition on a read breakpoint restricting it based on whether the variable was last written by a particular OpPC, to help me more quickly determine if something is used. Something similar for registers would be nice, though I'm less sure what it would look like since registers can't be breakpoint targets. This would be more useful still if it kept a list of influencing writes (one instruction stored and then another shifted, so the value depends on both of them, but another store later clears the list because the previous writes don't matter anymore), but I suspect that could have performance concerns.

(This is kind of along the lines of the advanced code/data logging I'd really like to have at some point that could understand the history of values for being able to correctly mark moved bytes as code/data based on how they get used later, or understand table boundaries, etc. Having a fairly comprehensive log would make it very easy to quickly answer these sorts of questions without even using the debugger. The debugger condition is smaller scoped, but because it'd be running all the time rather than just when logging, I worry about the performance impact of tracking more than just the last write.)
Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Mesen - NES Emulator

Post by Sour »

Fiskbit wrote:One minor issue I noticed tonight is that stepping backward out of a function causes the next step backward to mark the instruction as the start of a function. There's also no way to remove this mark that I can find
Thanks, this should be fixed. The function start data is stored in the CDL file, so you can clear them by resetting the CDL log (from the Tools->Code/Data Logger menu)

RE: more advanced tracking of reads/writes, I kind of feel like this would be a better match for Lua scripting than it would be for adding directly to the debugger itself. A script that monitors reads/writes and keeps tracks of whether or not the values are used should be pretty trivial to add, whereas adding this directly to the debugger would have at least some sort of performance impact (and would probably not even be as flexible as a Lua script, either).
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Mesen - NES Emulator

Post by Dwedit »

Hello, I have a little feature request for Mesen.
It is known that the NES has three possible CPU/PPU synchronization phases every power on, and I'd like to see an option to select which synchronization phase you get.

I'm asking because of Dragon Warrior 4. This game has a very tricky random number generator that is sensitive to the exact number of times it can execute a INC loop at the end of every frame. Even losing a single PPU dot worth of timing can throw off all the numbers drastically.

For background, the game runs this at the end of every frame: (14 CPU cycles per iteration)

Code: Select all

FF77:
 NOP
 NOP
 INC $12
 CMP $050C
 BEQ $FF77
This is also not the only operation involving memory $12, it is also used as part of a linear feedback shift register RNG.

Here is a test procedure you can use to test out Dragon Warrior 4's RNG:
* Delete all saved games
* Create a saved game in slot 1. Name: A, Male hero, Message speed Fast
* Save at the first House of Healing in the castle town.
* Power off.
* Power on and Hold Down+A from the copyright screen and file selection screen
* After "Welcome back, Ragnar!" appears, but before the second line of dialog appears, release A and continue to hold Down
* Continue to hold down, and note how the blue guy near you moves around.

Dragon Warrior 4's RNG is so tricky that every emulator seems to disagree on exactly what RNG values you will get. I have tested out many emulators, and they all gave different RNG output. The only match was between Mesen and Nintendulator, and that does not necessarily indicate that both are "correct" and the others are "wrong".
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Mesen - NES Emulator

Post by Sour »

I was on the vague impression that NTSC had 4 different power up alignments? (e.g: this thread).

That being said, there's a way to tweak (code-wise) the number of PPU cycles that run during the CPU's boot-up sequence which has impact on a number of things, maybe that would be sufficient to trigger the different RNG results in DQ4?

Otherwise, emulating the actual alignments requires emulating down to the master clock level, I think? That would probably cause a noticeable hit on performance, I'd imagine..
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Mesen - NES Emulator

Post by koitsu »

Dwedit wrote:Hello, I have a little feature request for Mesen.
It is known that the NES has three possible CPU/PPU synchronization phases every power on, and I'd like to see an option to select which synchronization phase you get.
To me, this sounds like you're asking for essentially a way to cheat -- or rather, "make consistent -- the RNG algorithm, probably related to things like this. Am I wrong?
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Mesen - NES Emulator

Post by Fiskbit »

Since there are 3 dots per CPU cycle, I think this is specifically about the alignment of whole dots and whole cycles (which dot a CPU cycle begins on), rather than which of 4 positions within a dot a CPU cycle begins on.


@Sour: I've been encountering some crashing on the newest couple builds of Mesen (0.9.7.92/93). In my current ROM, if I have the following two breakpoints enabled simultaneously, the emulator closes without any sort of error message.

CPU:-W- $0010
CPU:-W- $8000-FFFF where OpPC < $FFAC || OpPC > $FFBF

Another issue I found is that the add breakpoint option when right clicking on a tile in the Nametable Viewer tries to add a wrong kind of breakpoint (Palette RAM breakpoint type on address $1F).

Also, any chance you could add a breakpoint option to that Nametable Viewer right click menu for the attribute byte?

Finally, is there any way right now to view mapper register state? From past experience, I am guessing the issue I'm currently debugging is an errant write above $8000, which causes me to load the wrong bank later (MMC1), but being able to actually see around swap time that the mapper state isn't right would definitely help track this sort of thing down. Given that there are so many different mappers, though, I'd not be surprised if there were no way to get at the info.


Regarding LUA scripting for better logging, that hadn't crossed my mind and is a great idea. I'll look into that when I manage to free up some time from other projects!
Post Reply