Mesen Debugger - Feedback/Feature Requests? (2018 edition)

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems. See the NESdev wiki for more information.

Moderator: Moderators

Post Reply
Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by Sour »

lidnariq wrote:I think just showing bounds of when it's safe to do certain things? But on a scanline-to-scanline basis, most of these edges fall into two places- 1- the rightmost edge of active video, or 2- dot 320.
I could always add a checkbox to enable/disable a vertical region on the right that shows the cycles where it's safe to change the registers, etc.
tepples wrote:How would your separately tracked call stack treat the message dispatch idiom used in several of my own games?
This ends up calling RTS more times than JSR, so it would have the opposite effect of what MM1 does (e.g the callstack would empty itself and become out of sync). Your idea would prevent the stack from growing infinitely like it does in some cases, though. On top of that, I would need to try and detect when RTS is used as a way to perform a jump to another subroutine (e.g after pushing the address on the stack), and replace the top of the callstack with the new location after the RTS is executed. Overall it can probably be improved by a few things like this, though I don't think it'll ever be "perfect". I'll try to tinker some more with it when I get a chance.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by Fiskbit »

FCEUX's hex editor only commits when the full byte is written, but the difference there is that it's not possible to ever select just the low nybble, so that does make the interface problem harder. I'd be happy for an option to sacrifice low-nybble selection if it made the write committing make more sense; I usually want to select the high nybble, anyway, and get the low nybble by accident because I'm not used to it and it has a larger size (since it includes the space between bytes).

Regarding the call stack, the issue in Zelda is that it uses JSR to pass the address of jump tables to an indirect jump function (JSR to function, PLA PLA to get the table address, JMP ($xx) to use the entry from the table). This is done all over the code. Coming from FCEUX, I'm used to just looking at the stack for return addresses rather than having a call stack feature, but with how small Mesen's stack view is (only 3-4 bytes visible at a time vs 20+ in FCEUX), it doesn't seem that practical to use in this manner. Given that, any improvements you may make to the call stack would be very welcome.

I think I'm finally about out of feedback, at least for now. Thanks for all the work you've put into this!
User avatar
Yvar de Goffau
Posts: 11
Joined: Mon Dec 04, 2017 3:57 am
Location: Saint-Beat, Haute Garonne, France

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by Yvar de Goffau »

Nice debugger... a shame I can't get OSX to run mono!

One thing I couldn't find and that saved me the day during Gameboy development on BGB was the ability to break on reading unitinitialised variables...
Seeing the exact list of uninitialised variables greyed-out over the whole RAM memory map could come in handy to see how much memory is left. Also, one could cherry-pick variables to watch for cheats whilst one is playing, which would probably make TASers very happy!

Finally, in the long term it would be cool to merge such an emulator with an actual assembler, barring the need to manually compile code each time and allowing one to immediatly spot the differences a change in the source has on the actual game... but maybe this just-in-time assembly is still a bit to much too futuristic.
Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by Sour »

I've done some improvements to the callstack, it seems to be able to handle Zelda & MM1 properly now. Essentially, when it hits a RTS/RTI instruction, if the previous callstack info doesn't match the CPU's stack, it will try to find a matching callstack line & remove all the lines above it. If it can't find a matching line, it will add a new line to the callstack instead (and assume the code pushed an address to the stack and then called RTS to jump to it). Hopefully this takes care of most games - let me know if you still find issues with it.

I also changed the stack display to show more bytes (should show up to about 10 bytes now, instead of ~4) and fixed a bug that was causing it to display an extra byte in the list that wasn't supposed to be shown.

The build also has some fixes I mentioned earlier (e.g step out), some more improvements based on rainwarrior's feedback in the Mesen thread (customizable font, a few new buttons/options, a new pause icon for when the debugger is opened), and should also improveme emulation performance when debugging tools are opened by about 60-70% (on the windows build).

Build: download
Yvar de Goffau wrote: a shame I can't get OSX to run mono!
I wish it was easier for me to try and make Mesen work on OSX - it's fairly likely it could be made to work without that much effort, unfortunately I don't own a Mac, and it sounds like trying to make OSX work in a VM would be a bit of a pain (plus I think you technically need to buy a Mac to have a license to use OSX?).

There is a way to highlight uninitialized variables at the moment (but it requires that the debugger be attached from the ROM's power on), but no way to break on them. Uninitialized memory is definitely one of the most common issues with homebrew though, so it might be worth trying to improve this a bit.
Cherry-picking variables to watch them can technically be done with the watch window, although it might be a little less "advanced" in some aspects to FCEUX's memory viewer (if that's what it was called).

It's already technically possible to edit code in PRG ROM with the built-in assembler, but I understand what you mean. What you're asking for would be essentially the same "Edit-and-Continue" in Visual Studio... and that feature doesn't work properly half the time in my experience, so I'm pretty confident it would be pretty hard/impossible to get it to work properly in this case :p
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by tokumaru »

Sour wrote:it sounds like trying to make OSX work in a VM would be a bit of a pain
When I tried it a year or so ago (some coworkers were using a few Mac-only programs and I don't own a Mac) it was fairly easy, pretty much the same as setting up a Windows VM. I don't remember if the image I used was modified in any way, though. Either way this is probably illegal, I don't think you're allowed to run Mac OS on anything but a Mac.
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by rainwarrior »

It's a violation of Apple's license to use their OS X software to run it on anything but their official products. Doing so is not necessarily illegal, though. Overriding antitrust/etc. laws might be in effect depending on where you live. (Practically speaking, though, it won't really matter either way, because this part of the EULA isn't actually enforced in any meaningful way.)

I've run a few versions of OS X in VirtualBox, and they work well enough. Even was compiling code with XCode on one of them. Haven't tried more recent versions of the OS but there weren't really any notable problems except the usual low performance of a VM.
Oziphantom
Posts: 1565
Joined: Tue Feb 07, 2017 2:03 am

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by Oziphantom »

To get OSX into a VM you usually need to modify and tweak the OSX installation. It helps if you have Hackentosh compatible hardware, as Macs have very limited driver support. So it can be as easy as using a Hackentosh modded OSX image, or it can be pain. YMWV
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by tokumaru »

Yeah, I used a Hackintosh image and VirtualBox, it worked fine.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by Fiskbit »

In the latest version, I'm getting the following exception when opening the debugger with ctrl+D after starting a ROM:

System.ArgumentOutOfRangeException: Length cannot be less than zero.
Parameter name: length
at System.String.Substring(Int32 startIndex, Int32 length)
at Mesen.GUI.Debugger.ctrlConsoleStatus.UpdateStack(UInt16 stackPointer)
at Mesen.GUI.Debugger.ctrlConsoleStatus.UpdateStatus(DebugState& state)
at Mesen.GUI.Debugger.frmDebugger.UpdateDebugger(Boolean updateActiveAddress, Boolean bringToFront)
at Mesen.GUI.Debugger.frmDebugger.<>c__DisplayClass22_0.<_notifListener_OnNotification>b__1()
at System.Windows.Forms.Control.InvokeMarshaledCallbackHelper(Object obj)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Windows.Forms.Control.InvokeMarshaledCallback(ThreadMethodEntry tme)
at System.Windows.Forms.Control.InvokeMarshaledCallbacks()


Aside from that, after using this more extensively over the last week, I've got a little bit more feedback:

- One can copy code from the debugger to paste into a text editor, but there seems to be no way to include any addresses, which is a big problem for branches/jumps/calls. I've been having to fall back on FCEUX when doing this.
- When setting a breakpoint, it would be much more convenient if the textbox for 'Specific' address were selected by default so one could just type an address and hit enter, rather than having to first click in the box.
- Using ctrl or shift with the mouse to select multiple addresses in the breakpoints section doesn't behave properly; rather than highlighting multiple entries, it toggles the current breakpoint. Shift with the arrow keys works correctly, however.
- The "Bring debugger to front on break" option unexpectedly applies to loading saved states.
Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by Sour »

RE: OSX, I guess I'll have to take a proper look eventually. I am running a Ryzen CPU at the moment, but it looks like some other people have had success running OSX in a VM with one. For the moment I'd rather focus on adding missing features/fixing bugs rather than add support for another platform, though (since every additional platform adds an overhead in terms of testing & releasing new versions, etc.)
Fiskbit wrote:I've got a little bit more feedback:
Thanks for the feedback!

A few fixes/changes:
-The crash you saw should be fixed - it occurred when the stack pointer was set to $FF
-I added options to configure the copy's behavior (whether or not to copy addresses and/or byte code)
-Breakpoint window now focuses on the address field when it's opened
-Ctrl/Shift clicking on the breakpoint list should be working normally now

I can't seem to reproduce the save state issue you are getting, though.
I've tried loading savestates in a few games and the debugger never comes to the front. Can you try saving a state and loading it right away to see if it causes the behavior, too? What "Break on..." options are enabled? Does the debugger actually break, or does it just come to front and the emulation keeps on running?

Build: download
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by Fiskbit »

The saved state behavior seems to only happen when the emulator is paused. I have the emulator paused (with Escape) and press an F key to load a state, and the debugger comes to the front if the the "Bring debugger to front on break" option is selected. Apologies for the incomplete report before; I'm frequently paused when using saved states for development and didn't realize the behavior differed when not paused.

The crash issue is fixed, the copy feature seems to work great, and I'll give the rest of it a try another night. Thanks for the update!


Edit:
I've encountered a few problems with the March 6 version:

- The callstack seems much improved, but it doesn't correctly handle Step Back in Zelda.
- Step Back seems to not be working properly and I'm not sure exactly what's going on. In Zelda, if I set a write breakpoint on $EC (next_screen) and walk off the edge of a screen, the breakpoint correctly triggers at Bank5_B53A. In order to change the value being written, I use Step Back and Step Into to get to the start of the instruction and then set A to my desired value. In this version, instead of taking me to Bank5_B538, Step Back (almost always) takes me to Bank5_83E6. The PPU state is 3 cycles earlier on the same scanline, but the code it's at is actually later in the same frame. This reproduces fairly consistently for me, but I had one instance where it worked properly and I'm not sure what I did differently that time.
- When opening the debugger, the checkboxes for inactive breakpoints aren't drawn.
Bananmos
Posts: 552
Joined: Wed Mar 09, 2005 9:08 am
Contact:

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by Bananmos »

So, I got around to trying out my hacked version of Years Behind debugwrite-hack (i.e. replacing the scroll register writes with setting/clearing the monochrome bit) on my PAL NES.

Like I remembered, the timing is about as vertical as can be - perhaps being just *slightly* off for one or two playback rates. Here's a video capture of the DMC-enabled tunes: https://drive.google.com/open?id=1JVvSY ... KsBdD9SCcr

The purple vertical bars bounces around in a chaotic way and gets a bit wobbly, so it's not obvious at first. But doing a closer comparison in an emulator clearly shows way less skew than what Mesen - and FCEUX/Nintendulator - end up with.

So yeah, it does look like our understanding of the PAL version of the DMC DMA is incomplete, and no emulator seems to emulate PAL DMC DMA timings correctly. Given that the PAL DMC fixed the hardware bug causing incorrect controller reads, it's not unlikely that the rule for how many cycles to pause could be very different from the NTSC version.

Not sure what would be the best way to get the true values. But re-visiting the DMA tests that Blargg/cpow/Disch have done is probably a good starting point...
Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by Sour »

Some more fixes/additions:
-Debugger shortcuts can now be customized in Options->Configure Shortcut Keys (but they are limited to keyboard shortcuts, gamepads will not work). This includes a "Toggle Break/Continue" shortcut to mimic the "Pause/Play" behavior on the main window - both are mapped to Esc by default. If there's any toggle/action that doesn't have a shortcut binding let me know and I can add one. Hopefully this takes care of a number of usability issues with the debugger.

-Added a "byte editing mode" to the hex editor that will only "commit" a change once both nibbles are written (enabling this disables the ability to select the lower nibble by clicking)
-Fixed a bug with the step back code - when stepping back, the emulator rewinds X number of frames and plays them back, pausing when it reaches the previous statement. There was a bug that caused all buttons to be considered as unpressed when replaying the very last frame before pausing, which could cause the execution to diverge and end up breaking on the wrong line.
-Loading a state while the debugger is paused will no longer bring the debugger to the front when "bring to front on break" is enabled

About step back vs the call stack, currently the rewinder has no knowledge of the call stack - so while it rewinds, it affects the callstack like if it was executing normally. I can probably fix it, but it'll take some effort, I'll keep it in my notes.

For the inactive breakpoints not showing up properly, could you post a screenshot of what you mean? I haven't been able to reproduce this on my end (and can't really think of a reason why it might happen in general)

Build: download
Bananmos wrote:[...]
The PAL NES in general hasn't really been tested all that much, compared to the NTSC version. I'd help if I could, but I'm no 6502 expert, don't have a custom board/powerpak/everdrive, and don't have a PAL NES, either :p
It would definitely be interesting to figure out what's causing the discrepancy between emulators & the actual PAL NES, though.
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by rainwarrior »

Possible bug report:

A VRC6 (mapper 24) ROM I made a while back:
http://rainwarrior.ca/music/sundried.nes

The PPU viewer says "horizontal" mirroring but is incorrectly showing the result of vertical mirroring. (This was the first ROM I ever made, but it doesn't fail on any other emulator I know of.)

Edit: apparently the fault was my ROM, but it was not exposed by several emulators I had tried it with due to obscurity of implementation. See below.
ROM has been updated, used to initialize $B003 with $04, now uses $24.



BTW being able to select font for the disassembly is fantastic. I can easily see twice as many lines in there now.
Last edited by rainwarrior on Sat Mar 10, 2018 9:29 pm, edited 2 times in total.
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: Mesen Debugger - Feedback/Feature Requests? (2018 editi

Post by lidnariq »

That's partially an error in early VRC6 documentation. You have to write $24, not $04, to register $B003.

(The layout in the PPU viewer and the memory map in the debugger agree on vertical mirroring; I don't know why the "Mirroring Type" text disagrees)
Post Reply