It is currently Sun Dec 16, 2018 6:39 am

All times are UTC - 7 hours



Forum rules





Post new topic Reply to topic  [ 17 posts ]  Go to page 1, 2  Next
Author Message
PostPosted: Mon Nov 19, 2018 8:22 am 
Offline

Joined: Sun Jul 22, 2018 2:36 pm
Posts: 25
[note that this topic is a bit of a fork from some previous discussion in the "bsnes-plus" thread here]

One thing I'm working on - in fits and starts - is trying to improve the debugging experience of SNES homebrew and patch development, especially by leveraging Visual Studio Code as an IDE of sorts, to provide an experience similar to debugging Python script or compiled C code. For example, being able to drop breakpoints in source ASM files, and step through the program counter and see the PC reflected in the source ASM, as demonstrated here:

https://www.youtube.com/watch?v=hkuMV-1LLmI

Status of this project as of Nov 26:

  • Head of wla-dx (no point release has yet been made) can now output the address-to-line data, and other source file information, required for mapping ROM addresses to file and line numbers. Both the spec for WLA's symbols, as well as information on how to generate the data in question, is described here.

  • I'm working on local modifications to Asar to output a similar suite of information in its wla-style symbols data. WIP material is available here. I currently have a pending PR on Asar to incorporate this.

  • I'm working on local modifications to bsnes-plus to load in and process the symbols and source files, and accept many interactions over VS Code's Debug Adapter Protocol. WIP material is available here. This still has a lot of little things I need to address and fix. e.g. performance issues, code quality, debugging the emulator while debugging, and some occasional issues when using the emulator as a debugger (esp. with breakpoints). This is still a ways away from being prepped as a PR to Revenant/devinacker's bsnes-plus but I'm not looking to, like, fully implement the DAP before that point. I just want to get to a point where it's possible to comfortably debug 65816 ROM data.

  • I'm working on an extension for VS Code to serve as a general tool for debugging of ROMs in VS Code. It needs a lot of work, and I'll put it up on the VS Code marketplace when it is a bit more featureful. I have a repo for it, and the only thing it does right now is just kick an executable of your choice and establish a debug adapter connection with that exe. If you want to install it, you can grab the .vsix file in the root and manually install that as an extension to VS Code. There are a lot of wishlist items I have on this extension that I'd like beyond just establishing that connection, though - for example, I think it should be possible to even have the extension create multiple views in VS Code for things like VRAM or WRAM data.


Last edited by CypherSignal on Mon Nov 26, 2018 4:36 pm, edited 2 times in total.

Top
 Profile  
 
PostPosted: Mon Nov 19, 2018 8:46 am 
Offline

Joined: Sun Jul 22, 2018 2:36 pm
Posts: 25
One particular thing I wanted to create this topic for - besides promotion of this project - was also to ask a longer-form question from, perhaps, other folks here that are more experienced in patch development.

This is an issue I ran into while testing the Asar implementation of the addr-to-line data against a patch, specifically, which you can see demonstrated here (and I'll be referring back to for context). The assembly listing here is from dotsarecool's SMWPracticeCart.

The subroutine every_frame, is explicitly specified in the ASM to be located at PC 0x978000 (!_F is defined in patch.asm as 0x800000). In the symbol file, this lines up nicely, without a problem. The following is from the output symbols:

Code:
[labels]
97:8000 every_frame

[source files]
000b 444edb08 src/every_frame.asm

[addr-to-line mapping]
97:8000 000b:00000005
97:8001 000b:00000006
97:8003 000b:00000007
97:8006 000b:00000008
97:8009 000b:00000009
97:800c 000b:0000000b
97:800f 000b:0000000c
97:8011 000b:0000000d
97:8015 000b:0000000e
97:8016 000b:0000000f


So, no problem. 0x978000 maps to every_frame.asm with instructions at line 5, 6, 7, and so on.

However, there's another case where the SMWPracticeCart patches existing, non-zero, code: hijacks.asm. In this case, it writes out new subroutine jumps starting at PC 0x809510, and this is reflected in the symbols file:

Code:
[labels]
80:9515 every_frame_hijack

[source files]
0000 ef8b5108 src/hijacks.asm

[addr-to-line mapping]
80:9510 0000:00000016
80:9514 0000:00000017
80:9515 0000:00000019
80:9518 0000:0000001a
80:951c 0000:0000001b
80:951d 0000:0000001d
80:9521 0000:0000001e


But, the original SMW ROM actually has the PC go not to address 0x809510, but rather 0x009510, due to fastrom/slowrom memory mapping and shadowing. The addr-to-line mapping doesn't have information to map 0x009510 to any source line, so the address doesn't resolve, even though that area in memory does have source line data to map to. Of course, setting !_F from 0x800000 to 0x000000 fixes the problem here: hijack's addr-to-line and label data points to 00:9510, and the mapping matches how the program is executing.

Theoretically, it should be possible to modify how the PC or other addresses are interpreted to try and guess at intended mappings (either when trying to get source line locations from addresses, or address from source line locations) but I can't help but imagine that that will come out the other end looking gnarlier than I'd like - it doesn't seem like it'd be possible to just mask all addresses by 0x7FFFFF and call it a day.

So, I want to try and get a read as to whether this is a "legitimate" issue to be concerned about: If a developer writes some code that is mapped to one location of ROM but executed at a different address in reality, due to shadowed memory banks, should they understand that they're doing something non-standard, and expect the symbols to not line up? Should the emulator attempt to bridge that gap automatically? Should it be recommended to just not have the fastrom offset be applied in a case like this? Is this even a common enough thing that it's worth being concerned about, i.e. is this only an issue when setting up entry points on ROM patches?


Top
 Profile  
 
PostPosted: Mon Nov 19, 2018 9:54 am 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 20882
Location: NE Indiana, USA (NTSC)
Would it be possible to use some sort of manifest to map each shadowed address to its canonical address? I know Mega Man X, for example, relies on shadowing.


Top
 Profile  
 
PostPosted: Mon Nov 19, 2018 10:39 am 
Offline

Joined: Sun Mar 27, 2016 7:56 pm
Posts: 181
CypherSignal wrote:
So, I want to try and get a read as to whether this is a "legitimate" issue to be concerned about: If a developer writes some code that is mapped to one location of ROM but executed at a different address in reality, due to shadowed memory banks, should they understand that they're doing something non-standard, and expect the symbols to not line up? Should the emulator attempt to bridge that gap automatically? Should it be recommended to just not have the fastrom offset be applied in a case like this? Is this even a common enough thing that it's worth being concerned about, i.e. is this only an issue when setting up entry points on ROM patches?

Every game that executes out of banks $80-ff will run into this, because reset/NMI/IRQ vectors always jump to bank $00.


Top
 Profile  
 
PostPosted: Mon Nov 19, 2018 10:47 am 
Offline

Joined: Fri Feb 24, 2012 12:09 pm
Posts: 733
CypherSignal wrote:
it doesn't seem like it'd be possible to just mask all addresses by 0x7FFFFF and call it a day.

For most cartridges you could probably do that. Just give it a try.
And best keep some option in your source code (or an option in user interface) for disabling the feature if it doesn't work.
There are some special carts (with coprocessors etc) where it won't work, though you could detect such carts, and auto-disable the mirroring feature for them.

The main reason for using bank 00h instead of 80h is probably that exception vectors (reset/irq/nmi/etc) always point to bank 00h.
Small irq/nmi handlers might execute right in bank 00h, and larger handlers should manually far-jump to fast rom in bank 80h.
And don't forget hirom's also having exception vectors in bank 00h mirrored to bank 40h and C0h.

For fast rom source code, it might be best/easiest be to use bank 80h and up for the whole program (when doing it that way, exception vectors would be defined as being in bank 80h rather than bank 00h).
But people might do it this way, or another way. And, yeah, maybe some homebrew patches won't care about anything at all : )


Top
 Profile  
 
PostPosted: Mon Nov 19, 2018 11:05 am 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 20882
Location: NE Indiana, USA (NTSC)
I imagine that treating $000000-$7DFFFF and $800000-$FDFFFF the same should work for the vast majority of LoROM (mode $20/$30) and HiROM (mode $21/$31) carts. It won't work for the uncommon memory maps of ExHiROM (mode $25/$35, Tales of Phantasia) and coprocessor carts (as nocash mentioned), and there will be issues for battery RAM and $7E0000-$7FFFFF.

HiROM introduces another wrinkle that reset, NMI, and IRQ handlers will end up executed from $008000-$00FFFF as a mirror of $C08000-$C0FFFF.


Top
 Profile  
 
PostPosted: Mon Nov 19, 2018 11:22 am 
Offline

Joined: Sun Jul 22, 2018 2:36 pm
Posts: 25
Nicole wrote:
Every game that executes out of banks $80-ff will run into this, because reset/NMI/IRQ vectors always jump to bank $00.

Oh. Huh. Okay, this is definitely going on the feature list, then. Thanks for the info!

tepples wrote:
Would it be possible to use some sort of manifest to map each shadowed address to its canonical address? I know Mega Man X, for example, relies on shadowing.

Well, digging a little bit more, it looks like (of course) bsnes has a bunch of internal support for memory mapping already set up, including definitions across a slew of different (all of the officially released?) chips: https://github.com/devinacker/bsnes-plu ... tridge.hpp

It'll take some investigation to see how/if I can piggyback off of that information, but that'll be my first check.

Edit: Maybe the existing code for breakpoint tests across mirrored memory would be an even better starting point :roll:


Top
 Profile  
 
PostPosted: Mon Nov 19, 2018 6:14 pm 
Offline
User avatar

Joined: Tue Jun 24, 2008 8:38 pm
Posts: 2130
Location: Fukuoka, Japan
I'm not working on any snes project yet but any tools that will help remote debug from vs code will be greatly appreciated. Once it is becoming more concrete, I will be more than happy to try it.

Would love to have the same thing for ca65/cc65 too sometime ^^;;


Top
 Profile  
 
PostPosted: Tue Nov 20, 2018 9:25 pm 
Offline

Joined: Tue Nov 20, 2018 9:23 pm
Posts: 1
Can work on assemblies? I use the WDC tool chain for the c compiler.


Top
 Profile  
 
PostPosted: Wed Nov 21, 2018 6:18 am 
Offline

Joined: Sun Jul 22, 2018 2:36 pm
Posts: 25
Banshaku wrote:
Would love to have the same thing for ca65/cc65 too sometime ^^;;

hibber22 wrote:
Can work on assemblies? I use the WDC tool chain for the c compiler.

Having asar and wla-dx export the required data are the only things on my todo-list right now, just because those seemed to be the two most popular and actively-maintained assemblers out there.

As well, it'd be great to have this for a C-centric toolchain, but every time I look at the state of C toolchains for SNES I instantly find myself morose and miserable :P

At the risk of being that guy saying "Hey if you have a problem with this project, submit a PR!", I certainly won't have to be a bottleneck in getting that stuff up and running, in the case of someone wanting to stand up the support required to export WLA-style symbols in ca65 (or cc65). The only things that should be necessary will be having a tool that exports a .sym file with the appropriate file spec, which would be the to-do on whatever compiler/assembler toolchain you've got; and an emulator that reads in that .sym file (with support for the Debug Adapter Protocol) which I'm working on.


Top
 Profile  
 
PostPosted: Fri Nov 23, 2018 8:29 am 
Offline

Joined: Sun Jul 22, 2018 2:36 pm
Posts: 25
Well, it's pretty brute-force (e.g. address resolutions to labels and source lines all, each and every time, generate a list of all possible mirror addresses across the CPU bus) but this seems to work pretty well: https://i.imgur.com/brunZOZ.png.

Before, the callstack could not resolve the appropriate label for address 0x00951b because every_frame_hijack was supposed to be at 0x80951b, and, similarly, the source file/line number for 0x00951b could not have been determined, but now both are intact.

Even compiling the emulator in debug - though admittedly on a decent desktop PC - I can't glean an apparent performance loss when stepping through code due to this change given the afore-mentioned brute-force nature of it. In any case, there's obvious low-hanging fruit like caching potential mirrors, and that sort of thing, which could be done that would collapse any noted performance concerns.

In any case, I think cracking this nut may have been my only concern with the WIP stuff I had for Asar (even though technically that was present for the wla impl, too :shock: ) so next task will be polishing up the PR for that.


Top
 Profile  
 
PostPosted: Mon Nov 26, 2018 4:32 pm 
Offline

Joined: Sun Jul 22, 2018 2:36 pm
Posts: 25
Just a small update (OP has been updated with current events). Prepped the PR for Asar, and decided to scrub the few nicks off of the VS Code extension so that others can use it.

So, if you're feeling intrepid and you do want to try this all out against whatever stuff you may already have:

  • Fetch the respective Asar or wla-dx repo above.
  • Build the assembler, point your build scripts to it, and rebuild your ROM to generate a companion .sym file.
  • (if you use wla-dx, note the extra instructions you need to add to your ROM build script to generate symbols)
  • Fetch my vscode-stackframes branch of bsnes-plus and build it.
  • Fetch the .vsix file from the vscode-retrorom-debug repo and install that into VS Code.
  • Set the emulator path in VS Code's settings to the output of bsnes above, and create a standard "RetroROM Debug" debug configuration.

...and I think that's all you'd need to head off to the races.


Top
 Profile  
 
PostPosted: Tue Nov 27, 2018 9:15 pm 
Offline

Joined: Mon Jul 02, 2012 7:46 am
Posts: 771
Man, if we could get this and the disassembler enhancements here merged upstream, we'd have a seriously powerful debug environment.


Top
 Profile  
 
PostPosted: Wed Nov 28, 2018 12:05 pm 
Offline

Joined: Sat Apr 25, 2015 1:47 pm
Posts: 427
Location: FL
qwertymodo wrote:
and the disassembler enhancements here merged upstream

it's happening


Top
 Profile  
 
PostPosted: Wed Nov 28, 2018 12:59 pm 
Offline

Joined: Sun Jul 22, 2018 2:36 pm
Posts: 25
Revenant wrote:
qwertymodo wrote:
and the disassembler enhancements here merged upstream

it's happening


Oh, awesome! I'll definitely be keeping an eye on when you submit that into your trunk, because that'll be an interesting merge on my part :lol:

Ah, I just noticed that the 'newdebugger' branch is based off of the qt5 branch. Think that'll all be done in one swoop?


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 17 posts ]  Go to page 1, 2  Next

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group