Using VS Code as an IDE for debugging assembly with source

Discussion of hardware and software development for Super NES and Super Famicom.

Moderator: Moderators

Forum rules
  • For making cartridges of your Super NES games, see Reproduction.
CypherSignal
Posts: 34
Joined: Sun Jul 22, 2018 2:36 pm

Using VS Code as an IDE for debugging assembly with source

Post by CypherSignal » Mon Nov 19, 2018 8:22 am

[note that this topic is a bit of a fork from some previous discussion in the "bsnes-plus" thread here]

One thing I'm working on - in fits and starts - is trying to improve the debugging experience of SNES homebrew and patch development, especially by leveraging Visual Studio Code as an IDE of sorts, to provide an experience similar to debugging Python script or compiled C code. For example, being able to drop breakpoints in source ASM files, and step through the program counter and see the PC reflected in the source ASM, as demonstrated here:

https://www.youtube.com/watch?v=hkuMV-1LLmI

Status of this project as of Dec 16:
  • v9.8 of wla-dx can now output the address-to-line data, and other source file information, required for mapping ROM addresses to file and line numbers. Both the spec for WLA's symbols, as well as information on how to generate the data in question, is described here.
  • Head of Asar (point release TBD) can also output the same data.
  • I'm working on local modifications to bsnes-plus to load in and process the symbols and source files, and accept many interactions over VS Code's Debug Adapter Protocol. WIP material is available here. This still has a lot of little things I need to address and fix. e.g. performance issues, code quality, and some occasional issues when using the emulator as a debugger (esp. with breakpoints). This is still a ways away from being prepped as a PR to Revenant/devinacker's bsnes-plus but I'm not looking to, like, fully implement the DAP before that point. I just want to get to a point where it's possible to comfortably debug 65816 ROM data.
  • I'm working on an extension for VS Code to serve as a general tool for debugging external applications in VS Code. I have a repo for it, and the only thing it does right now is just kick an executable of your choice and establish a debug adapter connection with that exe. It is currently available on the VS Code marketplace, here: https://marketplace.visualstudio.com/it ... -app-debug. There are a lot of wishlist items I have on this extension that I'd like beyond just establishing that connection, though - for example, I think it should be possible to even have the extension create multiple views in VS Code for things memory watches from the external application.
Last edited by CypherSignal on Sat Aug 24, 2019 6:42 am, edited 8 times in total.

CypherSignal
Posts: 34
Joined: Sun Jul 22, 2018 2:36 pm

Re: Using VS Code as an IDE for debugging assembly with sour

Post by CypherSignal » Mon Nov 19, 2018 8:46 am

One particular thing I wanted to create this topic for - besides promotion of this project - was also to ask a longer-form question from, perhaps, other folks here that are more experienced in patch development.

This is an issue I ran into while testing the Asar implementation of the addr-to-line data against a patch, specifically, which you can see demonstrated here (and I'll be referring back to for context). The assembly listing here is from dotsarecool's SMWPracticeCart.

The subroutine every_frame, is explicitly specified in the ASM to be located at PC 0x978000 (!_F is defined in patch.asm as 0x800000). In the symbol file, this lines up nicely, without a problem. The following is from the output symbols:

Code: Select all

[labels]
97:8000 every_frame

[source files]
000b 444edb08 src/every_frame.asm

[addr-to-line mapping]
97:8000 000b:00000005
97:8001 000b:00000006
97:8003 000b:00000007
97:8006 000b:00000008
97:8009 000b:00000009
97:800c 000b:0000000b
97:800f 000b:0000000c
97:8011 000b:0000000d
97:8015 000b:0000000e
97:8016 000b:0000000f
So, no problem. 0x978000 maps to every_frame.asm with instructions at line 5, 6, 7, and so on.

However, there's another case where the SMWPracticeCart patches existing, non-zero, code: hijacks.asm. In this case, it writes out new subroutine jumps starting at PC 0x809510, and this is reflected in the symbols file:

Code: Select all

[labels]
80:9515 every_frame_hijack

[source files]
0000 ef8b5108 src/hijacks.asm

[addr-to-line mapping]
80:9510 0000:00000016
80:9514 0000:00000017
80:9515 0000:00000019
80:9518 0000:0000001a
80:951c 0000:0000001b
80:951d 0000:0000001d
80:9521 0000:0000001e
But, the original SMW ROM actually has the PC go not to address 0x809510, but rather 0x009510, due to fastrom/slowrom memory mapping and shadowing. The addr-to-line mapping doesn't have information to map 0x009510 to any source line, so the address doesn't resolve, even though that area in memory does have source line data to map to. Of course, setting !_F from 0x800000 to 0x000000 fixes the problem here: hijack's addr-to-line and label data points to 00:9510, and the mapping matches how the program is executing.

Theoretically, it should be possible to modify how the PC or other addresses are interpreted to try and guess at intended mappings (either when trying to get source line locations from addresses, or address from source line locations) but I can't help but imagine that that will come out the other end looking gnarlier than I'd like - it doesn't seem like it'd be possible to just mask all addresses by 0x7FFFFF and call it a day.

So, I want to try and get a read as to whether this is a "legitimate" issue to be concerned about: If a developer writes some code that is mapped to one location of ROM but executed at a different address in reality, due to shadowed memory banks, should they understand that they're doing something non-standard, and expect the symbols to not line up? Should the emulator attempt to bridge that gap automatically? Should it be recommended to just not have the fastrom offset be applied in a case like this? Is this even a common enough thing that it's worth being concerned about, i.e. is this only an issue when setting up entry points on ROM patches?

tepples
Posts: 22277
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Using VS Code as an IDE for debugging assembly with sour

Post by tepples » Mon Nov 19, 2018 9:54 am

Would it be possible to use some sort of manifest to map each shadowed address to its canonical address? I know Mega Man X, for example, relies on shadowing.

Nicole
Posts: 218
Joined: Sun Mar 27, 2016 7:56 pm

Re: Using VS Code as an IDE for debugging assembly with sour

Post by Nicole » Mon Nov 19, 2018 10:39 am

CypherSignal wrote:So, I want to try and get a read as to whether this is a "legitimate" issue to be concerned about: If a developer writes some code that is mapped to one location of ROM but executed at a different address in reality, due to shadowed memory banks, should they understand that they're doing something non-standard, and expect the symbols to not line up? Should the emulator attempt to bridge that gap automatically? Should it be recommended to just not have the fastrom offset be applied in a case like this? Is this even a common enough thing that it's worth being concerned about, i.e. is this only an issue when setting up entry points on ROM patches?
Every game that executes out of banks $80-ff will run into this, because reset/NMI/IRQ vectors always jump to bank $00.

nocash
Posts: 1316
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Re: Using VS Code as an IDE for debugging assembly with sour

Post by nocash » Mon Nov 19, 2018 10:47 am

CypherSignal wrote:it doesn't seem like it'd be possible to just mask all addresses by 0x7FFFFF and call it a day.
For most cartridges you could probably do that. Just give it a try.
And best keep some option in your source code (or an option in user interface) for disabling the feature if it doesn't work.
There are some special carts (with coprocessors etc) where it won't work, though you could detect such carts, and auto-disable the mirroring feature for them.

The main reason for using bank 00h instead of 80h is probably that exception vectors (reset/irq/nmi/etc) always point to bank 00h.
Small irq/nmi handlers might execute right in bank 00h, and larger handlers should manually far-jump to fast rom in bank 80h.
And don't forget hirom's also having exception vectors in bank 00h mirrored to bank 40h and C0h.

For fast rom source code, it might be best/easiest be to use bank 80h and up for the whole program (when doing it that way, exception vectors would be defined as being in bank 80h rather than bank 00h).
But people might do it this way, or another way. And, yeah, maybe some homebrew patches won't care about anything at all : )

tepples
Posts: 22277
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Using VS Code as an IDE for debugging assembly with sour

Post by tepples » Mon Nov 19, 2018 11:05 am

I imagine that treating $000000-$7DFFFF and $800000-$FDFFFF the same should work for the vast majority of LoROM (mode $20/$30) and HiROM (mode $21/$31) carts. It won't work for the uncommon memory maps of ExHiROM (mode $25/$35, Tales of Phantasia) and coprocessor carts (as nocash mentioned), and there will be issues for battery RAM and $7E0000-$7FFFFF.

HiROM introduces another wrinkle that reset, NMI, and IRQ handlers will end up executed from $008000-$00FFFF as a mirror of $C08000-$C0FFFF.

CypherSignal
Posts: 34
Joined: Sun Jul 22, 2018 2:36 pm

Re: Using VS Code as an IDE for debugging assembly with sour

Post by CypherSignal » Mon Nov 19, 2018 11:22 am

Nicole wrote:Every game that executes out of banks $80-ff will run into this, because reset/NMI/IRQ vectors always jump to bank $00.
Oh. Huh. Okay, this is definitely going on the feature list, then. Thanks for the info!
tepples wrote:Would it be possible to use some sort of manifest to map each shadowed address to its canonical address? I know Mega Man X, for example, relies on shadowing.
Well, digging a little bit more, it looks like (of course) bsnes has a bunch of internal support for memory mapping already set up, including definitions across a slew of different (all of the officially released?) chips: https://github.com/devinacker/bsnes-plu ... tridge.hpp

It'll take some investigation to see how/if I can piggyback off of that information, but that'll be my first check.

Edit: Maybe the existing code for breakpoint tests across mirrored memory would be an even better starting point :roll:

User avatar
Banshaku
Posts: 2396
Joined: Tue Jun 24, 2008 8:38 pm
Location: Japan
Contact:

Re: Using VS Code as an IDE for debugging assembly with sour

Post by Banshaku » Mon Nov 19, 2018 6:14 pm

I'm not working on any snes project yet but any tools that will help remote debug from vs code will be greatly appreciated. Once it is becoming more concrete, I will be more than happy to try it.

Would love to have the same thing for ca65/cc65 too sometime ^^;;

hibber22
Posts: 7
Joined: Tue Nov 20, 2018 9:23 pm

Re: Using VS Code as an IDE for debugging assembly with sour

Post by hibber22 » Tue Nov 20, 2018 9:25 pm

Can work on assemblies? I use the WDC tool chain for the c compiler.

CypherSignal
Posts: 34
Joined: Sun Jul 22, 2018 2:36 pm

Re: Using VS Code as an IDE for debugging assembly with sour

Post by CypherSignal » Wed Nov 21, 2018 6:18 am

Banshaku wrote:Would love to have the same thing for ca65/cc65 too sometime ^^;;
hibber22 wrote:Can work on assemblies? I use the WDC tool chain for the c compiler.
Having asar and wla-dx export the required data are the only things on my todo-list right now, just because those seemed to be the two most popular and actively-maintained assemblers out there.

As well, it'd be great to have this for a C-centric toolchain, but every time I look at the state of C toolchains for SNES I instantly find myself morose and miserable :P

At the risk of being that guy saying "Hey if you have a problem with this project, submit a PR!", I certainly won't have to be a bottleneck in getting that stuff up and running, in the case of someone wanting to stand up the support required to export WLA-style symbols in ca65 (or cc65). The only things that should be necessary will be having a tool that exports a .sym file with the appropriate file spec, which would be the to-do on whatever compiler/assembler toolchain you've got; and an emulator that reads in that .sym file (with support for the Debug Adapter Protocol) which I'm working on.

CypherSignal
Posts: 34
Joined: Sun Jul 22, 2018 2:36 pm

Re: Using VS Code as an IDE for debugging assembly with sour

Post by CypherSignal » Fri Nov 23, 2018 8:29 am

Well, it's pretty brute-force (e.g. address resolutions to labels and source lines all, each and every time, generate a list of all possible mirror addresses across the CPU bus) but this seems to work pretty well: https://i.imgur.com/brunZOZ.png.

Before, the callstack could not resolve the appropriate label for address 0x00951b because every_frame_hijack was supposed to be at 0x80951b, and, similarly, the source file/line number for 0x00951b could not have been determined, but now both are intact.

Even compiling the emulator in debug - though admittedly on a decent desktop PC - I can't glean an apparent performance loss when stepping through code due to this change given the afore-mentioned brute-force nature of it. In any case, there's obvious low-hanging fruit like caching potential mirrors, and that sort of thing, which could be done that would collapse any noted performance concerns.

In any case, I think cracking this nut may have been my only concern with the WIP stuff I had for Asar (even though technically that was present for the wla impl, too :shock: ) so next task will be polishing up the PR for that.

CypherSignal
Posts: 34
Joined: Sun Jul 22, 2018 2:36 pm

Re: Using VS Code as an IDE for debugging assembly with sour

Post by CypherSignal » Mon Nov 26, 2018 4:32 pm

Just a small update (OP has been updated with current events). Prepped the PR for Asar, and decided to scrub the few nicks off of the VS Code extension so that others can use it.

So, if you're feeling intrepid and you do want to try this all out against whatever stuff you may already have:
  • Fetch the respective Asar or wla-dx repo above.
  • Build the assembler, point your build scripts to it, and rebuild your ROM to generate a companion .sym file.
  • (if you use wla-dx, note the extra instructions you need to add to your ROM build script to generate symbols)
  • Fetch my vscode-stackframes branch of bsnes-plus and build it.
  • Fetch the .vsix file from the vscode-retrorom-debug repo and install that into VS Code.
  • Set the emulator path in VS Code's settings to the output of bsnes above, and create a standard "RetroROM Debug" debug configuration.
...and I think that's all you'd need to head off to the races.

qwertymodo
Posts: 775
Joined: Mon Jul 02, 2012 7:46 am

Re: Using VS Code as an IDE for debugging assembly with sour

Post by qwertymodo » Tue Nov 27, 2018 9:15 pm

Man, if we could get this and the disassembler enhancements here merged upstream, we'd have a seriously powerful debug environment.

Revenant
Posts: 442
Joined: Sat Apr 25, 2015 1:47 pm
Location: FL

Re: Using VS Code as an IDE for debugging assembly with sour

Post by Revenant » Wed Nov 28, 2018 12:05 pm

qwertymodo wrote:and the disassembler enhancements here merged upstream
it's happening

CypherSignal
Posts: 34
Joined: Sun Jul 22, 2018 2:36 pm

Re: Using VS Code as an IDE for debugging assembly with sour

Post by CypherSignal » Wed Nov 28, 2018 12:59 pm

Revenant wrote:
qwertymodo wrote:and the disassembler enhancements here merged upstream
it's happening
Oh, awesome! I'll definitely be keeping an eye on when you submit that into your trunk, because that'll be an interesting merge on my part :lol:

Ah, I just noticed that the 'newdebugger' branch is based off of the qt5 branch. Think that'll all be done in one swoop?

Post Reply