bsnes-plus and xkas-plus (new debugger and assembler)

Discussion of hardware and software development for Super NES and Super Famicom.

Moderator: Moderators

Forum rules
  • For making cartridges of your Super NES games, see Reproduction.
byuu
Posts: 1543
Joined: Mon Mar 27, 2006 5:23 pm
Contact:

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by byuu » Fri Jun 22, 2018 11:44 pm

Updated emulation of S-DSP's ENDX state (fixes Magical Drop) [AWJ]
Sigh, that's not a correct fix. We've already confirmed Magical Drop locks up on real hardware from time to time.

Jonas' research into DSP register initialization was incomplete, and attempting a partial fix to run this game will just push the issue off everyone's radar instead of getting it supported properly.

Oh well, I guess I've been guilty of this in the past too, like my pre-dot renderer hack for Uniracers.

Sorry for yet another bummer post by me. The rest of the changes all look great though, great work!

Revenant
Posts: 440
Joined: Sat Apr 25, 2015 1:47 pm
Location: FL

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by Revenant » Sat Jun 23, 2018 12:14 am

I became aware of that a little while after the change was originally made upstream (a bit shy of a year ago), but I decided in this case I was willing to err slightly in the user's favor until we actually know what the correct behavior is. Consider it temporary, of course.

(hacks are hacks, I know...)

byuu
Posts: 1543
Joined: Mon Mar 27, 2006 5:23 pm
Contact:

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by byuu » Sat Jun 23, 2018 9:49 am

Yeah, and I can't really complain anymore with bsnes v107. I've got some game title matching to disable the new faster PPU/DSP cores on an as-needed basis.

I was just really hoping we'd get some interest in figuring out this behavior fully. It's a really fun game and deserves proper emulation.

lidnariq
Posts: 9020
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by lidnariq » Sat Jun 23, 2018 10:39 am

Revenant wrote:Added experimental 'snesmusic' plugin for SPC/SNSF file playback [Revenant]
Am I supposed to be able to just load a SPC file? If not, how can I play around with this? If so, what am I doing wrong such that it doesn't work?

Revenant
Posts: 440
Joined: Sat Apr 25, 2015 1:47 pm
Location: FL

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by Revenant » Sat Jun 23, 2018 12:05 pm

Yeah, loading any spc/snsf/minisnsf file should work as long as snesmusic.dll (or your OS' equivalent) is present:

Image

Otherwise, it might have a problem with whichever specific file(s)? you're trying to load? I can check it out if you're having issues.

lidnariq
Posts: 9020
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by lidnariq » Sat Jun 23, 2018 12:19 pm

Ahh, problems from not installing things. Have to put snesmusic.so in the system-wide library directories, or else run with LD_LIBRARY_PATH=/path/to/directory/containing/so/file
Thank you!

(Made a stupid shell script to let me continue to not install things.)

Revenant
Posts: 440
Joined: Sat Apr 25, 2015 1:47 pm
Location: FL

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by Revenant » Sat Jun 23, 2018 12:42 pm

Cool, glad it works :)

It's definitely a pretty rudimentary player; I originally made it for the completely silly reason of being able to debug some strange SPCs that I dumped from a Super Wild Card DX firmware a while ago. It's a good companion for the (equally rudimentary) sound viewer window though.

CypherSignal
Posts: 34
Joined: Sun Jul 22, 2018 2:36 pm

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by CypherSignal » Mon Jul 23, 2018 8:05 pm

I'm trying to get started on some from-scratch homebrew work, and was very quickly vexed by the relative unfriendliness of the debugging process when dealing with existing asm code, especially compared to, say, debugging a C++ project in Visual Studio.

I've started prodding at some parts of the toolchain so far, mostly just to get source-assembly files mapped to rom memory addresses in the disassembler, and wanted to gauge if there was any interest in this. So far, I've just got a small proof-of-concept here. (old disassembler is in the top left, and my local version showing source info is in the top right, plus a couple of the source asm files for one of the projects from Neviksti's Snes starter kit provided for reference). This consists of a couple small changes to wla-dx's linker to output the obj list info into the final symbol file (and generate the address-to-line map) and changes to bsnes to acknowledge that sym file and reformat the disassembler accordingly.

In terms of long-term goals, I'd like to going beyond in-emulator debugging, though. Ideally, I'd like to get to a point where one can have a VS Code debugger extension that talks to bsnes in a robust capacity. For example, from VS Code, launch bsnes as a debug target, drop a breakpoint in Code in an asm file (which bsnes is able to acknowledge as an exec-breakpoint on the corresponding address), and be able to present some debug information in Code when the emulation is paused. That may be a ways away, but I wanted to sort of read the room to see if adding such functionality to bsnes - or at least this debug-centric fork of bsnes - would be up for consideration.

Oziphantom
Posts: 785
Joined: Tue Feb 07, 2017 2:03 am

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by Oziphantom » Mon Jul 23, 2018 11:36 pm

So, this is really poor to debug when I compare it to the best debugger on the market ;) That is really not a fair comparison.

I guess when you have an external linker like WLA-DX you really don't know where stuff is compared to where you write it... that would really suck.

Having a fully integrated VSCode module would be awesome. However for a first step ( and yes I know I'm a broken record at this point BUT ) having a remote "monitor" would be a more useful addition. Then we can use the monitor to debug and test the code. Then the VS Code debugger can interface with the monitor keeping it neatly separated.

CypherSignal
Posts: 34
Joined: Sun Jul 22, 2018 2:36 pm

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by CypherSignal » Tue Jul 24, 2018 6:03 am

Oziphantom wrote:I guess when you have an external linker like WLA-DX you really don't know where stuff is compared to where you write it... that would really suck.
That's kind of touching on something I was wondering myself: is there a toolchain I'm not aware of where that isn't the case, or am I missing some part of the process people pursue normally when doing rom dev where this functionality wouldn't be desired right from the get-go? I mean, aside from "most rom development has just been hacking at existing rom files and there really isn't much of a homebrew scene" :P
Having a fully integrated VSCode module would be awesome. However for a first step ( and yes I know I'm a broken record at this point BUT ) having a remote "monitor" would be a more useful addition. Then we can use the monitor to debug and test the code. Then the VS Code debugger can interface with the monitor keeping it neatly separated.
So, based on my understanding of how such a debug extension (and corresponding suite) would exist, that is kind of how it should be architectured, anyway. There would have to be a module added in the emulator that operates on stdin to be able to respond to external actions and stdout to relay information about the emulator's current state, e.g. the c++ debug extension can talk with GDB through stdin/stdout, and afaik the python debug extension talks with python's existing pdb module in a similar capacity. The debug extension in Code is then used to be the adapter/interface between Code's host UI and the target in question.

I haven't dug into that part of it too much - don't want to put the cart before the horse too much - but the "module in bsnes" would probably turn into "Expand the console in Debugger to include a lot more information, add a field to allow for user input for various commands and interactions, and then pipe each of those through stdout and stdin respectively".

Oziphantom
Posts: 785
Joined: Tue Feb 07, 2017 2:03 am

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by Oziphantom » Tue Jul 24, 2018 6:39 am

Most of the good SNES tools are Win95. Back then we had X816 which doesn't have a linker, Geiger's Debugger ( which has now been superseded ) and a bunch of viewing tools. Before Geiger's we had the ZSNES debugger.

Using a linker assembler like WLA-DX will complicate things, I don't use WLA-DX ;)

STDIN/STDOUT I guess works, I'm more use to using sockets and a telnet interface, which is basically the same code wise, just you can
a.) attach mid run and not have to set it up at the start
b.) can be piped through to another machine/VM - which is handy for when you need to run some Linux VM to get bleeding edge version that fixes some bug in the emulator you need, and its not in the windows build yet ;)

Having
M
D
Bank
R
Break
Watch
Trace
>
and I guess something that lets us change if A/Index should be 8/16bit is the bare minimum
having A would also be handy, L, S super handy as well ;)

CypherSignal
Posts: 34
Joined: Sun Jul 22, 2018 2:36 pm

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by CypherSignal » Wed Aug 08, 2018 4:13 pm

Been working away at this, and finally got to a proof of doing some light debugging in VS Code, i.e. being able to pause and step from VS Code, display the current PC in a file, and also setting/clearing breakpoints before launching a session or during a session: https://www.youtube.com/watch?v=hkuMV-1LLmI

Some parts of it are still rough and needs some cleanup and all that sort of thing, but it's at the point of being functional. As mentioned before, the VS Code interface really is pretty simple once you get things going: it throws down json requests to bsnes via stdin, and bsnes sends json responses and events about the current state of the debugger back through stdout.

I've already got an issue/task filed on wla-dx to add the necessary addr-to-line mapping to the WLA symbol file, but haven't yet filed the PR for a few reasons, but I will get that going once I polish a few things up. I'll also take a look at asar to see how easy it'd be to add the same information to its wla-style symbol file output, since that's a popular tool.

Once I polish up the bsnes stuff as well, I'll also file a PR (or just a task) to devinacker to add this functionality to bsnes. So far, it's basically taking a bunch of work benny already did around symbol loading/mgmt, adding the functionality to communicate with vscode, adding a git submodule for another json project (though, Qt5 upgrade will deprecate that) and uplifting a handful of other systems accordingly.

Also, ofc, got to prep the extension itself to launch the bsnes executable from vscode and publish that somewhere. It's not a whole lot - we don't need a debug adapter because bsnes supports the debug protocol directly - but it's just a little bit.

Oziphantom
Posts: 785
Joined: Tue Feb 07, 2017 2:03 am

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by Oziphantom » Thu Aug 09, 2018 3:36 am

Do you have a good source that explains how to make VSCode plugins like this. Everything either just does simple stuff or assumes you are doing some JS stuff and know all this 80 things on how it works...

CypherSignal
Posts: 34
Joined: Sun Jul 22, 2018 2:36 pm

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by CypherSignal » Thu Aug 09, 2018 5:17 am

Yeah, tell me about it... Ironically, despite the frustrations in trying to shake down the various pages and documentation on this, furiously asking "What do you want from me?!?! :x :x " the answer ended up being rather simple - but obfuscated because of much of the intended utility all of the debug adapter extension shenanigans.

So, first off, there is the (newly created) site for outlining the debug protocol that msft put together. It still focuses a lot on "So you want to make a debug adapter extension or something, eh?" and doesn't discuss the raw mechanics. However, those raw mechanics just aren't that much:
and that's...pretty much it. With some additional notes:

For one, in this case, I'm only launching the process because it implements the debug protocol directly - there isn't an adapter that has to reinterpret things to talk to another existing debugger (see: gdb, pdb, etc) or run in a node.js env. This is defined in what is currently a very light extension, that basically only Contributes a debugger that defines the path to bsnes to launch, and a couple configuration attributes that will be tacked onto the Launch request (e.g. "program" to define what ROM to load up, or "stopAtEntry" to have the game start in a frozen state) which the bsnes debugger has to acknowledge.

Two, the specification for requests, responses, and events, are available here (this was also available historically as, at the very least, this TypeScript definition file in the VSCode github repo). As mentioned, those are JSON objects in HTTP messages, so the format of everything will be, with cr/lf visible, along the lines of:

Code: Select all

Content-Length: 331\r\n
\r\n
{"command":"initialize", "arguments": {"clientID":"vscode", "clientName":"Visual Studio Code", "adapterID":"bsnes debug", "pathFormat":"path", "linesStartAt1":true, "columnsStartAt1":true, "supportsVariableType":true, "supportsVariablePaging":true, "supportsRunInTerminalRequest":true, "locale":"en-us"}, "type":"request", "seq":1}
And the responses back have to be the same way, including the Content-Length header, and including the "\r\n\r\n", no more, no less (see "responses and events are sent back..." above). I only mention this part because I was tearing my hair out for a couple of hours trying to understand why my initial implementation wasn't working, and it turned out I had stdout in text mode, not binary, so my "\r\n\r\n" was turning into "\r\r\n\r\r\n" which was causing a silent failure.

Getting back to the original question, though, this implementation of a debug protocol written in C to do debugging of lua is what you're probably looking for in terms of raw code that peels these covers back. Of course, you can get a tease of my current implementation of this stuff over here, too. Not yet shippable, and there's some stuff that's going to change as I go, but this is kind of the skeleton of it so far (note that ExternDebugHandler::processRequests is called as part of the Application::run tick, if you're wondering about the re-entry point of it all)

Revenant
Posts: 440
Joined: Sat Apr 25, 2015 1:47 pm
Location: FL

Re: bsnes-plus and xkas-plus (new debugger and assembler)

Post by Revenant » Thu Aug 09, 2018 9:31 am

Cool stuff :)

Although I'm not (currently) one myself, VSCode users may also be interested in Optiroc's syntax definitions for SNES assembly. They're part of libSFX which is based around ca65 instead of WLA, but ideally I'd like to support loading symbols from both, especially since libSFX is already designed with some of bsnes-plus's existing debugging features in mind (namely software breakpoints, etc.) and it'd be great to support even tighter integration between a SNES development framework and other external tools like that.

Anyway, while I've mostly been working on Cx4 and Satellaview emulation recently, I don't have a whole lot more to do for those for the time being (until I start setting up some more hardware tests on my actual Satellaview unit, anyway). Pretty soon I'll start to integrate Benny's debug frontend stuff a little bit at a time in order to get feedback/make some potential adjustments as needed. Once that's further along I can start to get this stuff integrated as well.

(I also have a more-or-less working Qt5 branch of the existing frontend ready to go, so I might go ahead and merge that into master before I start doing any more big-time GUI updates; what do you think?)

Post Reply