It is currently Sat Nov 17, 2018 7:52 am

All times are UTC - 7 hours





Post new topic Reply to topic  [ 18 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Reverse engineering?
PostPosted: Wed Apr 04, 2018 4:50 am 
Offline
User avatar

Joined: Fri Mar 16, 2018 1:52 pm
Posts: 38
Location: Finland
Not sure if this is the correct sub-forum, but how do you go about reverse engineering a NES game? The debugger on FCEUX is pretty nice, but I kinda don't know where I should start in the process. Are there any recommendations on this to ensure that the process goes smoothly? Should I start by mapping all the RAM that I can figure out with RAM watching or should I start with a specific portion of the code right away? I'd like to know how you usually go about doing this.


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Wed Apr 04, 2018 5:04 am 
Offline
Formerly WheelInventor

Joined: Thu Apr 14, 2016 2:55 am
Posts: 1783
Location: Gothenburg, Sweden
The cc65 suite has a disassembler at your disposal.

Else, here is a list (scroll down) of disassemblers. http://6502.org/tools/asm/

Note that there's no sure-proof way for a disassembler to tell data from code; it makes its best guess.

Once you have a disassembled file (which hopefully can be reassembled), you have the job of sorting what goes where, create a map of labels, give them descriptive aliases, define constants, etc etc.

_________________
http://www.frankengraphics.com - personal NES blog


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Wed Apr 04, 2018 5:28 am 
Offline
User avatar

Joined: Fri May 08, 2015 7:17 pm
Posts: 2333
Location: DIGDUG
It would really help if you have a good idea about what all the hardware registers do, and what all the 6502 opcodes mean.

One thing you can do is, in FCEUX, debugger, hit STEP INTO, that will freeze execution (I think the spacebar does the same), then hit RESET, then step slowly through the game code. If there is a loop, set a breakpoints for the line of code just past the loop, and hit RUN. It should break at your breakpoint.

Another way is to trace 1 game loop. I think backslash advances the game 1 frame, and freezes. So hit backslash. Then turn on the Trace logger. Hit backslash. Turn off the trace logger. You can then look through the frame and see what it did.

And just look at RAM in the hex editor. Use save states to reload the same moment over and over, like when a button is pressed (jump) how does the RAM change? You can also freeze RAM addresses, and manually change them (to zero, for example), and then see how that affects the game.

Trial and error.

_________________
nesdoug.com -- blog/tutorial on programming for the NES


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Wed Apr 04, 2018 6:10 am 
Offline
User avatar

Joined: Fri Mar 16, 2018 1:52 pm
Posts: 38
Location: Finland
Are there any good ways to keep track of what you have figured out?

Also, I noticed something annoying in the debugger. Sometimes when you step through the code everything is fine, but when you start scrolling through the code the debugger changes the code to something different because it disagrees with the current view on what the code should look like. Its probably because there is other data in the middle of the code and it's end doesn't line up with how the opcodes are begin written in the debugger.


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Wed Apr 04, 2018 7:38 am 
Offline
Formerly WheelInventor

Joined: Thu Apr 14, 2016 2:55 am
Posts: 1783
Location: Gothenburg, Sweden
Quote:
Are there any good ways to keep track of what you have figured out?

Other than writing it down innotepad/np++/sublime/your favorite text editor?

You could perhaps use a spreadsheet or a database (for example from the google suite or openoffice suite) to map out and comment each RAM cell, maybe.

Or use a mindmapper software like xmind, freemind etc for creating thought bubbles and linking them together with descriptive relationship arrows.

This disasembler wasn't mentioned on 6502.org but it is meant to accompany asm6.
viewtopic.php?t=7466

_________________
http://www.frankengraphics.com - personal NES blog


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Wed Apr 04, 2018 7:46 am 
Offline
User avatar

Joined: Mon Jan 03, 2005 10:36 am
Posts: 3137
Location: Tampere, Finland
SusiKette wrote:
Also, I noticed something annoying in the debugger. Sometimes when you step through the code everything is fine, but when you start scrolling through the code the debugger changes the code to something different because it disagrees with the current view on what the code should look like. Its probably because there is other data in the middle of the code and it's end doesn't line up with how the opcodes are begin written in the debugger.

As far as I know, Mesen incorporates information from code/data logging by default in its disassembler. You might want to give it a shot.

_________________
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: fo.aspekt.fi


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Wed Apr 04, 2018 7:54 am 
Offline

Joined: Tue Feb 07, 2017 2:03 am
Posts: 625
Use this http://csdb.dk/release/?id=149429 its an interactive disassembler and has some build in NES labels for you. However it doesn't do banking, so if you have a ROM that banks, you will need to pull out all the extra banks and tackle them 1 by 1.

Step 1.
Use cheats and work out every thing you can, menu item index, lives, health, player X,Y etc SCORE is really important to obtain.
Then put add labels in regenerator, this way as you skim through code you will start to be able to see what somethings do.

Step 2. Armed with this do a Code/Data/Pointer table pass.

Step 3. Stalk Registers, this will allow you to find things like the "read joypad routine" then mark the ram addresses that hold joypad data. If something hits an Audio register then its audio. You want to find the SFX setup routine. Then use RAM hacking/cheats etc to build a list of What number/address = what sound effect.

Step 4. Armed with your SCORE routines, and SFX hunt through code to find the lines that call SFX X and add points Y. This will expose Killed Thing, Collected Item Y so you can add labels as you go.

Step 5. Find things that put sprites into OAM and tiles into VRAM, use Mesen of similar to track what a function stores and uploads. This will let you find enemy spawn, title screen, menu, text etc systems. What moves objects, updates the player etc

Step 6. At this point you might want to look at RESET and NMI Vectors and follow them as you then "Fill in the blanks", just find things you don't have labels on and work them out as you can.

If you have N banks, you can open them in regenerator, save the confiq. Open up the other file, bring in all the shared code and RAM locations, re-open the file and you will have all the useful stuff from the "boot bank confiq"


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Wed Apr 04, 2018 8:40 am 
Offline
User avatar

Joined: Thu Aug 13, 2015 4:40 pm
Posts: 310
Location: Rio de Janeiro - Brazil
It depends on what you want to do with the ROM.
I use fceux debugger features, all of them. Read the fceux FAQ througly, they are all useful for this process.

For me I do ROM hacks, and what I do is this:
Open the debugger and the code/data logger. Turn on logging. Locate the reset vector, set a break on that address. Open the hex editor, reset the rom, then step in the code until you reach the main menu loop. There you will probably have a few sub routines being called before looping. Hex edit that call (20xxxx) to #EAEAEA, reset the game and test controls, etc. Something will stop working because of your edit, so you undo the change and use the debugger's renaming functionality to give that sub routine a name depending on what stopped working (like for example "read controls", or "execute controls" or "blink press start" or "load palette").
Don't worry if you got it wrong. It will at least be related to the name you give it and you can fix it later. Any name you give is better than just hex numbers.
Go on doing that same thing forever. As it gets commented it becomes easier to see what's going on in the rom.

Also, cheats as Oziphantom mentioned.

SusiKette wrote:
Are there any good ways to keep track of what you have figured out?

Yes, rename everything you can on fceux's debugger as you go.

SusiKette wrote:
Also, I noticed something annoying in the debugger. Sometimes when you step through the code everything is fine, but when you start scrolling through the code the debugger changes the code to something different because it disagrees with the current view on what the code should look like. Its probably because there is other data in the middle of the code and it's end doesn't line up with how the opcodes are begin written in the debugger.

I believe you're talking about switched banks. This is very annoying at first. I recommend you reverse engineer a rom that doesn't bank switch the PRG so you don't get confused.

_________________
https://twitter.com/bitinkstudios <- Follow me on twitter! Thanks!


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Wed Apr 04, 2018 10:28 am 
Online
User avatar

Joined: Sat Feb 12, 2005 9:43 pm
Posts: 10973
Location: Rio de Janeiro - Brazil
FCEUX and Mesen have symbolic debugging, meaning you can name variables and subroutines right there in the disassembly window, as you figure out what they're used for.


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Wed Apr 04, 2018 12:24 pm 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 6955
Location: Canada
FrankenGraphics wrote:
Note that there's no sure-proof way for a disassembler to tell data from code; it makes its best guess.

That is what code/data logs are for. You can turn this feature on in FCEUX's menu, and it will keep track of what has been executed and what has been loaded instead. Mesen can also do this. There are rare cases where something is both code and data, or some weird assembly tricks that might confuse a disassembler, but a CDL is just about as rock solid as you can get. The main problem is trying to play enough of the game that all the code gets executed or all the data gets loaded, there will be gaps, but the more info you get in the better it will get.

I've heard of diassemblers which try to recursively follow branches to figure out what code is executable, but I'm not sure how well that works in practice. They can probably give a pretty good starting guess that way. The main thing here is that the automated tools will help, but you still have to inspect the gaps and errors in it and correct them as you figure it out.

Anyhow, step 1 for me is just to turn on the CDL and play the game for a while (try to do a little bit of everything, cheating and savestating is okay here too).

SusiKette wrote:
Are there any good ways to keep track of what you have figured out?

Since I use cc65's disassembler, da65, I would put the information into an info file for that disassembler. Frequently I'll run the disassemble and then reassemble the ROM to gain new debug info that I feed back into the debugger, i.e. all the names now show up there, which helps figure out the next information...

Also, you can process that code/data log into the disassembler info file to tell it what to leave as data. That's important too. There will be gaps in the CDL, but just disassemble everything unknown as code as first, and if it looks like garbage nonsense code it's probably data. Eventually you can sort through everything that couldn't automatically be identified.

I made a disassembly of part of StarTropics a while back. It has an example debugger info file. https://forums.nesdev.com/viewtopic.php?f=6&t=12040

SusiKette wrote:
Also, I noticed something annoying in the debugger. Sometimes when you step through the code everything is fine, but when you start scrolling through the code the debugger changes the code to something different because it disagrees with the current view on what the code should look like. Its probably because there is other data in the middle of the code and it's end doesn't line up with how the opcodes are begin written in the debugger.

Sometimes the top of the debugger window in FCEUX starts on a byte address that's in the middle of an instruction. You can scroll up up or down to get realigned.


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Wed Apr 04, 2018 1:26 pm 
Offline
Formerly Espozo
User avatar

Joined: Mon Sep 15, 2014 4:35 pm
Posts: 3381
Location: Richmond, Virginia
Nobody has mentioned it yet, but IDA Pro is a great disassembler that supports most any platform. I'm pretty sure it natively recognizes NES games, so all you'd need to do is drag and drop it into the program and it'll try to find what's code and separate it into routines. From what I've seen of it with SNES games though and what Rainwarrior was hypothesizing about these sort of tools, its usefulness varies considerably in that many games require you to show it different entry points where code executed from, because otherwise, there's large gaps that IDA Pro doesn't recognize as anything in the program. The biggest problem though, is that if you don't want to pirate, it's $800...

I'm far from the greatest person to give advise on this sort of thing, but I think it largely comes down to doing a variety of methods that supplement each other and fill in the knowledge gaps. It's a ton of work no matter what you do, but it brings it into the realm of possibility.


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Wed Apr 04, 2018 1:40 pm 
Offline
Formerly WheelInventor

Joined: Thu Apr 14, 2016 2:55 am
Posts: 1783
Location: Gothenburg, Sweden
rainwarrior wrote:
That is what code/data logs are for. You can turn this feature on in FCEUX's menu, and it will keep track of what has been executed and what has been loaded instead. Mesen can also do this. There are rare cases where something is both code and data, or some weird assembly tricks that might confuse a disassembler, but a CDL is just about as rock solid as you can get. The main problem is trying to play enough of the game that all the code gets executed or all the data gets loaded, there will be gaps, but the more info you get in the better it will get.


Thanks for the insightful explanation!

Might be going overboard, but, maybe this approach could play well in hand with one of those machine learning projects where you set the goal like beating the game or a level and avoid some other cases and the ai will repeatedly try its best. Set up some objectives and run the emulation at top speed and you might get the ai to try a lot more stuff than a human would, in less time.

_________________
http://www.frankengraphics.com - personal NES blog


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Wed Apr 04, 2018 2:10 pm 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 6955
Location: Canada
FrankenGraphics wrote:
Might be going overboard, but, maybe this approach could play well in hand with one of those machine learning projects where you set the goal like beating the game or a level and avoid some other cases and the ai will repeatedly try its best. Set up some objectives and run the emulation at top speed and you might get the ai to try a lot more stuff than a human would, in less time.

Well, AI will definitely try more stuff, but the important goal here is to get it to expose more code. Generally speaking, AI is very bad at that secondary goal.

E.g. you could jump from every single pixel you can stand on in Super Mario Bros. level 1 but most of those won't cause any new code to be executed. Usually things that get more code logged are things designed with human intent. It's useful to interact with one of each type of thing that exists in the game, but it's not so useful to be interacting with every instance of them. Getting to the end of the game is critical for logging the code that handles the ending. Etc.

I dare you to show me an SMB1 AI that can actually beat the game. ;P

Really, the amount of work it takes to write effective game playing AI overshadows the time it takes to just thoroughly play through the whole game. Orders of magnitude more work, here.


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Thu Apr 05, 2018 2:49 am 
Offline
User avatar

Joined: Fri Mar 16, 2018 1:52 pm
Posts: 38
Location: Finland
I've been doing some data logging now and I think I have gone through everything that's possible in the game. However, I'm left with some data that's not logged (25,89% PRG and 16,26% CHR). I'm not too concerned about CHR because I know that there is some unused graphics in the game. In the data logger there is a section called "CHR logged as data". What does this mean? Is it like palettes and nametables etc.?

I made a list of all the things I did. I don't think I have missed anything, but if you notice something that I didn't do (that is if you know the game), let me know.

Anyway, I guess this is at least enough to get started.


Top
 Profile  
 
 Post subject: Re: Reverse engineering?
PostPosted: Thu Apr 05, 2018 3:14 am 
Offline
User avatar

Joined: Sun Sep 19, 2004 9:28 pm
Posts: 3686
Location: Mountain View, CA
SusiKette wrote:
However, I'm left with some data that's not logged (25,89% PRG and 16,26% CHR).

You won't see 100% coverage for either of those in most scenarios. It would require the emulator to run every single piece of code that's present in the ROM, and/or access every piece of graphical information (CHR) in the ROM. Meaning: you'd literally have to play it all the way through, and cover every single "edge case" there is (go through every menu, every button combination, EVERYTHING -- even stuff that doesn't do anything). And even if you did, there's often stuff that's never used/left over (from debugging, other in-progress ideas, whatever) in commercial games. 100% is pretty unlikely. Some of my romhacking projects had around 95-96% after very, very thorough playthroughs.

IMO, CDLs aren't a good starting point for reverse-engineering something. They're overwhelming and often half-ass -- they sound great on paper, but what matters is if related tools can actually use/benefit from them, and there are *extremely* few.

Again IMO, here's a better idea: find something you want to learn about in the game. Be specific. Some examples: "I want to find out what piece of code handles me pressing Start to start the game", "I want to change the letter 'A' on the title screen to the letter 'Z'", or "I want to change the letter 'A' on the title screen to resemble a smiley face." These are good starting beginning points (and actually how a lot of people get involved in romhacking/fan translation).

P.S. -- Recca is probably one of **the worst** games you could start with for reverse-engineering. This game is literally ridiculous, and would fall into probably the top 100 or 200 "technically complex" games to RE. If this is your first attempt at RE'ing something, especially on the NES, then I suggest you start with a simpler game. Try a first-gen title that doesn't use a mapper if at all possible: games like Golf, Ice Hockey, Tennis, or Balloon Fight to name a few. Super Mario Bros is also sometimes a starting point for people.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 18 posts ]  Go to page 1, 2  Next

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group