nesdev.com

I have now confirmed that neither Old PPU or New PPU in FCUEX, nor Bizhawk correlate to any hardware configuration, whether Front or Top Loader with Everdrive, or Front or Top Loader with original cart. I am not able to get these emulators to correlate to the original hardware, and I am not sure if there is a way to do this. If there is a way to write in a value or two in an address that would modify how these emulators work, even if I have to try 255 values in a single address to get a correlation, I will do it, but I would need to know if a correlation is even possible.

So I have confirmed that changing the core in Bizhawk from QuickNES to NesHawk gives yet another set of values. So FCEUX(Old PPU), FCEUX(New PPU), Bizhawk(QuickNES), and Bizhawk(NesHawk) all yield different results, so four different outcomes. Once again changing the initial RAM State either in Bizhawk or FCEUX does not modify outcomes since the Rom initializes its own values. What changes this? Why such a difference between emulator outcomes? And yet none of these outcomes match original cart with Front Loader, or with Top Loader, nor use of Everdrive with Front/Top Loader. There must be a way to know why and have the ability to sync some of this. Any thoughts?

I have also confirmed that resetting the game does not retain RNG values so it is not required to power cycle the NES in order to do RNG manipulation.

I have also confirmed that it does not matter the hardware that you use. I have been able to get the same RNG manipulation for real cart with Top Loader, real cart with Front Loader, Everdrive with Top Loader, and Everdrive with Front Loader. All the outcomes are the same, just can't get any emulator with a TAS editor to sync with it.

I have concluded that the state of NES emulation is not yet sufficient in order to TAS this game. It is a personal opinion of mine that it isn't the same if the emulator is not synced with the NES, and can't be reproduced on original hardware in the same way, therefore I will not TAS it. This non-sync issue will need to be corrected in the future so that emulators can be utilized to TAS 1000's of variations of this game which would otherwise take weeks. Until that day comes...

What exactly is the test you are doing?

I had found that I can get the same item placements in the game by doing inputs to manipulate the RNG. For example, I will press and hold A then hit reset, I will continue to hold A until after the screen blacks out after hitting start. When I press start I continue to hold it with the A button until the same black out. This way all the input manipulation can be duplicated and the only thing I had to reset over and over again for was to begin pressing start on the same frame as before. I increased the world record several times using this method and since I knew where all the items were with that outcome I was able to focus on optimizing the route. So naturally I wanted to search for other routes that would be faster and focus on running that route to improve the record further. I kept a log of all the trials I ran with the TAS editor for both FCEUX(Old PPU), and FCEUX(New PPU). https://docs.google.com/spreadsheets/d/ ... =903343879

I did over 2000 trials only to find that none of the results actually correlated with original hardware. I thought maybe the memory was not initialized and therefore impacted by the Initial Ram State, but neither of these were the case. It seems that the state of NES emulation is not yet accurate enough in order to produce the same results as original hardware. I have been able to reproduce the same outcomes with original cart/Top Loader, original cart/Front Loader, Everdrive with both. All of these hardware configurations all yielded the same results. It baffles me why I can get four different results from the emulators: Bixhawk(quickNes), Bizhawk(NesHawk), FCEUX(Old PPU), and FCEUX(New PPU). None of these yielded the same results as the other,s and none of them especially did not sync with original hardware.

The plan was to TAS 1000's of variations of input manipulation that I could duplicate in real time and watch the item address values. I found a route that was 19 seconds faster and it would have been nice to actually use the input manipulation in order to achieve it with original hardware. I was able to observe outcomes for each trial within 10 seconds and it was incredibly expedient. As you can tell from the spreadsheet it was very rare for me to find an outcome that would have been a faster route than what I discovered by luck by simple trial and error with my console. I would have TASed the outcome I received from my input manipulation with original hardware but once again I am not able to force the emulator to simulate original hardware.

I was able to drive the world record down to 4:36 with my discovery: https://www.twitch.tv/videos/160313557

I am hoping that maybe I can force one of these emulators to produce the same outcomes by maybe writing a Lua script to change a value or two in some important address that influences the course of the process. I am sure through a process of trial and error I would be able to produce the same results with the input manipulation I do with original hardware. But I am not sure it is really that simple, or if the issue is much more complicated and not easily solved, even where to begin. I spent some time with the debugger but I was not able to make sense out of what I observed, and then concluded that maybe the state of NES emulation is not yet perfected and there is nothing that I can do until it improves. That is to say that perhaps the matter is simply beyond both present software limitations and my lack of knowledge in this area.

Can you give step by step instructions about the test, though? You started to explain about holding A etc. but what is the result? What do you do in the game to see that the PRNG has been set to something specific?

Certainly. I have been holding A from power on and press and hold start around the first opportunity to do so, and I continue to hold A and Start until the following addresses populate with values which do so within 4 frames during a dead zone transition between the title screen and the screen that show how many lives you begin with. If I began to press start on the right frame I will get the following results. I can not say exactly what the frame is when I press start because it is on console, but when I check for the same outcome in any of the 4 emulator configurations I obtain results from the first 20 available frames to find a match, which has not occurred.

0657 has 40
064B has 42
0639 has 29
0631 has 21
0645 has 53
0661 has 6
065D has 2
0663 has 16
065F has 13

I use the TASeditor to apply the inputs and I see what the result is in these addresses. That is exactly what I am doing.

I have noticed that the values in addresses 0012, 0013, 0014, and 0015 differ between emulator configurations, meaning Bizhawk(quickNes), BizHawk(NesHawk), FCUEX(Old PPU), FCEUX(New PPU). I am not sure what PRNG is, nor where have I found a way that I understand to dig deeper.

So basically I use the Taseditor to replicate my input manipulation and watch the results from initializing my start input from frame 1 to 20 of first opportunity which is around frame 534.

What about on hardware though? Specifically I wanted to know how you test the result of the PRNG in the game (not to do with memory values you see in a debugger).

So...

1. Hold A from power on.
2. Press and hold Start when the "PRESS START" thing has finished fading in.

I'm unclear on what to do next. Eventually the game begins, and I'm in a room and I can punch Roger. Where do I go, what do I do to see the statue of the PRNG?

Here, I'll try and go through the steps of how I'd look into this using FCEUX. You mentioned that the memory address $0657 holds some randomly generated value for "shoes". Let's look at that...

Step 1: Go to Tools > RAM Watch and add an entry for 0657 so wee can look at it as the game is running.

Step 2: Start the game and play it until it changes. Here I notice it happens when leaving the "PRESS START" screen.

Step 3: Go back to before it changed (by resetting or with a savestate, etc.) and try to get close to the point where it is about to change and pause the emulator.

Step 4: Go to Debug > Trace Logger, select Log to File and click browse to create a file. Start Logging.

Step 5: Go to Debug > Debugger and add a breakpoint on writes to 0657, this is going to automatically pause when it gets written to.

Step 6: Click Run in the debugger, or otherwise unpause. Execution will proceed slowly (the trace is writing everything that's happening to a file), and it will eventually automatically pause when $0657 is written to. Once it pauses click "Step Into" once just to finish the instruction that actually writes $0657.

Step 7: Stop logging in the trace. Open the file you created and go to the bottom. Here's the last two instructions:


So we see here that a value $29 is loaded from $0006 and written to $0657. We want to find out where that value came from...

Step 8: Use a text search tool to search backwards for "0006" from that line until you find a write to $0006.

This ends up taking a little while because there's a long repetitive series of this loop:


It seems to look at a list of previously generated values and compare against it. You said this number determines the location of the shoes, right? Well, this is probably trying to make sure it doesn't put 2 objects in the same location.

Step 9: Here we see that $0006 comes from a series of 2 table lookups (i guess some kind of "location" table), but those lookups are determined by a combination of $0012 and $0005. Next to find out how those two values were filled... keep searching backwards!


Step 10: A little futher back we see that $0005 is actually just the value from $0004 but times 4. So now we have to keep searching for $0004 instead. In the same place we also find where $0012 came from, and this looks like our main PRNG step, maybe?


$0012-0014 seem to be part of a 24-bit PRNG seed, though now there's also a value from $0002 that seems to be involved. If I keep searching back from here, I just see iterations of this same step for $0012-0014. My trace log doesn't seem to go far enough back to figure out how they were initially set up, but that's fine, we have identified that they were important for setting up $0657 and we can begin a new investigation from here.

Similarly the determination for $0002 is not immediately clear from the trace log I have saved, but it's something we can now dig into.

Step 11: Add $0002, $0012-0014 to the ram watch again. I notice that they're changing all the time. $0002 in particular is increasing by 1 each frame. Putting a breakpoint on writes to it, I notice that it's incremented by 1 each frame, during the NMI routine. So... this appears a frame counter.

So... Observation 1: the random setup depends on how many frames have elapsed since power-on before you start the game.

Step 12: Putting a breakpoint on writes to $0012, I find this routine that runs in a loop many many times each frame during startup:



This just runs being called over and over in an infinite loop until an NMI interrupts it and eventually manipulates the stack to get it out of that infinite loop. This is setting up the 24-bit PRNG seed. It maybe runs 40 times per frame.

Observation 2: this random seed loop is broken only by the NMI, so this will require precise PPU emulation to get correct. If it's even one cycle different, the PRNG seed could be changed.

Step 13: That seed generating loop from before also depends on other values from other places, in particular $0016-0019 are where controller inputs are stored (16 = gamepad 0, 17 = gamepad 0 new presses this frame, 18-19 = gamepad 1 similar).

Observation 3: the random seed is influenced by controller input before the start.

$0011 also seems to fold into it but it appears to be a constant $80 after startup, so I won't bother looking into it.


SUMMARY

So... I mean we could keep going here, but here's what the situation looks like to me:

• For a consistent PRNG setup, you have to press START on the very first possible frame that you can.
• Input on both gamepads before that start does influence the PRNG seed.
• Power-on RAM values so far do not appear to have any influence.
• Even one cycle difference in the specific timing of the NMI can also alter the seed.

That last one in particular is one that emulators might easily behave differently on, especially reset/warmup behaviour of the PPU. It's also possible that on hardware the variability of PPU alignment might mean you only have a 1 in 4 chance of getting a consistent alignment on reset/power-up too, even if you could get that frame-perfect START every time.

Does this agree/disagree with your experience with the game?

This reply is absolutely amazing! I can not thank you enough! It certainly deserves a much more though response than this, but I will need to unpack what has been said here and watch through the steps to better understand how I might be able to reproduce your findings. While understanding its progress will most likely elude me it may help me see more than I had before in terms of learning.

As for how I see if I have the right PRNG or not... I simply have to walk up to the top store location every time to see if the shoes are sitting at that location, if so I go two buildings to the right and if there is a rattle behind the mouse then I am on the right 'seed" 100% of the time. Otherwise I simply reset and try again.

Are you saying that even if I am frame perfect and do the same input manipulation every time that there is still a 75% chance that I will get different results than the expected "seed"? If so, then I can say that I have often observed longer periods of time between successful attempts than I would expect. There have been times when it has taken me 20-30 mins before I get the right "seed" and I have had a faint suspicion that maybe there is something else going on. I was not certain of this because getting a frame perfect input is not always easy, in fact at times even with an audio cue it can be hard to hit a 7 frame window such as in Teenage Mutant Ninja Turtles in order to get a good Technodrome location, but again having achieved 10 consecutive frame perfect punches on Mike Tyson before I felt that something else was happening. If this is indeed the case, then could it be that there are times when original hardware may in fact intersect with one of these emulator configurations? I have ran several trials and this does not seem to be the case.

I will also play around with overwriting values to some of these new found addresses to see if I can manipulate the outcomes, but wondering if this would be an over-simplified attempt to solve a broader problem?

Well, as far as I can tell controller input is only read once per frame, so anything you can influence with input should not require anything more detailed than that. Maybe I misspoke when I said "first" frame. You could get a consistent seed from any consistent number of frames from the start, I think.

So... you have one "known" PRNG seed that you have managed to get several times in your tests, right? I would presume that you don't know, however, which exact frame you triggered start on this successful seed, though. If you are trying to find a TAS that hits that seed, if the emulator was sufficiently equal to your hardware version in behaviour you would also need to use the same start frame too. (I think you already mentioned this above anyway, but just trying to confirm.)

It's a bit strange to me that this game ticks it PRNG so many times per frame like this. Without polling the controller more than once per frame, it's basically the same as just ticking it once per frame as far as entropy-gathering goes.

So do you think that it would be possible for me to write in a value in one of these addresses mentioned above via Lua Script early on in the TAS that would modify the results and potentially get the outcome I am expecting, or is it more complicated than that?