PPU BGnHOFS latching behavior is emulated wrong everywhere

Discussion of hardware and software development for Super NES and Super Famicom. See the SNESdev wiki for more information.

Moderator: Moderators

Forum rules
  • For making cartridges of your Super NES games, see Reproduction.
Post Reply
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

PPU BGnHOFS latching behavior is emulated wrong everywhere

Post by Near »

Long ago, anomie observed a horizontal scrolling bug in Theme Park. The fix he came up with is pasted below in code blocks, but essentially it was to only update the top eight bits with the new byte on BGnHOFS writes, and for the low eight bits ... the top five came from the previous register write, and the bottom three from the previous register value.

I don't know why he came up with this fix, or why it seemed to help. One possibility is it's related to having a dot-based renderer? Seems unlikely, though. Cydrak suggested that perhaps Snes9X wasn't using a shared latch for all BGnxOFS registers.

But regardless ... this behavior breaks the Pac-Man homebrew game by RTS. The title screen horizontal scrolling becomes extremely shaky, because the game is performing 16-bit writes to both BG3HOFS and BG3VOFS at the same time.

Upon removing the weird logic and treating BGnHOFS writes exactly like BGnVOFS writes, the problem is resolved in Pac-Man ... and there is no observable difference in either the NTSC-J or PAL releases of Theme Park anywhere in the game that I can observe. Definitely no issue on the main map where there used to be the bug.

Note that this errant behavior is pervasive. The bug can be seen in bsnes/higan, Snes9X, SNESGT, no$sns, Kindred, and ZSNES.

Corrected higan code: (do this for all BGnHOFS registers)

Code: Select all

  //BG3HOFS
  case 0x2111: {
    bg3.io.hoffset = data << 8 | latch.bgofs;  //correct behavior
  //bg3.io.hoffset = data << 8 | (latch.bgofs & ~7) | (bg3.io.hoffset >> 8 & 7);  //incorrect behavior
    latch.bgofs = data;
    return;
  }

  //BG3VOFS
  case 0x2112: {
    bg3.io.voffset = data << 8 | latch.bgofs;
    latch.bgofs = data;
    return;
  }
Again, I don't have my scanline renderer anymore. I can't say if reverting this will break Theme Park for other emulators again or not, but ... the behavior is almost surely wrong. I don't know when or how Theme Park started working correctly in higan without this hack.

...

anomie's regs.txt:

Code: Select all

210f  ww+++- BG2HOFS - BG2 Horizontal Scroll
2110  ww+++- BG2VOFS - BG2 Vertical Scroll
2111  ww+++- BG3HOFS - BG3 Horizontal Scroll
2112  ww+++- BG3VOFS - BG3 Vertical Scroll
2113  ww+++- BG4HOFS - BG4 Horizontal Scroll
2114  ww+++- BG4VOFS - BG4 Vertical Scroll
        ------xx xxxxxxxx

        Note that these are "write twice" registers, first the low byte is
        written then the high. Current theory is that writes to the register
        work like this:
          BGnHOFS = (Current<<8) | (Prev&~7) | ((Reg>>8)&7);
          Prev = Current;
            or
          BGnVOFS = (Current<<8) | Prev;
          Prev = Current;

Code: Select all

The registers $210d-$2114 are all write-twice to set the 16-bit value. The way
this works, the last write to any of these registers is stored in a buffer.
When a new byte is written to any register, the current register value, the
previous byte written to any of the 6 registers, and the new byte written
are combined as follows:
  For BGnHOFS: (NewByte<<8) | (PrevByte&~7) | ((CurrentValue>>8)&7)
  For BGnVOFS: (NewByte<<8) | PrevByte
For the most part, the details don't really matter as most games always write
two bytes to one of these registers. However, some games write only one byte,
or they do other odd things.
Thanks to BMF54123 for bringing this bug to my attention.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: PPU BGnHOFS latching behavior is emulated wrong everywhe

Post by tepples »

Perhaps it's because on the NES, the fine scroll bits have qualitatively different behavior from the rest of the scroll position, as described in "The skinny on NES scrolling". But as I understand it, mode 0-6 backgrounds on the Super NES instead continuously add the scroll position to the scanline position to form each layer's nametable address and fine scrolls, so this separation isn't quite as necessary. Am I correct in this understanding?
Revenant
Posts: 462
Joined: Sat Apr 25, 2015 1:47 pm
Location: FL

Re: PPU BGnHOFS latching behavior is emulated wrong everywhe

Post by Revenant »

All this time I assumed that background just looked that way on a real console too, just because of less-than-perfect hardware understanding among homebrew authors in 1993.

Well, at least I was right about the latter part, since after BMF mentioned it to me on IRC last night, I realized the game was attempting to do a single 16-bit write to $2111 and then another 16-bit write to $2112, rather than double 8-bit writes to either. Of course in theory, this would have totally fucked up the scrolling, so I was surprised when it didn't seem to on the real thing.

I'll go ahead and make this change in bsnes-plus once I get home from work. I'll be sure to check out Theme Park in the scanline renderer afterwards to see if anything weird happens.

edit: done, no problem with Theme Park as far as I can tell.
creaothceann
Posts: 611
Joined: Mon Jan 23, 2006 7:47 am
Location: Germany
Contact:

Re: PPU BGnHOFS latching behavior is emulated wrong everywhe

Post by creaothceann »

byuu wrote:The bug can be seen in bsnes/higan, Snes9X, SNESGT, no$sns, Kindred, and ZSNES.
And probably in jwdonal's emulator too.
My current setup:
Super Famicom ("2/1/3" SNS-CPU-GPM-02) → SCART → OSSC → StarTech USB3HDCAP → AmaRecTV 3.10
AWJ
Posts: 433
Joined: Mon Nov 10, 2008 3:09 pm

Re: PPU BGnHOFS latching behavior is emulated wrong everywhe

Post by AWJ »

Unfortunately, byuu's suggested change does break Theme Park, and the breakage is quite subtle:
Theme Park main screen in current bsnes-classic master
Theme Park main screen in current bsnes-classic master
Theme Park (J) good.png (9.19 KiB) Viewed 7590 times
Theme Park main screen with byuu's change applied - layers are now misaligned
Theme Park main screen with byuu's change applied - layers are now misaligned
Theme Park (J) bad.png (9.23 KiB) Viewed 7590 times
The following patch, on the other hand, works both for Theme Park and the homebrew Pac-Man:

Code: Select all

diff --git a/bsnes/snes/alt/ppu-compatibility/mmio/mmio.cpp b/bsnes/snes/alt/ppu-compatibility/mmio/mmio.cpp
index a2537c4..0cb9bc8 100644
--- a/bsnes/snes/alt/ppu-compatibility/mmio/mmio.cpp
+++ b/bsnes/snes/alt/ppu-compatibility/mmio/mmio.cpp
@@ -111,8 +111,9 @@ void PPU::mmio_w210d(uint8 value) {
   regs.m7_hofs  = (value << 8) | regs.m7_latch;
   regs.m7_latch = value;
 
-  regs.bg_hofs[BG1] = (value << 8) | (regs.bg_ofslatch & ~7) | ((regs.bg_hofs[BG1] >> 8) & 7);
-  regs.bg_ofslatch  = value;
+  regs.bg_hofs[BG1] = (value << 8) | (regs.bg_ppu1ofslatch & ~7) | (regs.bg_ppu2ofslatch & 7);
+  regs.bg_ppu1ofslatch = value;
+  regs.bg_ppu2ofslatch = value;
 }
 
 //BG1VOFS
@@ -120,44 +121,47 @@ void PPU::mmio_w210e(uint8 value) {
   regs.m7_vofs  = (value << 8) | regs.m7_latch;
   regs.m7_latch = value;
 
-  regs.bg_vofs[BG1] = (value << 8) | (regs.bg_ofslatch);
-  regs.bg_ofslatch  = value;
+  regs.bg_vofs[BG1] = (value << 8) | (regs.bg_ppu1ofslatch);
+  regs.bg_ppu1ofslatch = value;
 }
 
 //BG2HOFS
 void PPU::mmio_w210f(uint8 value) {
-  regs.bg_hofs[BG2] = (value << 8) | (regs.bg_ofslatch & ~7) | ((regs.bg_hofs[BG2] >> 8) & 7);
-  regs.bg_ofslatch  = value;
+  regs.bg_hofs[BG2] = (value << 8) | (regs.bg_ppu1ofslatch & ~7) | (regs.bg_ppu2ofslatch & 7);
+  regs.bg_ppu1ofslatch = value;
+  regs.bg_ppu2ofslatch = value;
 }
 
 //BG2VOFS
 void PPU::mmio_w2110(uint8 value) {
-  regs.bg_vofs[BG2] = (value << 8) | (regs.bg_ofslatch);
-  regs.bg_ofslatch  = value;
+  regs.bg_vofs[BG2] = (value << 8) | (regs.bg_ppu1ofslatch);
+  regs.bg_ppu1ofslatch = value;
 }
 
 //BG3HOFS
 void PPU::mmio_w2111(uint8 value) {
-  regs.bg_hofs[BG3] = (value << 8) | (regs.bg_ofslatch & ~7) | ((regs.bg_hofs[BG3] >> 8) & 7);
-  regs.bg_ofslatch  = value;
+  regs.bg_hofs[BG3] = (value << 8) | (regs.bg_ppu1ofslatch & ~7) | (regs.bg_ppu2ofslatch & 7);
+  regs.bg_ppu1ofslatch = value;
+  regs.bg_ppu2ofslatch = value;
 }
 
 //BG3VOFS
 void PPU::mmio_w2112(uint8 value) {
-  regs.bg_vofs[BG3] = (value << 8) | (regs.bg_ofslatch);
-  regs.bg_ofslatch  = value;
+  regs.bg_vofs[BG3] = (value << 8) | (regs.bg_ppu1ofslatch);
+  regs.bg_ppu1ofslatch = value;
 }
 
 //BG4HOFS
 void PPU::mmio_w2113(uint8 value) {
-  regs.bg_hofs[BG4] = (value << 8) | (regs.bg_ofslatch & ~7) | ((regs.bg_hofs[BG4] >> 8) & 7);
-  regs.bg_ofslatch  = value;
+  regs.bg_hofs[BG4] = (value << 8) | (regs.bg_ppu1ofslatch & ~7) | (regs.bg_ppu2ofslatch & 7);
+  regs.bg_ppu1ofslatch = value;
+  regs.bg_ppu2ofslatch = value;
 }
 
 //BG4VOFS
 void PPU::mmio_w2114(uint8 value) {
-  regs.bg_vofs[BG4] = (value << 8) | (regs.bg_ofslatch);
-  regs.bg_ofslatch  = value;
+  regs.bg_vofs[BG4] = (value << 8) | (regs.bg_ppu1ofslatch);
+  regs.bg_ppu1ofslatch = value;
 }
 
 //VMAIN
diff --git a/bsnes/snes/alt/ppu-compatibility/mmio/mmio.hpp b/bsnes/snes/alt/ppu-compatibility/mmio/mmio.hpp
index aeb1c3a..7068afb 100644
--- a/bsnes/snes/alt/ppu-compatibility/mmio/mmio.hpp
+++ b/bsnes/snes/alt/ppu-compatibility/mmio/mmio.hpp
@@ -45,7 +45,8 @@ struct {
   uint16 bg_tdaddr[4];
 
   //$210d-$2114
-  uint8  bg_ofslatch;
+  uint8  bg_ppu1ofslatch;
+  uint8  bg_ppu2ofslatch;
   uint16 m7_hofs, m7_vofs;
   uint16 bg_hofs[4];
   uint16 bg_vofs[4];
diff --git a/bsnes/snes/alt/ppu-compatibility/ppu.cpp b/bsnes/snes/alt/ppu-compatibility/ppu.cpp
index 768ca4c..6c631b6 100644
--- a/bsnes/snes/alt/ppu-compatibility/ppu.cpp
+++ b/bsnes/snes/alt/ppu-compatibility/ppu.cpp
@@ -191,7 +191,8 @@ void PPU::power() {
   regs.bg_tdaddr[BG4] = 0x0000;
 
   //$210d-$2114
-  regs.bg_ofslatch = 0x00;
+  regs.bg_ppu1ofslatch = 0x00;
+  regs.bg_ppu2ofslatch = 0x00;
   regs.m7_hofs = regs.m7_vofs = 0x0000;
   regs.bg_hofs[BG1] = regs.bg_vofs[BG1] = 0x0000;
   regs.bg_hofs[BG2] = regs.bg_vofs[BG2] = 0x0000;
diff --git a/bsnes/snes/alt/ppu-compatibility/serialization.cpp b/bsnes/snes/alt/ppu-compatibility/serialization.cpp
index 078ad48..d69e545 100644
--- a/bsnes/snes/alt/ppu-compatibility/serialization.cpp
+++ b/bsnes/snes/alt/ppu-compatibility/serialization.cpp
@@ -53,7 +53,8 @@ void PPU::serialize(serializer &s) {
 
   for(unsigned n = 0; n < 4; n++) s.integer(regs.bg_tdaddr[n]);
 
-  s.integer(regs.bg_ofslatch);
+  s.integer(regs.bg_ppu1ofslatch);
+  s.integer(regs.bg_ppu2ofslatch);
   s.integer(regs.m7_hofs);
   s.integer(regs.m7_vofs);
   for(unsigned n = 0; n < 4; n++) s.integer(regs.bg_hofs[n]);
(the corresponding fix for the other two PPU implementations should be obvious)

Good job pointing this out, even if your suggested fix wasn't correct.
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: PPU BGnHOFS latching behavior is emulated wrong everywhe

Post by lidnariq »

I'm surprised I was close.

Anyway, just because those two images are really subtly different, here's a GIF that flips between them:
Attachments
Theme Park (J) comparison.gif
Theme Park (J) comparison.gif (12.89 KiB) Viewed 7582 times
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Re: PPU BGnHOFS latching behavior is emulated wrong everywhe

Post by Near »

Jesus Christ that is subtle. Have we actually confirmed that microscopic difference on real hardware? How did anyone ever even find it??

> Good job pointing this out, even if your suggested fix wasn't correct.

And all the same, we really should confirm your fix as well with a test ROM. I wasn't intending to leave my fix alone untested, and indeed was just writing up a test ROM scenario to verify my claim before you posted.

Nonetheless, great work as well coming up with something that truly works for both! I'm hoping you are proven right, as it's a fairly elegant fix.

Another theory I was considering was ... Theme Park writes HOFS during Hblank. Whereas Pac-Man does so during NMI. It's unlikely, but maybe the latching behavior low three bits doesn't apply outside of active rendering. I know, simpler is better, but again, if we can make a test ROM, I would be willing to put this to bed.

We got in this mess because even I refused to verify anomie's finding the first time and just went with it. Let's not make the same mistake again.

We'll need someone with a display that can show the very left or very right edge of the SNES output cleanly. We can eke out the existence of a PPU2 latch only on HOFS registers pretty easily. I'd also wonder if the latch -only- applied to BGnHOFS, or if perhaps the PPU2 MDR comes into play here from other registers as well.
AWJ
Posts: 433
Joined: Mon Nov 10, 2008 3:09 pm

Re: PPU BGnHOFS latching behavior is emulated wrong everywhe

Post by AWJ »

byuu wrote:Jesus Christ that is subtle. Have we actually confirmed that microscopic difference on real hardware? How did anyone ever even find it??
Someone had sharp eyes back in 2005 :) That 3-pixel difference is how Anomie discovered the fine hscroll latching quirk in the first place: http://board.zsnes.com/phpBB3/viewtopic ... &view=next

You can run additional tests if you want, but I'm extremely confident that my fix is hardware-accurate. As I pointed out in the thread lidnariq linked, the fine hscroll registers (i.e. the low 3 bits of BGnHOFS) have to be on PPU2 rather than PPU1.
Theme Park writes HOFS during Hblank.
As far as I can tell, Theme Park only writes to the BG1 scroll registers once, at startup (BG1 is the layer that ends up misaligned if you don't emulate the latching properly). All the "scrolling" ingame is done by redrawing the entire screen--that's why it's so chunky.
I'd also wonder if the latch -only- applied to BGnHOFS, or if perhaps the PPU2 MDR comes into play here from other registers as well.
That is so unlikely as to be impossible. Think about it: if writing to some unrelated PPU register had side effects on horizontal scrolling, there'd be chaos.
Revenant
Posts: 462
Joined: Sat Apr 25, 2015 1:47 pm
Location: FL

Re: PPU BGnHOFS latching behavior is emulated wrong everywhe

Post by Revenant »

Well, shit. That slipped by me too.

For what it's worth, a hardware screencap:

Image
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Re: PPU BGnHOFS latching behavior is emulated wrong everywhe

Post by Near »

Agh, I just went through the trouble of verifying this one myself, too, heh.

Remembered I have a complete Super Famicom collection. So 20 minutes of dicking with 15 cables later, and ...

Image

Yeah, fascinating. I'll go with AWJ's behavior for now because it's clearly more right and we have a visible breakage (BMF54123 confirmed Pac-Man works on real hardware), but let's try to come up with some test ROMs. I'll see if I can help this time instead of just asking for them to be made.

But by all means if someone wants to beat me to it ... ;)

> You can run additional tests if you want, but I'm extremely confident that my fix is hardware-accurate. As I pointed out in the thread lidnariq linked, the fine hscroll registers (i.e. the low 3 bits of BGnHOFS) have to be on PPU2 rather than PPU1.

How confident are you that no other PPU register write from $2100-213f affects the bg_ppu2ofslatch value you've added?

> As far as I can tell, Theme Park only writes to the BG1 scroll registers once, at startup (BG1 is the layer that ends up misaligned if you don't emulate the latching properly). All the "scrolling" ingame is done by redrawing the entire screen--that's why it's so chunky.

I added print statements to every BGnHOFS write, and it was spamming the terminal every single scanline during the map display.

> That is so unlikely as to be impossible. Think about it: if writing to some unrelated PPU register had side effects on horizontal scrolling, there'd be chaos.

Would there really? We went ten years thinking anomie's behavior was exactly correct. To the point that every single emudev I'm aware of added the behavior.

At least 99.9% of games clearly write each register twice always, so such an effect wouldn't be observable in anything but maybe Theme Park and/or Pac-Man.
AWJ
Posts: 433
Joined: Mon Nov 10, 2008 3:09 pm

Re: PPU BGnHOFS latching behavior is emulated wrong everywhe

Post by AWJ »

And done.

https://github.com/awjackson/bsnes-clas ... decfd67d51
I added print statements to every BGnHOFS write, and it was spamming the terminal every single scanline during the map display.
The BG2 scroll registers are continuously HDMAed, and nothing weird is going on with BG2 (the ticket booths are BG1, the ground is BG2). The BG1 scroll registers aren't touched at all once you're ingame.
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Re: PPU BGnHOFS latching behavior is emulated wrong everywhe

Post by Near »

Just a quick note for anyone testing that the glitches on the edge of the screen when scrolling in this game, like so:

Image

Also occur on real hardware. This just really isn't a very well-programmed game.
Revenant
Posts: 462
Joined: Sat Apr 25, 2015 1:47 pm
Location: FL

Re: PPU BGnHOFS latching behavior is emulated wrong everywhe

Post by Revenant »

I noticed the same thing when testing the patch, and was surprised (or relieved, maybe) to see it happening on the actual system too. You can also make the gate disappear one tile early if you scroll about the same distance in the other direction.
AWJ
Posts: 433
Joined: Mon Nov 10, 2008 3:09 pm

Re: PPU BGnHOFS latching behavior is emulated wrong everywhe

Post by AWJ »

It just occurred to me a way of thinking about the scroll registers that perhaps makes them less surprising.

It's long been known that the BG1 scroll registers are "two in one": there's one set that's used in modes 0-6, and one set that's used in mode 7, and the two sets can get out of sync (contain different values) if you don't always write twice to each register before writing to a different one. The reason they can get out of sync is that the mode 0-6 set shares a latch/FIFO with the BG2/BG3/BG4 scroll registers, whereas the mode 7 set shares a latch/FIFO with the mode 7 matrix registers instead.

It's also known thanks to Theme Park that the horizontal scroll registers are also "two in one" in a way: the upper 7 bits and the lower 3 bits can get out of sync if you don't always write twice to each register (this means that BG1HOFS is "three in one": there's the mode 7 value, the mode 0-6 coarse value and the mode 0-6 fine value)

The new discovery thanks to the homebrew Pac-Man is simply that the lower 3 bits of the horizontal scroll registers, which were already known to be independent from the upper bits, also have their own shared latch/FIFO which works exactly like the other two.
tepples wrote:Perhaps it's because on the NES, the fine scroll bits have qualitatively different behavior from the rest of the scroll position, as described in "The skinny on NES scrolling". But as I understand it, mode 0-6 backgrounds on the Super NES instead continuously add the scroll position to the scanline position to form each layer's nametable address and fine scrolls, so this separation isn't quite as necessary. Am I correct in this understanding?
For vertical scrolling this appears to be correct: at the register level there is no distinction between "coarse" and "fine" vertical scrolling, unlike the NES. At the time of that 2005 forum post, Anomie believed that the coarse/fine distinction applied to both BGnHOFS and BGnVOFS, but later someone evidently found a game that was broken if emulated that way (I have no idea what game it was or even whether it was a licensed or a homebrew game) Anomie's 2007 register document correctly indicates that the separate coarse and fine fields only apply to BGnHOFS.
Post Reply