Hblank on line 153?

Discussion of programming and development for the original Game Boy and Game Boy Color.
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Hblank on line 153?

Post by Near »

So I recently worked on emulating the GBVideoPlayer.

Source: https://github.com/LIJI32/GBVideoPlayer
Result: https://twitter.com/byuu_san/status/686436982407315456

Aside from obviously needing a pixel-based renderer (already had one), the trick seemed to be that it was expecting 145 Hblank IRQs to fire. And if you were one short, it'd fetch bad control bytes and go completely haywire, displaying nothing but gibberish.

Obviously, lines 0-143 trigger Hblank. But upon adding another to line 153, on the theory of it being a way to set up special effects for the first scanline, the video began to work correctly.

But it kind of bothers me that gambatte doesn't emulate this ROM. The gibberish in it is identical to what I get without the extra Hblank at line 153. However, the video ROM has been shown running on a real GBC, so obviously it's an emulation issue.

So, does anyone know anything about this? Can anyone confirm the existence of an extra Hblank IRQ?
Shonumi
Posts: 342
Joined: Sun Jan 26, 2014 9:31 am

Re: Hblank on line 153?

Post by Shonumi »

Perhaps this is related to some undocumented (or rather, super poorly documented) behavior I stumbled upon while debugging Line 0 LYC-STAT IRQs. Apparently, Line 153 can be treated as Line 0 (I assume at this time, the LCD has actually returned to the top of the screen), and this is the appropriate time to fire LYC = 0 STAT IRQs. Without it, my IRQs always fired late, causing issues like these for me and other devs.

So, essentially, what we call Line 153 is Line 0, but technically still in VBlank state, I think. After Line 153/Line 0 runs for 456 cycles, it exits VBlank, leaves LY at 0, then returns to Mode 2, so it "appears" as if Line 0 is run twice. This is the only thing I can come up with that makes sense from a hardware point of view, and it works in emulation. It's easy to verify on HW, just haven't gotten around to it lately.

Anyway, here's my theory. What if we suppose that the LCD controller just does something else weird on Line 153/Line 0? What if it, even though it's in VBlank, it treats Line 153 as if it were some sort of dummy Line 0, e.g. triggering STAT and HBlank IRQs? From what I can see, we can already assume that such a dummy Line 0 triggers STAT IRQs in commercial games, so what's to stop it from doing HBlank IRQs as well? byuu, if you want, I can make a quick test ROM and verify it on real HW. Actually, I'll just do it now since I'm off today :P
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Hblank on line 153?

Post by tepples »

Having this question in the back of your mind might help you come up with hypotheses: When does it fetch the pattern data for a sprite on the top scanline of the screen?
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Re: Hblank on line 153?

Post by Near »

> Perhaps this is related to some undocumented (or rather, super poorly documented) behavior I stumbled upon while debugging Line 0 LYC-STAT IRQs. Apparently, Line 153 can be treated as Line 0 (I assume at this time, the LCD has actually returned to the top of the screen), and this is the appropriate time to fire LYC = 0 STAT IRQs. Without it, my IRQs always fired late, causing issues like these for me and other devs.

Well, I certainly agree that you want to be able to do effects on line 0. The SNES has an entire "dummy" scanline at line 0 for that exact purpose, so the behavior makes sense. I can see the Game Boy kind of messing up this clarity by calling SNES 0 == GB 153, SNES 1 == GB 0, etc.

What I'm curious about here is whether or not the real LY=0 line triggers these STAT (Hblank, etc) IRQs then. My guess is it'd have to.

Thus, you'd have them firing on LY={153,0,1,2,3,...,143}; or 145 times per frame. The final 143 one is rather useless. However, without firing exactly 145 Hblank IRQs, this GBVideoPlayer doesn't work.

> After Line 153/Line 0 runs for 456 cycles, it exits VBlank, leaves LY at 0, then returns to Mode 2, so it "appears" as if Line 0 is run twice.

Are you sure it's not meant to allow start/end effects on each line of the display, rather than "running twice"?

It feels more likely to me that it's intentional and not a duplication. And line 153 is simply just not rendering at all.

> What if it, even though it's in VBlank, it treats Line 153 as if it were some sort of dummy Line 0, e.g. triggering STAT and HBlank IRQs?

Exactly what I'm thinking.

> byuu, if you want, I can make a quick test ROM and verify it on real HW. Actually, I'll just do it now since I'm off today

Awesome, thank you very much :D

I don't really know LR35902 assembler syntax, don't have a cross assembler table for the syntax, etc. So I'm pretty useless for test ROMs right now, unfortunately. I get the feeling that'll probably change in the future. Like you said in your Github commit comment ... DMG documentation is just pitiful. There's really not much choice but to break out the hardware if you want to write a good emulator.
Shonumi
Posts: 342
Joined: Sun Jan 26, 2014 9:31 am

Re: Hblank on line 153?

Post by Shonumi »

I'm borrowing your style of quoting (it's just so much quicker!)

>Are you sure it's not meant to allow start/end effects on each line of the display, rather than "running twice"?
>
>It feels more likely to me that it's intentional and not a duplication. And line 153 is simply just not rendering at all.

The thing is, I believe the LY counter simply reads as 0 on Line 153 and Line 0, that's why I say Line 0 "appears" to run twice. It'd technically be Line 153 as far as the LCD itself is concerned. But during those 912 cycles (from Line 153 start to Line 0 end), LY would be 0, so to GB software it might look like Line 0 takes twice as long to finish. Like I said, I haven't had time to test it in my GBs. I debugged this behavior by observing what BGB did. In software at least, that's how BGB interprets hardware behavior, and it several commercial games appear to rely on this trick.

> I don't really know LR35902 assembler syntax, don't have a cross assembler table for the syntax, etc. So I'm pretty useless for test ROMs right now, unfortunately. I get the feeling that'll probably change in the future. Like you said in your Github commit comment ... DMG documentation is just pitiful. There's really not much choice but to break out the hardware if you want to write a good emulator.

Well, up until recently, I had been doing nearly all of my test ROMs in a hex editor + a text file to keep track of stuff (not kidding). Recently made my own minimal assembler that's only barely a step up from that. I call it LAZ80. It's a pun on my laziness to properly parse strings. The only thing it actually reads are hex values prefaced by "!" or "$", then builds the ROMs bit by bit. Basically the whole GB binary with comments, all in a text file. I have separate graphics tools for pushing data to ROMs though.

Anyway, I made this quickly. Source is attached as well. I tried to count how many HBlank interrupts occurred in a single frame. Code should start counting just as it enters VBlank, so if an HBlank interrupt is fired on Line 153, it should technically catch it. Unfortunately, ran it on my GBC, and it gave me a negative result (the ROM prints a giant "N" for only 144 HBlanks, and a giant "Y" for 145). So, as long as the code does what we think it does, it says only 144 HBlanks are fired. If you run it higan, you should see a Y, but apparently that's not supposed to happen :(

So where to go from here now?

EDIT - Perhaps there is a pending HBlank IRQ somewhere in VBlank for GBVideoPlayer? If the VBlank IRQ is serviced first and finishes, the pending STAT interrupt could still be triggered towards the end of VBlank. It would still have total access to VRAM at that point too.
Attachments
hblank_count.txt
(3.77 KiB) Downloaded 289 times
hblank_count.gb
(32 KiB) Downloaded 304 times
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Re: Hblank on line 153?

Post by Near »

> I'm borrowing your style of quoting (it's just so much quicker!)

Isn't it, though? :D

Worked for years with e-mail. But phpBB made everything a pain to do.

Whenever I get around to finishing that custom forum of mine, I'm definitely going to parse them into block quotes. And "<name> foo" for named quotes.

> The thing is, I believe the LY counter simply reads as 0 on Line 153 and Line 0

... that would be crazy. I'd very much like to see a test around this.

> So where to go from here now?

You may be right. I tried firing the Hblank on line 153 and NOT on line 0, and the GBVideoPlayer works. Of course, it could also be that line 143 -doesn't- fire an Hblank IRQ (why would it? You aren't going to be doing any scanline-level effects when line 144 doesn't render anything.) I'm inclined to believe the latter is correct, because it's a lot more sensible.

Doesn't matter whether I return 0 from $ff44 (LY) for line 153 or not, it still works with just the first change.

I would say next up is we'd want to just have a dumb loop that reads LY and writes it to save RAM. Fill the entire save RAM with a dumb loop. Then we can see if 0 appears twice as much as other values, and if 153 is indeed missing.

It'll be trickier to time the Hblank IRQs, but I think it's doable: have a main loop that just keeps incrementing a 16-bit number. Every time Hblank fires, write the value to RAM. As long as we have two frames worth, we should be able to determine where the Hblank events are firing. If we see a gap in one value that's twice as wide, we confirm the theory for LY=0 skips Hblank. If not, I believe that'll confirm it's LY=143 that's missing. And our emulators should get relatively close to the same numeric values as the save RAM files from real hardware, too.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Hblank on line 153?

Post by tepples »

byuu wrote:> I'm borrowing your style of quoting (it's just so much quicker!)

Isn't it, though? :D

Worked for years with e-mail. But phpBB made everything a pain to do.

Whenever I get around to finishing that custom forum of mine, I'm definitely going to parse them into block quotes.
Use CommonMark instead of BBCode, and you get that for free. Reddit, Discourse, and Stack Exchange do this, for instance.
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Re: Hblank on line 153?

Post by Near »

> Use CommonMark instead of BBCode, and you get that for free. Reddit, Discourse, and Stack Exchange do this, for instance.

That's most likely what I'm going to do. My website runs on DML, which is basically simplified CommonMark. But I'll need to add some specialization for eg "<name> quote" style.

I'll have to make it an option, though. Some people don't like CommonMark.
Shonumi
Posts: 342
Joined: Sun Jan 26, 2014 9:31 am

Re: Hblank on line 153?

Post by Shonumi »

Well, I must say byuu, you certainly know how to entertain people with mysteries :)

I think I've cracked this one though. It's not HBlanks you want to go after, but you're close. You still want to target STAT IRQs, but you want the OAM ones (with Bit 5 set in the STAT register). If you run the ROM of the Pokemon anime intro, you'll see it constantly uses this interrupt. Here's what I observed through BGB (it's one of the few emulators that "works" with it) and I noticed that OAM-STAT IRQs were being fired on Lines 0 through 144. That's up and including Line 144. That means the demo in question runs and services that interrupt 145 times.

Curious, I tried to make a test ROM that replicates and demonstrates that behavior. Like before, it counts every instance of a certain interrupt (in our case OAM-STAT ones). It runs one frame (until VBlank) then draws a "Y" for 145 successful instances, or "N" for only 144 instances (or a blank screen if something entirely different happens). Initially, I used VBlank IRQs to stop the test from counting. Strangely, I kept getting the same results as before. Real hardware showed only 144 instances of the OAM-STAT interrupt firing.

However, I saw that the GB Video demo does not use VBlank interrupts, so I wondered if that was somehow blocking the 145th OAM-STAT interrupt in my test from firing. Rewriting the test to read LY manually, suddenly the 145th OAM-STAT triggered! I verified this in BGB (displays the "Y"), VBAM (displays the "N" and runs like garbage on the GB Video demo), my own GBE+ (displays a blank screen, oops...), and finally my GBC (displays the "Y").

So here's what it looks like. OAM-STATs can fire 145 times a frame, from Lines 0 to 145. Even though Line 145 is technically in VBlank, perhaps the LCD hardware hasn't had enough time to tell itself to stop searching OAM and generating interrupts? I can't say, but I don't believe Lines 146 - 153 trigger STAT interrupts of any sort. What's more, this behavior is only observable when a VBlank interrupt does not occur. As soon as a VBlank IRQ is fired, the LCD fails to fire an OAM-STAT interrupt on Line 145; you don't even see the interrupt as pending in the IF register. I'm really interested in what logic the LCD controller is following internally that makes it go this route.

Here is the new source code and the new test ROM. Like I said, I already verified the result on real hardware, but please double check the code to make sure it does what we think it does. At any rate, I'm pretty sure we've pegged another undocumented quirk about the GB. Again, apparently some emu authors (*cough*BGB*cough*) knew about this stuff ages ago, but failed to enlighten anyone else...
Attachments
hblank_count_v2.gb
(32 KiB) Downloaded 312 times
hblank_count_v2.txt
(3.8 KiB) Downloaded 275 times
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Re: Hblank on line 153?

Post by Near »

Thanks, this seems like we're definitely on the right track here! Still lots of questions remaining, though.

What about LYC interrupts? Do they affect this behavior at all if they do/don't trigger?

Does this behavior happen only for the STAT-VBLANK interrupt, or the regular VBLANK one?

Does the display enable flag matter at all for this? (Do all STAT interrupts require the display to be enabled?)

What about the "read LY as 0 on LY=153" behavior?

Does a LYC of 0 trigger on LY 153 only, or on both 153 and 0? Testing Kirby's Dreamland 2, both work.

> Again, apparently some emu authors (*cough*BGB*cough*) knew about this stuff ages ago, but failed to enlighten anyone else...

No surprise there. What else would you expect from someone that hides their source?
Shonumi
Posts: 342
Joined: Sun Jan 26, 2014 9:31 am

Re: Hblank on line 153?

Post by Shonumi »

> What about LYC interrupts? Do they affect this behavior at all if they do/don't trigger?
>
> Does this behavior happen only for the STAT-VBLANK interrupt, or the regular VBLANK one?

If you had an LYC interrupt on line 144, you would have to read the STAT register to tell if an LYC interrupt would technically block the OAM interrupt like regular VBlank interrupts do. OAM and LYC interrupts trigger at the same time from what I understand, so you couldn't use a simple counter like this test does. You'd have to see what bits get flipped in STAT itself. It's something I'll keep in mind when I work on the next versions.

Now about other STAT interrupts (VBLANK especially), I don't know. I only tested OAM interrupts. I would like to know if I enabled STAT-OAM and STAT-VBLANK, would it still hit 145 instances for the STAT interrupt? That is to say, is the STAT-VBLANK just like the regular VBlank interrupt, blocking the 145th instance? If so, we can say there's something weird going on with the GB's LCD and how it checks VBlank interrupts of any kind. Ultimately, what I'd like to know for sure is, what kinds of STAT interrupts can occur on Line 144. I'll leave that for tomorrow though. Today's been pretty exciting already :D

> Does the display enable flag matter at all for this?

Pretty sure the LCD doesn't clock any cycles when it's turned off. Maybe I misremembered something, but that's what I recall. In that case no LCD interrupts can be triggered, so I don't imagine it would affect us here. Just have to be careful when counting lines because LY is reset to 0 again when switching the LCD off then on.

> What about the "read LY as 0 on LY=153" behavior?

I'll still have to check that out. I suspect that "dummy" Line 0 is what happens on a real GB. Tomorrow I'll make a new test ROM specifically for that stuff.
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Re: Hblank on line 153?

Post by Near »

> Pretty sure the LCD doesn't clock any cycles when it's turned off.

... holy shit. That would be absolutely chaotic if true.

You could effectively control the refresh rate completely with that.

I know Nintendo disallows turning it off while the screen is rendering. But if you turn it off in Vblank, and then wait for one full frame, you've turned your Game Boy into a 30fps display.

It would absolutely require an adaptive sync monitor to emulate. And even then, you could easily push it below the minimum value they support (not that any game would.)

I can easily emulate it under the caveat that you'll never get smooth scrolling, but ... yuck.

Would love a test ROM to confirm this, if you don't mind.
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Hblank on line 153?

Post by Dwedit »

Nintendo DS can also have its scanline reset during vblank time.
And some GBC games did turn off the screen during render time, like Conker's Pocket Tales.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Hblank on line 153?

Post by tepples »

Dwedit wrote:Nintendo DS can also have its scanline reset during vblank time.
As I understand it, this is intended for synchronizing multiple systems in WLAN multiplayer where one system's crystal may be slightly faster or slower than another's.
And some GBC games did turn off the screen during render time, like Conker's Pocket Tales.
GBC also had a radically different screen technology from the monochrome Game Boy. I wonder if ability to safely turn off rendering was one of the changes between the Game Boy's screen and the GBC's.
Shonumi
Posts: 342
Joined: Sun Jan 26, 2014 9:31 am

Re: Hblank on line 153?

Post by Shonumi »

@tepples - I think you're correct on both accounts. Echoes what I've looked into.

@byuu - Not enough time to make the rest of the Line 144 STAT IRQ tests, but I managed to replace OAM-STAT interrupts with HBLANK-STAT interrupts. Even with VBlank interrupts (regular and STAT based ones) disabled, HBlank never fires on Line 144. It seems OAM-STAT is a bit of an oddball. I still have to test it to see if VBLANK-STAT interrupts mess it up like regular VBlank interrupts. I think the 145th OAM-STAT interrupt probably triggers (when in theory it shouldn't) because of how early in a scanline it's supposed to fire. Like I said, I think the LCD controller doesn't properly recognize it's in VBlank as soon as it enters Line 144, so it gives OAM-STAT interrupts the OK. HBLANK-STAT interrupts happen later in the scanline, so maybe by that time the LCD controller correctly figures out VBlank is taking place. Just a theory, since I can't see what's going on with the actual circuits and whatnot.

Anyway, I did have time to get the LCD clocking demo done. Here's how it works. It creates 2 16-bit counters (C1 and C2), then increments them in two loops. The test waits for Line 152 (tail end of VBlank), then it shuts off the LCD (which resets LY to 0). The first loop updates C1 for 0x390 times (taking 32 cycles each time, so approximately 72 scanlines worth of looping). The values are written to cart RAM, then LCD is turned on. Now the next counter, C2, increments in a loop (also 32 cycles long) all the way until it reaches VBlank. On VBlank, it writes the counter values to RAM, then loops endlessly.

The idea is, if the LCD isn't clocking at all, LY isn't incremented at all during this period. If the loop for C1 finishes and the LCD doesn't clock anything, when it turns the LCD back on, LY is still 0 and the LCD starts clocking with a fresh start. In that case, when C2 counts up, it will be noticeably bigger than C1 because it has to run for all 144 scanlines.

If the LCD is clocking when turned off, however, it should still update LY. When C1 finishes its loop, LY should be 72 when the LCD is turned back on. Since C2 counts until the next VBlank, its value should be nearly equivalent to C1 (since both are supposed to run for 72 scanlines).

I tested it on my GBC, but the results indicate the former rather than the latter. The LCD doesn't clock cycles when its turned off. At least, it doesn't do anything with them. I'll leave the source code along with the ROM + save file. C1 is 0x1 - 0x2 of the save (MSB first), and C2 is 0x3 - 0x4 (again MSB first). C1 will always be 0x390, but real hardware shows C2 being 0x720 (so exactly 2x C1, meaning it ran for all 144 scanlines). So... what this means is you can delay how long you leave the LCD off for an arbitrary amount of time. When you turn it on, it always starts fresh from zero. Chaos as you said ;)
Attachments
lcd_clock_sav.txt
Can't upload .sav files, so just rename this.
(8 KiB) Downloaded 295 times
lcd_clock.gb
(32 KiB) Downloaded 294 times
lcd_clock.txt
(3.23 KiB) Downloaded 264 times
Post Reply