Stuck with PPU emulation (Donkey Kong)

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

User avatar
iOSBrett
Posts: 38
Joined: Wed Apr 18, 2018 1:09 am
Location: Australia

Re: Stuck with PPU emulation (Donkey Kong)

Post by iOSBrett »

Quick Sunday Night (Aussie time) update.

Have gotten attribute tables and palettes to work. Was initially quite hard to work out how to apply the attribute table byte, particularly working out which quadrant I was in when drawing a pixel. I read a post from a couple of years ago that said to check v&2 for horizontal quadrant and v&64 for vertical quadrant. I could only get it to work via checking v&4 for vertical quadrant.

I think this part had the least amount of complete documentation on how to work this out for an emulator. So here is my code for it as I am a better coder than explainer:

Code: Select all

     func drawPixel()
     {
        if cycle > 240 { return }

        let x = cycle
        let y = scanline

        let highTile = UInt8(shiftRegisterTileA)
        let lowTile = UInt8(shiftRegisterTileB)

        let bit = (7 - (cycle-1)%8)
        let index = UInt8((highTile.get( bit:bit) << 1) + lowTile.get( bit:bit))

       // https://wiki.nesdev.com/w/index.php/Nestech.txt#Attribute_Tables
        //        +------------+------------+
        //        |  Square 0  |  Square 1  |  #0-F represents an 8x8 tile
        //        |   #0  #1   |   #4  #5   |
        //        |   #2  #3   |   #6  #7   |  Square [x] represents four (4) 8x8 tiles
        //        +------------+------------+   (i.e. a 16x16 pixel grid)
        //        |  Square 2  |  Square 3  |
        //        |   #8  #9   |   #C  #D   |
        //        |   #A  #B   |   #E  #F   |
        //        +------------+------------+
        //
        //        33221100
        //        ||||||++-- Upper two (2) colour bits for Square 0 (Tiles #0,1,2,3) Top  Left
        //        ||||++---- Upper two (2) colour bits for Square 1 (Tiles #4,5,6,7) Top  Right
        //        ||++------ Upper two (2) colour bits for Square 2 (Tiles #8,9,A,B) Bottom Left
        //        ++-------- Upper two (2) colour bits for Square 3 (Tiles #C,D,E,F) Bottom Right
        //

        // https://forums.nesdev.com/viewtopic.php?f=3&t=14795
        //        (v & 2) is the bit that decides horizontal quadrant.
        //        (v & 64) is the bit that decides vertical quadrant.

        let right = (v & 2 == 2) ? 2 : 0
        let bottom = (v & 4 == 4) ? 4 : 0  // v&64 doesn't work
        let shift = right + bottom

        let attributeIndex = UInt8((attributeTableByte >> shift) & 0x03)
        let color = Palette.color( atIndex:Int(index | attributeIndex << 2))
This gives me the following. (Deja Vu @davecom):
Screenshot 2018-10-20 23.16.23.png
Screenshot 2018-10-20 23.19.00.png
Thankfully I had seen @davecom's post, so checked it again and he said that this was due to reading the Palettes directly without going through the PPU Palette data (0x3F00 - 0x3FFF). So changed the last line to these two lines:

Code: Select all

        let paletteIndex = mapper!.readByte( atLocation:UInt16(0x3F00) + UInt16(index | attributeIndex << 2))
        let color = Palette.color( atIndex:Int(paletteIndex))
and I now have:
Screenshot 2018-10-21 22.14.07.png
Screenshot 2018-10-21 22.15.55.png
Next is to fix the weird scrolling, pretty sure Donkey Kong is not supposed to scroll. Will post a video during the week.
User avatar
samophlange
Posts: 50
Joined: Sun Apr 08, 2018 11:45 pm
Location: Southern California

Re: Stuck with PPU emulation (Donkey Kong)

Post by samophlange »

Coming along nicely! How do you like working in Swift?
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Stuck with PPU emulation (Donkey Kong)

Post by koitsu »

Excellent progress!

Also, I'm really glad my nestech documentation came in handy; again, it's all about how you write/convey something, just raw deep technical details are not always helpful (and that's what makes the Wiki different than training material, but we've touched on this before).

Anyway: something I noticed -- the "discoloured" areas are all areas which have sprites near them. Whether or not that's the cause I don't know, but:

- The stack of barrels in the upper left having a slight discolouration on their right side: Donkey Kong himself is placed directly to their right (there is some overlap, I think)
- The white-bottomed girder/platform next to Donkey Kong houses Paulina
- I can't explain the black-white block floating way off to the right, but maybe it's Mario or the first thrown barrel? Not sure

You can kind of get a feel for this in one of your earlier screenshots.

I'm not saying "go implement sprites" in the least; I'm just looking at the images and thinking what might correlate with the visual anomalies.
User avatar
iOSBrett
Posts: 38
Joined: Wed Apr 18, 2018 1:09 am
Location: Australia

Re: Stuck with PPU emulation (Donkey Kong)

Post by iOSBrett »

@samophlange I love it, it quickly became my favourite language. I was mostly a C based programmer for 25 years, C, C++, Objective-C, and this is by far the language I enjoy coding in the most. Honestly coding a NES emulator in Swift for iOS is kinda dumb, but enjoyable. If I ever get it working I will probably port it to the Apple TV and get some 8bitDO bluetooth controllers. I doubt I will ever really play it on the iPhone or iPad.

@koitsu Oh I didn't notice most of those anomalies, only the ladder one, and the one off in Hblank. Thank you for pointing them out, they are going to be tricky to track down. Is kind of strange that they are all in the Sprite locations as I haven't included even a single like of code around Sprites. Also thanks for linking to the earlier screenshots, this should help in tracking down the issues. The one thing I did notice was that Donkey Kong himself is missing, am I correct in thinking he is a background tile and not a Sprite?

Currently I am looking at the issue shown in the video below, this cycles about every 5 seconds. I don't have a clue what could be going on here, but I haven't done much investigation yet. I have recently implemented horizontal mirroring and it may be something wrong there.
DonkeyKongGlitchShort.mov
(511.18 KiB) Downloaded 10265 times
Have added an animated GIF for those that can't play the mov. I recommend opening it in a new tab or downloading it and viewing it outside the browser, looks corrupted for me in Safari.
DonkeyKongGlitchShort.gif
User avatar
iOSBrett
Posts: 38
Joined: Wed Apr 18, 2018 1:09 am
Location: Australia

Re: Stuck with PPU emulation (Donkey Kong)

Post by iOSBrett »

I removed attribute table code so I could see why the colouring is wrong, as @koitsu pointed out it was wrong before I added that code.

I then compared it to @davecom's and noticed not only was my colouring incorrect, but my girders go edge to edge and my barrels touch the left side of the screen.

Something very wrong going on here.

Daves:
dk2.png
dk2.png (22.61 KiB) Viewed 12570 times
Mine:
Screenshot 2018-10-26 16.29.45.png
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Stuck with PPU emulation (Donkey Kong)

Post by koitsu »

It looks to me like you have either the screen size wrong, or you're doing something with scaling (at the graphical level, i.e. not necessarily PPU or NES emulation-related) that's weird.

The NES "screen space" is always 256x240 when it comes to how much nametable memory there is per screen. That never changes, and technically the NES always works off those sizes (I don't want to get into overscan or PPU rendering or other whatnots). It's 256x240 always.

The only differences there are for what gets visually shown are:

* NTSC vs. PAL. Without getting into a whole bunch of nitpicking and semantics and other nonsense that, IMO, would confuse everyone: NTSC will show 256x224 (with some variance per television; some may lose an additional 8 pixels at the top and bottom due to TV bezels and other whatnots), while PAL will show 256x240. (This is all separate from what goes on under the hood at scanlines outside of the region -- the Wiki has a perfectly fine document on all that hairiness!)

* $2001 bit 1 (background) and bit 2 (sprites): these control, independently, whether or not the furthest-left 8-pixel-column of the screen will show/display each respective thing. The screen doesn't shrink in size (visually or memory-wise) if they're enabled, just that that furthest column won't render either of the aforementioned things

Those aside: I'm including a picture of the render screen of Donkey Kong (screenshot comes from Mesen, but I've nearest-neighbour resized it to 512x480 to make everything more visible), with a 8x8 pink-ish tile grid enabled as an overlay. You should be able to use this to see what should be placed where on the screen, nametable-wise, and compare that to your own emulator.
Sprite-0004.png
Sprite-0004.png (10.06 KiB) Viewed 12553 times
Likewise, here's a picture of the same exact screen, except a blue grid that represents the attribute table sections (16x16 pixels or 2x2 tiles):
Sprite-0005.png
Sprite-0005.png (9.13 KiB) Viewed 12553 times
You have a very weird "offset problem" with your attribute table stuff in some manner; I can't describe or explain because you're the code author and know better than I what you're doing). You can tell because the "brown square" that represents Donkey Kong is offset too far to the left, another oddity between the two ladders to the immediate right of Donkey Kong (where Paulina stands), etc.. It really smells of some problem with a calculation formula. Maybe it's a bug relating not to the PPU calculation formula, but how you're choosing to take that data and render it graphically to a surface/texture/whatever you're doing via some graphics API? Don't know.

If it's of any help, during the attract screen, this is the contents of the attribute table ($23C0-23FF / $27C0-27FF since the game uses horizontal mirroring) so that you can compare it to your own:

Code: Select all

FF FF FF FF FF FF FF FF 55 AA 22 00 00 0F 0F 0F
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
It's also hard to get complete precision from screenshots when you're using some kind of resizing/scaling that looks like bilinear interpolation (the hardware or graphics system may be doing this itself natively), so everything looks fuzzy -- though this problem is "large" enough (visually) that it can be seen, so using nearest-neighbour wouldn't solve the problem. The reason I mention this: when I look at your screenshots, they're 530x528, rather than something precise like 256x240 or 512x480, and there's also that orange area outside the visual space; I believe you're using that to represent the actual overscan areas in some manner, but I'm not sure.

I really think a lot of this could be alleviated if you had an emulator with good debugging capabilities to compare with. It might be worth your time to set up a simple VM (virtual machine) running Windows 7 SP1 or newer, via something like VirtualBox, just so you could use Mesen or FCEUX. I think it would help in your troubleshooting; Mesen in particular gives a lot of useful internal data in the Debugger window, including PPU internals, as well as detailed visuals (including what's happening inside VBlank etc., via it's Event Viewer -- a feature that still blows my mind).
User avatar
davecom
Posts: 39
Joined: Mon Jul 16, 2018 2:57 pm

Re: Stuck with PPU emulation (Donkey Kong)

Post by davecom »

iOSBrett wrote:I removed attribute table code so I could see why the colouring is wrong, as @koitsu pointed out it was wrong before I added that code.

I then compared it to @davecom's and noticed not only was my colouring incorrect, but my girders go edge to edge and my barrels touch the left side of the screen.

Something very wrong going on here.

Daves:
dk2.png
Mine:
Screenshot 2018-10-26 16.29.45.png
Hi Brett,

I had a similar off-by-one error. I was missing scanline 261 and counting cycle 0 when I shouldn't for drawing. Here is the commit that fixed it for me:
https://github.com/davecom/DDNES/commit ... d06dbcc944

Hope this helps,
Dave
User avatar
iOSBrett
Posts: 38
Joined: Wed Apr 18, 2018 1:09 am
Location: Australia

Re: Stuck with PPU emulation (Donkey Kong)

Post by iOSBrett »

I have fixed a few of my issues. The size problem was a combination of two issues. In my drawPixel method I was exiting early if the cycle was 240 (oops that is the visible scanline limit, not the visible cycles). This gave me the two extra tile widths I was missing, however everything was still shifted too far left. This was because I was not buffering the data I was reading into any shift registers. Early on I mistakingly thought that I wouldn't need shift registers if I wasn't supporting scrolling yet, however not using them means the data gets used a tile too early. I also had a few issues similar to the one pointed out by @davecom, I have fixed these too and have a full 256x240 resolution. This has now fixed my layout (mostly).

I still have the colour issues, the really weird scrolling thing (not sure where to start on that one) and a duplication of the first two columns. I will need to improve my tools before progressing .

I am using nintanco emulator with a debugger but I don't think it is nearly as good as Mesen, however I don't really want to shell out for a copy of windows and I don't pirate software, even from Microsoft. However I am getting to the stage where I may have to buy it instead of relying on you guys to give me the data I need. First I might improve the debugging in my own Emulator, especially around the PPU. I already have a debugger built into my CPU which can pause, step, continue, see the stack and the registers and all CPU memory. I think I need to enhance it to see contents of name tables and attribute tables. No use in running Mesen if I don't have an easy way to compare it with what I have.

In regards to the weird size screenshots I was uploading. I am running on an iPhone simulator on my laptop and this does scale the screen. I was then just doing a free hand selection of a section of the window and uploading that. This was because I was lazy and hadn't implemented any way to save my pixel buffer to disk. I have done this now, I have also doubled the pixels in both direction before saving, I did this because @koitsu's screenshots of Mesen which he resized are way easier to see what is going on.

Better screenshots:
Title Screen.png
Attract Screen.png
Bad Colours.png
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Stuck with PPU emulation (Donkey Kong)

Post by tepples »

iOSBrett wrote:I am using nintanco emulator with a debugger but I don't think it is nearly as good as Mesen, however I don't really want to shell out for a copy of windows
Mesen debugger works in Mono under popular GNU/Linux disributions. I think as of right now, for running Mono applications, macOS is the odd one out. How much did you "shell out" for macOS, with respect to the price difference between a Mac and an entry level non-Apple PC?
iOSBrett wrote:and I don't pirate software
What'd you use to dump your Donkey Kong Game Pak? Does INL-Retro (either the old Kazzo based version or the new version) work well on macOS?
User avatar
iOSBrett
Posts: 38
Joined: Wed Apr 18, 2018 1:09 am
Location: Australia

Re: Stuck with PPU emulation (Donkey Kong)

Post by iOSBrett »

tepples wrote:
iOSBrett wrote:I am using nintanco emulator with a debugger but I don't think it is nearly as good as Mesen, however I don't really want to shell out for a copy of windows
Mesen debugger works in Mono under popular GNU/Linux disributions. I think as of right now, for running Mono applications, macOS is the odd one out. How much did you "shell out" for macOS, with respect to the price difference between a Mac and an entry level non-Apple PC?
Not really interested in these old Windows vs Mac arguments.
tepples wrote:
iOSBrett wrote:and I don't pirate software
What'd you use to dump your Donkey Kong Game Pak?
Fair point, I'll give you that one :-D
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Stuck with PPU emulation (Donkey Kong)

Post by tepples »

iOSBrett wrote:Not really interested in these old Windows vs Mac arguments.
Then don't use Windows. I'd be interested to hear what breaks if you install a VM in macOS, and install Xubuntu, Mono, and Mesen in the VM.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Stuck with PPU emulation (Donkey Kong)

Post by koitsu »

Banshaku's already covered this subject in a couple threads recently: viewtopic.php?p=225891#p225891

I just tend to recommend using VMs because they're generally easier and you don't have to waste time futzing around.
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Stuck with PPU emulation (Donkey Kong)

Post by tokumaru »

iOSBrett wrote:I don't really want to shell out for a copy of windows and I don't pirate software, even from Microsoft.
I don't know if this is still the case, but last time I checked, you could use Windows 10 indefinitely without activating it, with minor restrictions, completely legally (source).
User avatar
iOSBrett
Posts: 38
Joined: Wed Apr 18, 2018 1:09 am
Location: Australia

Re: Stuck with PPU emulation (Donkey Kong)

Post by iOSBrett »

I have been stuck on the same issue for almost 2 months now and am not making any progress. If anyone has any pointers I would greatly appreciate it.

I have been comparing my code with @davecom's NES emulator as that runs nicely on the Mac, the code is very easy to read, and he is not yet up to implementing sprites (neither have I). I have even been reading through the annotated Assembly code of Donkey Kong.

My issue is that when my Emulator is writing the background for the Kong animation in the attract screen to the Nametable/Attribute table it is becoming corrupted (See October post with video). I thought this was due to the fact that the rendering (2001 bits 3 and 4) is not being switched off during writing to 2006/2007. When I only allow writes to 2006/2007 when rendering is off then most of the corruption of the name tables doesn't occur and the scrolling also doesn't occur. To me this makes sense as both the writes to 2006/2007 and the rendering effect the VRAM address. However putting in this "fix" also means that Kong is never shown.

However when I debug @davecom's emulator I also see that he is writing to 2006/2007 when rendering is enabled and he has no corruption.

Also looking at the Donkey Kong code it looks like rendering is not always switched off when writing to 2006/2007.

Can anyone clarify what should happen when writing to 2006/2007, should rendering always be off? Also can anyone think why this issue may be happening in my Emulator, am I missing something obvious?

UPDATE:
The difference between @davecom's and mine is that his writes to 2006/2007 are happening in Vblank and mine are not. Very strange as I can see my NMI is triggered at the start of Vblank. Not sure why the NMI routine is taking so long to execute.
User avatar
iOSBrett
Posts: 38
Joined: Wed Apr 18, 2018 1:09 am
Location: Australia

Re: Stuck with PPU emulation (Donkey Kong)

Post by iOSBrett »

I fixed it. I wrote out a long explanation and when I uploaded some screenshots I got logged out and lost it.

So the short explanation. My PPU code was fine (well almost fine, still has attribute table issues). The issue was a typo in the CPU code, I was adding the number of cycles + page boundary cycles to my LUT instruction instead of a variable named instructionInfo that is used to pass information back to the main emulator loop. This number was increasing over and over, so instead of passing back a number of cycles in the range 3 -> 7, I was passing back numbers that reached the thousands.

This number was used to perform PPU catchup, so my PPU was running way ahead of my CPU after even a few seconds. I fixed this and my writes to the name table are now done in vertical blank as they should be.

Have learnt heaps about the Donkey Kong code while fixing this bug so that is an upside, also my Mac is not running as hot. The downside is that my frame rate dropped dramatically on the simulator. I should really spend a couple of hours and learn how to create a Mac App as @davecom has done, would speed up development a lot.

Here are some screenshots with my progress:
Simulator Screen Shot - iPad Pro (10.5-inch) - 2018-12-19 at 12.05.36.png
Simulator Screen Shot - iPad Pro (10.5-inch) - 2018-12-19 at 12.09.21.png
DK1.png
DK2.png
Post Reply