The state of PowerPak mappers

Discuss hardware-related topics, such as development cartridges, CopyNES, PowerPak, EPROMs, or whatever.

Moderators: B00daW, Moderators

User avatar
koitsu
Posts: 4217
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

The state of PowerPak mappers

Post by koitsu » Tue Feb 19, 2019 3:20 am

Is anyone here up-to-date on the current "state" of PowerPak mappers? The Wiki page on the subject is pretty good about referencing all the forks and variances, but assisting with a project recently that uses MMC3/mapper 4 caused me to have to deal with variance in behaviours and left (again) a bitter taste in my mouth. I'm just sort of ranting and will get to the point eventually -- but I don't even know where to start...

It doesn't seem like any of the individuals who have worked on these mappers collaborate, so we have literally 3 forks of the base/stock mapper sets, with each fork having fixes/tweaks/whatever (they're rarely well-documented) and behave differently, in addition to more one-offs from other users/developers. Some, like PowerMappers, implement whole new features (save states); others, like loopy's, are just a nebulous .zip file with no versioning so you have no way of knowing if you have the "latest version" or not.

End-users have absolutely no idea which of those mapper sets should be used, how they should be used (I've been extracting the stock set, then extracting loopy's on top of that + overwriting, then extracting PowerMappers on top of that + overwriting, then adding in whatever other mappers are found/discussed/whatever).

Furthermore, some sets (ex. loopy) are basically complete re-release of the stock set with miscellaneous files changed. Has anyone looked into the stock set .zip file? There's all sorts of (IMO) crap in there -- .txt files (some of these look very important, and are in binary (!!), others look just like readmes, others are empty but probably important?), .pdf files, a .jpg showing a resistor mod but without any text or pin descriptions or even if the console was a top-loader or front-loader, an .smc ROM (yes, for the SNES!), and even a .nes ROM. You can't even tell what goes where, you're just told "put all this in your POWERPAK directory on the CF". There's no way for anyone to know how to clean this up without knowing all about it.

I know nothing of VHDL, and very little of FPGAs or CPLD devices, so I really can't help here. I'm looking at it purely from the perspective of an end-user with a flash cartridge used for development/testing.

Is there some way to consolidate all of these? Is that even the right process? Furthermore, where is the source code for PowerMappers v23, and most of the other one-off mappers? Why isn't this stuff consolidated and stored in, say, GitHub with release builds made for everything, with everyone working together -- even if all just working off the master branch? Does anyone maintain their own "merge" of all of these somewhere, with proper versioning of releases, so users know what they've got? If not, why has none of this stuff been handed back to Bunnyboy for review/consolidation?

Like I said: this is kind of a rant, but the above really is the state of all of this. The EverDrive N8 isn't much different in this regard either, from what our Wiki page says.

Why does it matter? Well, when it comes to testing software like I am, it absolutely matters. For example, I definitely need to know if quirks/issues I'm seeing when testing a NES game with a particular mapper are due to a specific mapper pack -- and whose (and what version/release of that pack) -- so that I can report it back to the NES game developer so they can include in their readme/docs something like "PowerPak users should avoid blah blah pack v4.66 else the game will crash, please use v4.67 instead" etc..

User avatar
rainwarrior
Posts: 7803
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: The state of PowerPak mappers

Post by rainwarrior » Tue Feb 19, 2019 3:59 am

This has been the "standard" for a long time, and it's actually what Bunnyboy would tell you to do when asked:

1. Take the last official mapper set.
2. Put loopy's set on top of this, overwriting any files with the same name.

If you want savestate support, TheFox's mapper set is another overlay step:

3. Put TheFox's set on top of that. (Then do some extra setup for savestates, it's in the readme.)


All of the the other mappers are miscellaneous things that don't overlap with those three.


There's some difference between how TheFox implemented MMC3 and how Loopy did, but I don't know what the difference is. I think it's mainly to do with the scanline IRQ.


Loopy released source of some of his mappers, but not all. TheFox didn't want to release any source of his. Bunnyboy released a few example sources. This is all linked at the bottom of the wiki page.

The complaints you're making are things I (and I think others) have asked of these people multiple times in the past. Nothing was done about them. Eventually I started collecting mapper information on the Wiki page so I could have something to refer to.


There were other versions released of all 3 of the major sets, I think, but only the most recent versions can really be found anywhere at this point. The old versions were more or less universally worse than the last version of each of them. All three of them have been several years since the last change, so there's not really much instability there. TheFox's are 3 years old, the other two are much older.

User avatar
rainwarrior
Posts: 7803
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: The state of PowerPak mappers

Post by rainwarrior » Tue Feb 19, 2019 4:22 am

Everdrive N8 is in a less fragmented state. Krikzz kept updating it for a while, stopped about 2 years ago I guess, but he may resume at any time. He provided example source and documentation which was IMO much better than what we had for PowerPak, and at least while he was updating it he would incorporate mappers contributed by others. (Also the Everdrive's USB upload interface is much better for testing mappers.)

User avatar
Banshaku
Posts: 2334
Joined: Tue Jun 24, 2008 8:38 pm
Location: Fukuoka, Japan
Contact:

Re: The state of PowerPak mappers

Post by Banshaku » Tue Feb 19, 2019 8:56 am

So the scanline IRQ could behave differently based on the mapper version used... That is "the" important information I needed to confirm for my current project. Thanks for confirming that part, it could be the cause of my current issue ;)

I may have to test is with my real devcart after all to make sure it's not the mapper IRQ implementation that could be causing the issue. What you just mentioned is now invaluable for my raster tests since it could put doubt that the test on the powerpak may be the cause after all.

Thanks again!

User avatar
koitsu
Posts: 4217
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: The state of PowerPak mappers

Post by koitsu » Tue Feb 19, 2019 1:31 pm

Thanks for the info.

Sad.

User avatar
rainwarrior
Posts: 7803
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: The state of PowerPak mappers

Post by rainwarrior » Sun Mar 03, 2019 9:35 pm

If you specifically want to know about MMC3, that's one of the ones Loopy did release source for:
viewtopic.php?p=173302#p173302

The PowerMappers do not have source, but there is a description in its readme about its method:
https://fo.aspekt.fi/
New MMC3 IRQ implementation (RAMBO-1 style, based on M2). Doesn't exactly match real MMC3 timings, but should be more robust than the previous implementation.
Presumably it refers to:
https://wiki.nesdev.com/w/index.php/RAM ... _operation

How close either of these is to the real MMC3 is an open question. I'm not much of an expert on the fine details of MMC3 interrupts, and it's been the subject of much confusion in general over the years. I think there may be different variants that are mostly compatible, except for a few games which depend on very specific timing. If you're doing development of a new game all I'd really recommend is test with many emulators and try to leave yourself as much margin for error as you can. I think in most cases it's very possible to write robust interrupt handlers that don't really care about these minute differences.

User avatar
koitsu
Posts: 4217
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: The state of PowerPak mappers

Post by koitsu » Sun Mar 03, 2019 11:50 pm

Thanks rainwarrior. Some context as to why this came up -- and apologies for the length:

I've been privately testing some stuff for Banshaku. The behaviour I see varies between each set (stock vs. loopy vs. PowerMappers). I did the testing myself and made videos for him (sorry, again they're private), including testing between NES vs. AVS. Sadly I don't have an actual MMC3B devcart (I do have an EPROM/EEPROM writer) so I can't test on an actual MMC3B cart, but I think InfiniteNesLives apparently tested it on hardware and found that there's a slight difference there too (it would really help if I could just test this all myself, haha). I'm being vague here when I keep saying "behaviour" and "difference" so I'll try to explain:

This particular code uses the MMC3 IRQ counter, triggered multiple times in a frame (more on that in a second). There is some nop timing involved to try and get certain things to happen at certain times. An important piece of the puzzle is that this isn't using the fire-the-IRQ-every-scanline technique by writing $00 to $C000. It's actually firing at a certain scanline/point, being acknowledged, doing some nop timing, followed by a CHR-ROM bank switch, followed by setting up the scanline counter to run again within the same frame several scanlines later, rinse lather repeat. To complicate matters, there are several different IRQ handler routines (i.e. the IRQ vector points to RAM, and the game changes it in real-time -- including some of the IRQ routines changing it themselves for the next subsequent firing). Sprites are not tweaked/involved during this operation.

I'll also add that some of these anomalies -- but not all -- actually show up in some emulators like Nestopia and NintendulatorDX 0.975, but not Nintendulator on Linux or OS X (I forget which), and not in emulators like Mesen. Everything varies massively!

The anomalies seen are graphical, and usually manifest as a partial scanline (maybe 16 pixels worth) showing an abnormal colour/line, often varying (horizontally) in size every frame. I might be able to make an animated GIF of this, or maybe a small cropped video of what they look like. Each powerpak set seems to vary slightly in where it happens, etc..

What's interesting is that "where" the anomalies happen (on some of the powerpak sets) correlate on-screen with events found using Mesen's Event Viewer. The "length" of the aberration, visually, also correlated with the number of PPU cycles (about 12, which is about 4 CPU cycles). My gut feeling was "this is PPU register tweaking causing the problem" (e.g. the whole $2000/2002/2005/2006 thing), but nope, it doesn't seem to be.

I was able to track down the responsible code to an sta $8000 -- that's 4 CPU cycles -- in one of the IRQ routines, which is just selecting a CHR-ROM bank via MMC3, specifically changing what part of PPU RAM addressing maps to what CHR-ROM bank. Like I said -- the visual anomaly doesn't happen in Mesen, so I can't use it to figure out if, say, it only happens when the value written to $8000 changes -- so it's difficult to determine exactly. Hardware ICE or maybe an oscilloscope would be able to figure stuff out exactly, but that's getting into serious hardware territory where we'd need someone like Ben Boldt hooking up tons of probes to address lines on the PPU to figure out what's happening when, hahaha :-) Way outside of my skill set.

All this made me wonder: is it possible that tweaking $8000 could cause some on-screen anomaly of this fashion -- a sequence of incorrectly coloured pixels on a single scanline, varying in horizontal length? MMC3#IRQ Specifics gets into some incredibly hairy stuff. I have a feeling the answer is "in how the hardware behaves", but I start to question whether or not this is truly an effect of the NES hardware or if the PowerPak -- or rather, the Verilog code in one or more powerpak sets -- are doing something "weird" behind the scenes that might cause this.

I kind of wonder if it'd be worth trying to put all of this into a single test ROM of sorts -- not a test in the sense of validating functionality, but something that mimicked what this game/code is doing (again: private stuff, can't talk about it or share code etc.) -- just so there was something out there that could be used to test and reproduce it.

Footnote: in addition to the above, some *other* anomalies were even weirder, like bizarre single-pixel blotches being shown in areas where nothing was going on (in Mesen, anyway), or part of the screen slightly bouncing/shaking/jiggling vertically -- where a subsequent power-cycle of the NES/PowerPak and re-run of the game showed no problems. The latter inconsistency makes me wonder if some variable/thing in one of these powerpak sets is not being initialised correctly, if I have a buggy/faulty PowerPak, or what. I suspect *these* anomalies are something entirely different and not related to the above.

User avatar
Banshaku
Posts: 2334
Joined: Tue Jun 24, 2008 8:38 pm
Location: Fukuoka, Japan
Contact:

Re: The state of PowerPak mappers

Post by Banshaku » Mon Mar 04, 2019 1:32 am

I always appreciate the fact that Koitsu always goes the extra mile while doing testing when the only thing I asked first was "do you see some artifacts at the split point?" :lol: I'm very grateful for all the effort he went trough for it and maybe some interesting stuff may comes out of it too! ;)

For now, even though it still an interesting problem, until I can test and stabilize the code on either an mmc3b cart, which I have but for now started to fail and I don't know why (the logo shows up then die, lucky me!) or on one of infinitesneslives cart, which I may be able to do soon. My first guess at the issue is the timing when events occurs are not hush/hush, causing some of the artifacts but emulators just shows it without issue.

Until I stabilize the code and confirm on real mmc3b/infinitesneslives carts, we may be trying to find a solution for something that can actually be fixed by code and only me can figure it out once I have a proper environment to do so.

Thanks for the testing, really appreciate it. I can't wait to talk about it but for now, until it's completed, I have to keep it a secret. It's nerve racking :lol:

User avatar
loopy
Posts: 394
Joined: Sun Sep 19, 2004 10:52 pm
Location: UT

Re: The state of PowerPak mappers

Post by loopy » Mon Mar 04, 2019 1:44 am

It's been ages since I've gone over the powerpak MMC3 code, but I'll dig it up if there's something that needs fixing. I have MMC3A,B,C carts I can test on if needed.

User avatar
Banshaku
Posts: 2334
Joined: Tue Jun 24, 2008 8:38 pm
Location: Fukuoka, Japan
Contact:

Re: The state of PowerPak mappers

Post by Banshaku » Mon Mar 04, 2019 1:53 am

That's nice of you, I really appreciate it.

For now, before I get more people involved in this project, I really, really need to test it on real hardware on my side to make sure it's not just my code first. If it's only my code then I would feel bad to have many people working on a non issue ^^;;;

Once enough tests is done on my side, I will let you know if testing on specific cart is required. For now infiniteneslives carts shows artifcats but they are not as strong as what koitsu saw on the powerpak. I will try to rewrite my flash eeproms again if I can this week to see if it was just a failed write on my eeprom or my devcart started to just fail.

User avatar
tokumaru
Posts: 11646
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: The state of PowerPak mappers

Post by tokumaru » Mon Mar 04, 2019 5:35 am

I find it strange that CHR bankswitching is causing so much trouble, since this is one of those operations that do not interfere with the PPU itself, so the worst that can happen if the switch is not stable enough is a little jitter with the wrong patterns near the switch point.

If the problem is indeed with timing CHR bankswitches, there's a simple trick you can do with the MMC3 that will buy you a lot of wiggle room for the bankswitch operations: instead of dividing the background in regions of 256 unique tiles, divide it in regions of 128 unique tiles, so that you're only ever using tiles $00-$7F OR $80-$FF, so you have the entire time during which one set is being displayed (at least 32 scanlines, which is what you can cover if each of the 128 tiles is used only once) to switch the other set, which is guaranteed not to be in use.

If you do this, then you don't have to worry about hblank or any of that precise timing bullshit, as you have a freaking huge window of at least 32 scanlines where it's safe to do the switch for the next 128-tile area. Of course, this is assuming your problems are limited to CHR bankswitching. If the MMC3 IRQ timings are causing problems elsewhere, you should definitely look into it.

User avatar
Banshaku
Posts: 2334
Joined: Tue Jun 24, 2008 8:38 pm
Location: Fukuoka, Japan
Contact:

Re: The state of PowerPak mappers

Post by Banshaku » Mon Mar 04, 2019 9:57 am

The only time there was issue with bank switching midframe was when the timing in hblank was too early or too late, causing artifacts (line of a different color in the picture) since the following lines were not black. When splitting where the text is shown and switch to the bank for fonts, you have more leeway about it.

So for now, I do not think I will have to reduce the tiles to that level since I think it may just all be issue with timed code that may not be executed properly in hblank and the powerpak may be showing it more based on which mapper pak is used. I think some mapper pak may be triggering the IRQ event not exactly at the same position on the raster compared to the mmc3 or some emulator (but I'm just speculating right now).

The real test on hardware should help confirms if the powerpak does trigger the irq at a different position than the mmc3 or emulator.

Thanks for the comments!

User avatar
tokumaru
Posts: 11646
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: The state of PowerPak mappers

Post by tokumaru » Mon Mar 04, 2019 10:47 am

I'm just saying you wouldn't need to worry about timing bankswitches to hblank at all if you switched more frequently (every 128 tiles instead of every 256), without having to sacrifice anything. With 2 independent 128-tile slots, you have 32 scanlines or more where the slot that's not being displayed can be changed with zero graphical glitches.

If this is not during gameplay, you could even do this using very roughly timed code (i.e. no IRQs), since the windows for doing the switches are so huge.

User avatar
Banshaku
Posts: 2334
Joined: Tue Jun 24, 2008 8:38 pm
Location: Fukuoka, Japan
Contact:

Re: The state of PowerPak mappers

Post by Banshaku » Mon Mar 04, 2019 6:34 pm

I will need someday to learn how to do "normal" timed code since for now, the only way I know how to do it is with the mmc3. I don't know why but for some reason, I always "kind of" figure out to some degree how to use the more complex stuff but stumble on the basics since, well, I never used them ^^;;;

Since I'm not sure what you mean by "the window to do the switches are so huge",. then it means I should learn more about it ;)

User avatar
rainwarrior
Posts: 7803
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: The state of PowerPak mappers

Post by rainwarrior » Mon Mar 04, 2019 7:24 pm

Banshaku showed me a video and it triggered some old memories that might be relevant...

I owned two PowerPak devices over the years. The first one I damaged (and later gave away), but I think the second one behaved differently for MMC3, despite using the same mappers. I also seem to recall that thefox's PowerMappers MMC3 made it better.

The different behaviour I recall was that sometimes SMB3 or Crystalis or other MMC3 games would jitter the raster split up and down by 1 line frequently.

I also vaguely seem to recall people discussing noise on PPU A12, and maybe adding a small filtering capacitor or something to the PowerPak to help the issue? (In fact this might even have been how/why I damaged my first PowerPak.)

Thefox's PowerMappers says it does the "RAMBO-1" thing, which means it waits for 2 M2 falls before applying the IRQ. This should have the effect of filtering a noisy A12 transition, right? (Eliminate an accidental double hit?)

Like maybe the issue is more of an analog one, specific to how sensitive the FPGA in the PowerPak is to a little bit of noise/bounce on that line? (Something emulators wouldn't be affected by at all.)


Anyhow, that's just an idea. I'm not sure if my memory is correct, it's been years since I had that other PowerPak, but I really do seem to remember jitter that I can't duplicate with my current PowerPak. (Edit: after more testing, I'm noticing the jitter is still there. It's an intermittent issue with Loopy's mapper 4. I think I started using the PowerMappers around the same time I got my second PowerPak, so I was probably just misremembering this as the new PowerPak fixing it, instead of correctly remembering that thefox's mappers were the fix.)


On another note, an interesting ROM for case study might be this old Hiatus Ward demo from 2012. It seems to have timing/jitter problems on more or less any emulator but on PowerPak it's a lot worse. (Though to be fair there are probably other issues than IRQ in that ROM, and trying to isolate the effect from the others would be a challenge.)
Last edited by rainwarrior on Tue Mar 05, 2019 1:48 am, edited 1 time in total.

Post Reply