It is currently Mon Oct 23, 2017 10:15 am

All times are UTC - 7 hours





Post new topic Reply to topic  [ 89 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next
Author Message
PostPosted: Thu Feb 26, 2015 6:18 am 
Offline
User avatar

Joined: Sat Feb 12, 2005 9:43 pm
Posts: 10067
Location: Rio de Janeiro - Brazil
rainwarrior wrote:
They probably didn't have any official documentation at all to work from. A lot of unlicensed NES work was done on the basis of reverse engineering.

Sure, and as far as I know, a big part of the reverse engineering process involved analyzing disassembles of official games. Surely they have analyzed every register on their own, but when you don't have any official documentation to guide you, the official software should provide you with guidelines for development. Just because the hardware you happen to be testing on behaves a certain way, you can't expect that behavior to be consistent across every revision (now or in the future) unless the manufacturers say so. That's common sense.

Now, my point isn't necessarily that they shouldn't have used such advanced/obscure techniques, but that it was very risky to use them for something as silly as title/menu screens that don't even look particularly impressive. As far as I can see, all the glitches happen in blank areas, so instead of wasting 8 scanlines on an obscure technique to sync with the PPU, wouldn't it be simpler to change 2 colors every HBlank during those 8 scanlines? With 85 pixels of HBlank, a 21-pixel jitter wouldn't get in the way of 2 palette writes and 2 more to move the VRAM address away from the palette range before the next scanline starts. Besides palette changes, what other kinds of raster effects are they doing? Horizontal scroll changes? Those don't need perfect PPU synchronization either.

Quote:
The game was also never released on Famicom, so even if the technique would have failed on that, it wasn't relevant.

Unlicensed or not, I'm sure developers would want their software to have a high compatibility rate, even if the initial target market was limited.

Quote:
If it works, it works. The fact that it's annoying to emulate is our problem, not theirs.

This is not about emulation, but about using the hardware in a safe way so that random customers don't show up complaining that the game doesn't work properly for them. I don't know how much they tested this technique (and on how many different consoles) before considering it safe to use, but the fact that it was not used in any official release (not that they have looked at every officially released game, but still) should have raised some flags. They took a chance when they chose this approach (which wasn't even absolutely necessarily to begin with), and luckily for them no consoles had trouble with it. Things might have been differently if the game was released in Japan.


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 8:45 am 
Online
User avatar

Joined: Mon Dec 29, 2014 1:46 pm
Posts: 713
Location: New York, NY
tokumaru wrote:
It's so weird that this game goes through all this trouble to pull off some stupid raster effects in menu screens that could have easily used timed code all the way, with the beginning of the frame and/or a sprite 0 hit as a reference point. But no, they decided to use an obscure/undocumented aspect of the console, that's not even guaranteed to work on all revisions (didn't OAM read back fail on some Famicom models?).


From the look of things, I suspect that they hired a bunch of devs who used to work on Atari 2600 games.

lidnariq wrote:
zeroone wrote:
I think you mean $2004 and it seems to be doing all that polling on the same scanline (the sprite 0 hit scanline).
I meant what I said, exactly as I said it.

It polls $2002, waiting for sprite 0 hit. That provides a 6-7 cycle/18-21 pixel jitter.

It then tests $2004 over eight successive scanlines, adjusting the PPU/CPU phase by one cycle (+1 pixel / -2 pixels) if necessary on each of those scanlines.


Agreed.

But, the wiki does not provide enough information about $2004 reads for me to enhance my emulator. How anyone solved this for Nintendulator or Nestopia is a mystery.


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 10:03 am 
Offline

Joined: Thu Aug 12, 2010 3:43 am
Posts: 1589
tokumaru wrote:
But no, they decided to use an obscure/undocumented aspect of the console, that's not even guaranteed to work on all revisions (didn't OAM read back fail on some Famicom models?).

It would not be the only time they do stuff like this, literally every single one of their Master System games rely on the revised VDP in order to have a higher resolution, despite the fact there isn't any single official game using it. I guess that their reverse engineering consisted on disassembling a couple of games to get a vague outline then poking stuff to figure out the rest.


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 10:31 am 
Online
User avatar

Joined: Mon Dec 29, 2014 1:46 pm
Posts: 713
Location: New York, NY
Not surprisingly, Bee 52 is also quite glitchy on my emulator.


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 10:46 am 
Online
User avatar

Joined: Mon Dec 29, 2014 1:46 pm
Posts: 713
Location: New York, NY
Dug something up from an older thread:

Quote:
In the title screen it reads $2004 8 times on scanlines 16..23 around scanline clock ~310:

FD6E BIT $2004
FD71 BMI $FD73
FD73 ...

So it adds an extra cycle whenever the value returned by $2004 has the top bit set. It's exploiting the fact that at certain times $2004 read always returns $FF and at certain times proper values from OAM, depending on which PPU cycle the read falls on. That way they get a more precise CPU-PPU sync.


Which PPU cycles will $2004 return $FF and which ones will it return OAM values? Of those that return OAM values, which OAM values are returned?


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 11:04 am 
Offline

Joined: Sun Apr 13, 2008 11:12 am
Posts: 6297
Location: Seattle
Did you find this page on the wiki? http://wiki.nesdev.com/w/index.php/PPU_ ... evaluation

It really should contain enough information for what you are doing.


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 11:11 am 
Online
User avatar

Joined: Mon Dec 29, 2014 1:46 pm
Posts: 713
Location: New York, NY
lidnariq wrote:
Did you find this page on the wiki? http://wiki.nesdev.com/w/index.php/PPU_ ... evaluation

It really should contain enough information for what you are doing.


I implemented that logic as best as I could interpret the page. Does $2004 effectively return whatever was last read from either primary or secondary OAM? For instance, at the bottom:

Quote:
Read the first byte in secondary OAM (while the PPU fetches the first two background tiles for the next scanline)


Does that mean that $2004 returns the first byte in secondary OAM?


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 11:44 am 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 5734
Location: Canada
tokumaru wrote:
Sure, and as far as I know, a big part of the reverse engineering process involved analyzing disassembles of official games. Surely they have analyzed every register on their own, but when you don't have any official documentation to guide you, the official software should provide you with guidelines for development. Just because the hardware you happen to be testing on behaves a certain way, you can't expect that behavior to be consistent across every revision (now or in the future) unless the manufacturers say so. That's common sense.

Without any official documentation, there's no reason at all to assume the behaviour isn't consistent. That's not common sense at all. Why would you assume that something that works now wouldn't later? You can only find this by being told by the designers, or testing the revisions in question, and obviously you can't test future revisions.

You act like disassembling a game isn't a monumental effort of its own. It's not like they could just set a breakpoint in an emulator and try out 100 games in an afternoon like we can today. Think of how much time (i.e. think of how much it cost) to disassemble a game for them. Every one they disassembled was burning time out of their development budget.

Imagine they know this technique works from their testing. A programmer shows it to their lead. The lead says "no don't do that. we haven't seen that in a game before. Go disassemble games until you find one that does this and then you will have my permission." If a lead used this strategy, the game would never be finished.

tokumaru wrote:
Unlicensed or not, I'm sure developers would want their software to have a high compatibility rate, even if the initial target market was limited.

You can't target an unknown platform. The only way to know what is out-of-design and what's not is to have the designer tell you. If you're not licensed, you're not given that information.

There's really no reason to assume any particular technique is taboo with the information they had.

If your goal is really to make the "most compatible" software possible, skipping the licensing process and relying entirely on reverse engineering is probably the last thing you should do. If you've accepted that they're willing to go rogue to cut costs, it's obvious that they're willing to take a risks like that to get their software out at lower cost to them.

Quote:
This is not about emulation, but about using the hardware in a safe way so that random customers don't show up complaining that the game doesn't work properly for them. I don't know how much they tested this technique (and on how many different consoles) before considering it safe to use, but the fact that it was not used in any official release (not that they have looked at every officially released game, but still) should have raised some flags. They took a chance when they chose this approach (which wasn't even absolutely necessarily to begin with), and luckily for them no consoles had trouble with it. Things might have been differently if the game was released in Japan.

I just tested it on my Famicom, and it is indeed slightly glitchy looking. There is a little bit of white marks on the right hand side of the screen, and a bit of 1-line jitter up and down on the character select screen. It doesn't make the game unplayable even slightly. It doesn't make any of the names or graphics unreadable. It doesn't put garbage faces on the characters. It's just a couple of white marks and a little bit of jitter. This is not to mention that the actual gameplay takes place entirely on screens without this technique, so playing the game is entirely unaffected.

As far as bugs go, this is pretty acceptable. It's not GOOD to have minor graphical glitches on the screen, but this is such a mild bug they might not even consider it worth fixing in a revision if they sold a lot of units and had an opportunity to get a new run of PRG ROMs made. How many well selling NES games had far worse problems than this? Lots of well selling games today even have bugs worse than this that the company doesn't want to pay to patch.

This is all moot, though, if they had released it on Famicom, they would have tested that platform and fixed the problem (or possibly decide it didn't need fixing). It seems really strange to me to complain about a bug that wasn't even present on any platform the release was for (i.e. not actually a bug).


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 12:28 pm 
Offline
User avatar

Joined: Sat Feb 12, 2005 9:43 pm
Posts: 10067
Location: Rio de Janeiro - Brazil
rainwarrior wrote:
Without any official documentation, there's no reason at all to assume the behaviour isn't consistent.

I agree with this regarding many other behaviors, but this one in particular is just too damn weird to be considered stable... If reverse engineering takes time, do you think they studied in depth what the values being read back from $2004 even meant? Do you think they got to figure out the sprite evaluation logic based on those reads? At first glance, a register that returns quickly changing values when you read it in a loop looks like the last think you'll want to trust.

Quote:
That's not common sense at all. Why would you assume that something that works now wouldn't later?

Because it's weird even when it's working. The values change faster than the CPU is able to keep up with and you have to use all sorts of trickery to do something useful with that register. Doesn't sound like something that was meant to be used like that. Again, not that I'm against being creative with the resources available, but I wonder if they were careful enough while going this route, considering that there were safer alternatives.

Quote:
You act like disassembling a game isn't a monumental effort of its own.

And figuring out the details of the rendering logic inside the PPU by fiddling with its registers isn't?

Quote:
There's really no reason to assume any particular technique is taboo with the information they had.

It's a register returning rapidly changing values that the CPU can hardly keep up with unless the OAM is organized in specific ways. It's weird.

Quote:
I just tested it on my Famicom, and it is indeed slightly glitchy looking. There is a little bit of white marks on the right hand side of the screen, and a bit of 1-line jitter up and down on the character select screen. It doesn't make the game unplayable even slightly.

Certainly not, but the reason they used the technique in the first place was so that they could make the game look better (otherwise they would have settled for less colors and effects), while glitches like these make the game look worse. If they had caught the compatibility issues earlier on, I'm sure they'd have done things differently.

Quote:
It seems really strange to me to complain about a bug that wasn't even present on any platform the release was for (i.e. not actually a bug).

I wasn't complaining. I don't give a shit if a game I don't play much (although I do sorta like Micro Machines) has glitches here and there or whether emulator authors are having trouble running it. If you read my first post in this thread again you'll see that I was just surprised that they did it the way they did instead of using a less complex technique or even not implementing the effects at all, since they don't add much to the experience. Maybe you are right and they simply didn't perceive their choices as risky, or just didn't care.


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 1:11 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19119
Location: NE Indiana, USA (NTSC)
tokumaru wrote:
rainwarrior wrote:
Without any official documentation, there's no reason at all to assume the behaviour isn't consistent.

I agree with this regarding many other behaviors, but this one in particular is just too damn weird to be considered stable... If reverse engineering takes time, do you think they studied in depth what the values being read back from $2004 even meant? Do you think they got to figure out the sprite evaluation logic based on those reads? At first glance, a register that returns quickly changing values when you read it in a loop looks like the last think you'll want to trust.

You'd be surprised. The Apple II had one of the registers return something from the state of the video circuitry. Its mouse card used that to synchronize to vertical blanking.


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 3:36 pm 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 5734
Location: Canada
tokumaru wrote:
If reverse engineering takes time, do you think they studied in depth what the values being read back from $2004 even meant? Do you think they got to figure out the sprite evaluation logic based on those reads? At first glance, a register that returns quickly changing values when you read it in a loop looks like the last think you'll want to trust.
...
I wonder if they were careful enough while going this route, considering that there were safer alternatives.
...
rainwarrior wrote:
You act like disassembling a game isn't a monumental effort of its own.

And figuring out the details of the rendering logic inside the PPU by fiddling with its registers isn't?

I can't speculate very specifically about what they knew, or how much time they spent on circuit analysis vs. disassembly, but given the unusual solutions present in Codemasters games, I'd guess they probably were able to do more analysis of the machine than disassembly of other software. I do think that at the time, with the tools available, disassembly was a lot more time intensive / expensive to do than hardware analysis. That's not at all the case now, of course, but I think it was for them.

I doubt they knew all the consequences and behaviour of $2004, but they knew enough to figure out a way to write some sort of scanline timer with it, and it worked reliably for them. Remember their goal was to build software for the machine, not a clone of the machine itself. They only needed to know things that help you build software.

Were they careful enough? I'd say yes. If it ran without problem on all target platforms, the risk they were taking paid off.

tokumaru wrote:
I wasn't complaining. I don't give a shit if a game I don't play much (although I do sorta like Micro Machines) has glitches here and there or whether emulator authors are having trouble running it. If you read my first post in this thread again you'll see that I was just surprised that they did it the way they did instead of using a less complex technique or even not implementing the effects at all, since they don't add much to the experience. Maybe you are right and they simply didn't perceive their choices as risky, or just didn't care.

I dunno, I guess to me... I look at this and see a successfully finished game, and I admire it for being that. You can look back on any game ever released, with all the knowledge gained across the years in your hindsight, and find all sorts of things and say "that's not how I would have done it". Maybe it puts me on the defensive because in my professional life I've seen so much time wasted in useless arguments about how something should be implemented. And you know, most of the time a bad implementation is not even how the developer wanted to do it, really, but a consequence of the limited time and money available.

If I were manging a project, I would not demand a perfect elegant solution to the problem I have from a (cheap) junior programmer. Conversely, I would expect an (expensive) experienced programmer not to waste my money refactoring an inelegant solution that works fine. The only place for perfect code is where the scope is small enough that time and money are irrelevant.

As far as I can tell, this feature in Micro Machines worked perfectly, so in that respect I kind of feel it's above reproach here. Obviously with what we know and practice now it's not a good technique to use, but I see nothing at all wrong with it in its original context.


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 4:13 pm 
Online
User avatar

Joined: Mon Dec 29, 2014 1:46 pm
Posts: 713
Location: New York, NY
From the wiki, $2004 returns $FF for PPU cycles 1--64 and likely not $FF for the rest of the scanline. Maybe that's the only thing that is relevant here.


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 4:56 pm 
Offline
User avatar

Joined: Sat Jan 22, 2005 8:51 am
Posts: 427
Location: Chicago, IL
Code:
if (rendering)
{
   if (current_cycle < 64)
      return_value = 0xFF;
   else if (current_cycle < 256)
      return_value = 0x00;
//Micro Machines relies on this:
   else if (current_cycle < 320)
      return_value = 0xFF;
//and this:
   else
      return_value = sprite_buffer[0];
}

_________________
get nemulator
http://nemulator.com


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 5:34 pm 
Online
User avatar

Joined: Mon Dec 29, 2014 1:46 pm
Posts: 713
Location: New York, NY
James wrote:
Code:
if (rendering)
{
   if (current_cycle < 64)
      return_value = 0xFF;
   else if (current_cycle < 256)
      return_value = 0x00;
//Micro Machines relies on this:
   else if (current_cycle < 320)
      return_value = 0xFF;
//and this:
   else
      return_value = sprite_buffer[0];
}


That does minimize the effect:

Image

That black line flickers. Many frames are completely line-free. It's almost doing the job.

I'm not sure where to take it from here though.


Top
 Profile  
 
PostPosted: Thu Feb 26, 2015 6:01 pm 
Offline
Formerly Fx3
User avatar

Joined: Fri Nov 12, 2004 4:59 pm
Posts: 3064
Location: Brazil
sprite_buffer[0] ?

Is this the primary OAM (max. 64 sprites) or the secondary OAM (max. 8 sprites)?


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 89 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group