Making New NES emulator (NESFaCE) - need help

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

6T4
Posts: 48
Joined: Sat Feb 19, 2011 8:56 pm

Making New NES emulator (NESFaCE) - need help

Post by 6T4 » Sat Feb 19, 2011 9:38 pm

I wanted to let everyone know about an upcoming NES emulator I'm programming and also ask for some help. The emulator is called NESFaCE and its website is http://code.google.com/p/nesface/. You can get in touch with me either here or there. There is no code uploaded at that site yet, but for what I need help with, you can refer to the code for one of the emulators NESFaCE is based on: NEStreme (http://www.cecs.csulb.edu/~hill/cecs497 ... howto.html). I'm having problems with that site right now, so just Google NEStreme to find the source elsewhere if that site doesn't work.

My question is about how to program some of the mappers. I understand (pretty much) how mappers work on the NES, but I'm not completely sure how they are implemented in NEStreme. In particular, I'm having trouble figuring out how to use different combinations of swappable prg and chr rom bank numbers and sizes. It seems that the author of NEStreme had this problem, too. For example, playing any game that uses mapper 3 gives messed up graphics. It looks to me like this is because swapping is only done on the first half of mapper 3's swappable chr rom bank in the code for the mapper 3 dll. There is an analogous problem with prg rom; I wrote a dll for mapper 7 that can play some sound on Wizards & Warriors but no graphics. So, to summarize, I know what the problem is but not how to fix it. Do I just make a small change in the mapper dll (like doing some arithmetic to make things happen in the right address range) or do I need to change the hierarchy of setting up the banks in Io.cpp, or maybe something else?

Please let me know if you can help with this or if you want to help code the emulator in general; it's a more significant project than some other NES emulators because of its goals. Mainly, I'm aiming for 100% compatibility and code that's as simple as possible, but I've also got some features in mind that will really make NESFaCE stand out not only among NES emulators but among all emulators. Thanks in advance for your help.

User avatar
Zepper
Formerly Fx3
Posts: 3192
Joined: Fri Nov 12, 2004 4:59 pm
Location: Brazil
Contact:

Re: Making New NES emulator (NESFaCE) - need help

Post by Zepper » Sun Feb 20, 2011 12:19 am

- Hmm, from what you said...
Mainly, I'm aiming for 100% compatibility and code that's as simple as possible, but I've also got some features in mind that will really make NESFaCE stand out not only among NES emulators but among all emulators.
- ...I'd like to ask: why not writing your own code, from scratch? How do you expect to reach such goals if you barely understand a 3rd part source code?

- Anyway, if there's an emulator that's not running 100% of "the games", it's because of a) lacking of supporting an obscure mapper and b) has homebrew in mind, as the software would be a testbed for it.
6T4 wrote:(...)but for what I need help with, you can refer to the code for one of the emulators NESFaCE is based on: NEStreme (http://www.cecs.csulb.edu/~hill/cecs497 ... howto.html). (...)

User avatar
tokumaru
Posts: 11519
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Making New NES emulator (NESFaCE) - need help

Post by tokumaru » Sun Feb 20, 2011 12:29 am

First, let me say that this emulator (NEStreme) appears to be very rudimentary, and that expanding it so that it's anywhere close to Nintendulator or Nestopia when it comes to accuracy will require a nearly total rewrite. What I mean is that this is a "bare bones" NES emulator... it has enough basic elements to get some simple games running, but it's internal workings are nothing like a real console. It also doesn't take into consideration a lot of things that mappers can do, like bankswitching CHR-ROM in blocks smaller than 4KB or PRG-ROM in blocks smaller than 16KB. It looks like it was written by a person that understands the basics of how an emulator works, but doesn't have deep knowledge about the NES itself, so many important details were overlooked.
6T4 wrote:My question is about how to program some of the mappers. I understand (pretty much) how mappers work on the NES, but I'm not completely sure how they are implemented in NEStreme.
From what I can see, the mapper files only have 3 functions, called for each of the following events: cartridge is loaded, PRG-ROM is read from and PRG-ROM is written to. The first one is where you do any necessary initialization, like mapping banks in and such. The second will have to retrieve the appropriate byte depending on the banks that are mapped in and such. The third is where you'll handle mapper writes, since most mappers have registers mapped in the $8000-$FFFF area. When writes occur to that range you have to map in the appropriate banks according to the mapper documentation.

This looks very limited, as there's much more that mapper can do than what appears to be possible with this system.
For example, playing any game that uses mapper 3 gives messed up graphics. It looks to me like this is because swapping is only done on the first half of mapper 3's swappable chr rom bank in the code for the mapper 3 dll.
Looking at the mapper 3 code, it's clear that it's only setting one of the 2 pattern table pointers this emulator uses. You should also set the second one, with something like pPPU->apbyPatternTables[1] = pabyCHRROM + (byData*0x2000) + 0x1000;

Note however that many mappers bankswitch CHR-ROM in chunks as small as 1KB, but this emulator only has 2 pointers to 4KB chunks. If you expect to support those mappers you should have a total of 8 pointers.
I wrote a dll for mapper 7 that can play some sound on Wizards & Warriors but no graphics.
Mapper 7 uses CHR-RAM, you have to take a better look at how this emulator handles that. Ad also how it handles name table mirroring, which this mappers can control.
it's a more significant project than some other NES emulators because of its goals. Mainly, I'm aiming for 100% compatibility and code that's as simple as possible
By "compatibility" do you mean playing all commercial games you can think of? If so, this is a pretty lame goal. If you make your emulator accurate, it will play everything without hacks or shortcuts, and will also be a decent platform for running homebrew games, which are not yet part of any compatibility list. And if your goal is to make it really accurate, it will by no means be simple. Even the most accurate emulators of today are constantly being updated to incorporate tiny details that are still being discovered about the NES.
but I've also got some features in mind that will really make NESFaCE stand out not only among NES emulators but among all emulators.
It looks like you have some interesting ideas for the GUI, control methods and such, and that's cool, but to make your emulator undoubtedly good, you have to make sure that the underlying simulation of the system is as accurate as possible, and I'm not sure that using a badly incomplete and outdated emulator as a base is such a good idea.

6T4
Posts: 48
Joined: Sat Feb 19, 2011 8:56 pm

Re: Making New NES emulator (NESFaCE) - need help

Post by 6T4 » Sun Feb 20, 2011 2:20 pm

Zepper wrote:
- ...I'd like to ask: why not writing your own code, from scratch? How do you expect to reach such goals if you barely understand a 3rd part source code?
I am using NEStreme as a basis since it is a very simple emulator... it's just a small step away from writing the code from scratch. Also, I believe that I can learn from all other emulators so I will look at and possibly use code from all of them. One reason I may not understand some things in NEStreme is because it's an incomplete emulator, as mentioned in the other reply; I think the goals I have set are reachable with a little help.

6T4
Posts: 48
Joined: Sat Feb 19, 2011 8:56 pm

Re: Making New NES emulator (NESFaCE) - need help

Post by 6T4 » Sun Feb 20, 2011 2:42 pm

tokumaru wrote:
Looking at the mapper 3 code, it's clear that it's only setting one of the 2 pattern table pointers this emulator uses. You should also set the second one, with something like pPPU->apbyPatternTables[1] = pabyCHRROM + (byData*0x2000) + 0x1000;
I tried exactly this; it didn't work. The emulator gave an error upon starting to run any mapper 3 game, before any video or audio output. So, does this mean I need to have a variable number of pointers for each mapper that is equal to however many swappable banks that mapper has? (1 chr rom pointer for mapper 3, for example, pointing to the whole bank instead of 2 each pointing to half of it).

tokumaru wrote:
Mapper 7 uses CHR-RAM, you have to take a better look at how this emulator handles that. Ad also how it handles name table mirroring, which this mappers can control.
Mapper 2 uses chr ram as well; there shouldn't be anything extra to put in the code to accommodate for this. I'm aware of how the name table mirroring is handled; I believe my trouble is due to mapper 7 having one 32K prg rom bank rather than either one or two 16K prg rom banks, analogous to my problem with the chr rom banks in mapper 3.
tokumaru wrote:
By "compatibility" do you mean playing all commercial games you can think of? If so, this is a pretty lame goal. If you make your emulator accurate, it will play everything without hacks or shortcuts, and will also be a decent platform for running homebrew games, which are not yet part of any compatibility list. And if your goal is to make it really accurate, it will by no means be simple. Even the most accurate emulators of today are constantly being updated to incorporate tiny details that are still being discovered about the NES.
By compatibility, I guess I mean writing all the mapper dlls and making sure that each board associated with each mapper is supported. I also want to make it as accurate as possible to ensure that anything that runs on a real NES or a real Famicom will run on my emulator. Also, by "simple", I don't mean "short". I realize accommodating all the quirks of the NES will take a fair amount of code. I just want to make sure my code is organized in a way that makes it easy to understand what each part is doing.
tokumaru wrote:
It looks like you have some interesting ideas for the GUI, control methods and such, and that's cool, but to make your emulator undoubtedly good, you have to make sure that the underlying simulation of the system is as accurate as possible, and I'm not sure that using a badly incomplete and outdated emulator as a base is such a good idea.
I have thought of some features that are much more interesting than just some gui enhancements. The most exciting (and probably hardest to implement) is an autoplay feature where the emulator would react to games and actually be able to play the game itself or be the second player. I also want to eventually incorporate surround sound and a 3d mode that would require 3d glasses. However, I realize that emulation accuracy is much more important than any of these features (and itself useful to the autoplay feature), and it is one of my primary goals.

User avatar
tokumaru
Posts: 11519
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Making New NES emulator (NESFaCE) - need help

Post by tokumaru » Sun Feb 20, 2011 7:23 pm

6T4 wrote:I tried exactly this; it didn't work.
I see... Well, what I suggested is what would make sense to me, and unfortunately I don't currently have any easy means to compile C++ code, so I can't debug this any further, sorry.
By compatibility, I guess I mean writing all the mapper dlls and making sure that each board associated with each mapper is supported.
Programming mappers should be pretty easy actually, as their hardware is significantly simpler than the NES itself. The thought that mapper support == compatibility is pretty archaic, as games are much more likely to fail because of some flaw in the NES simulation than in the mapper implementation.
I also want to make it as accurate as possible to ensure that anything that runs on a real NES or a real Famicom will run on my emulator.
That's a good goal, but hard to achieve when you are basing the architecture of your emulator on some badly incomplete and outdated emulator. I think you should decide what your main focus is: learning more about the NES from implementing it on software (in which case you should study a lot about the NES and design a better emulator from scratch than what the author of NEStreme did) or making the ultimate emulator with awesome features (in this case you should use a mature emulator such as Nestopia - I don't know if its license allows it though - for the underlying emulation and focus on the features you'll implement on top of it). If it's both, I think you should start from scratch too.
The most exciting (and probably hardest to implement) is an autoplay feature where the emulator would react to games and actually be able to play the game itself or be the second player.
See, this is an example of a good idea that I would just love to have on an emulator. But realistically speaking, I don't see it happening. Since each game has its own architecture, there is no way for a generic AI player to know the difference between enemies and items, floors and bottomless pits, and so on. An auto-play feature like this would have to be implemented on a game-per-game basis, upon careful (i.e. time-consuming) examination of how each game organizes its variables and models the game world.

Now, if you have actually thought this through and have other ways of implementing this in mind, I would simply LOVE to hear your ideas.
I also want to eventually incorporate surround sound
How would that be any good when the original musicians didn't anticipate this? Any separation you make of the audio will be arbitrary, and will not actually "surround" players. It's like the stereo mod where some channels go to the left and others go to the right, but I bet that just sounds weird because the composers never used any actual effects making use of this separation.
and a 3d mode that would require 3d glasses.
We already have that no? Or do you mean better 3D glasses than the red/blue cellophane paper ones?

6T4
Posts: 48
Joined: Sat Feb 19, 2011 8:56 pm

Re: Making New NES emulator (NESFaCE) - need help

Post by 6T4 » Sun Feb 20, 2011 10:08 pm

tokumaru wrote:Programming mappers should be pretty easy actually, as their hardware is significantly simpler than the NES itself.
Yes, but many emulators fail to include all of the boards for each mapper. Many mappers have more than one board assigned to them. Also, very few emulators support all the known mappers.
tokumaru wrote:See, this is an example of a good idea that I would just love to have on an emulator. But realistically speaking, I don't see it happening. Since each game has its own architecture, there is no way for a generic AI player to know the difference between enemies and items, floors and bottomless pits, and so on. An auto-play feature like this would have to be implemented on a game-per-game basis, upon careful (i.e. time-consuming) examination of how each game organizes its variables and models the game world.
I have thought about this a little bit, although I'm sure I'll think about it more and my thoughts may change. I think there would be a near infinite number of ways to implement this, but I envision them on a continuous spectrum from what I see as a low level emulation method to a high level emulation method. The LLE method would be to have the emulator read all of the instructions in the game, figure out what has to be done to move the game forward (move the character, fight a boss, select a level, etc), and then do it. The HLE method would be to have the emulator react to graphics and sound just like a human player would. (So maybe I should be thinking of this method as LLE and the other as HLE.) Practically, I think a mixture of these is needed. You say "there is no way for a generic AI player to know the difference between enemies and items, floors and bottomless pits." I disagree. To tell the difference between a floor and a bottomless pit, look at the graphics at the bottom of the screen. If there is a sprite change, make the character jump. This shouldn't be impossible to program. To tell the difference between an enemy and an item, use what I called the LLE method. The emulator should be able to figure out from the instructions if you die when you touch a certain sprite. If so, that sprite must be an enemy. If it's required that your character occupies a space that a certain sprite occupies which is then removed for the game to progress at some later point, then that sprite must be a required item. Those are just a couple examples.... but I hope they provide a better illustration of what I have in mind.
tokumaru wrote:How would that be any good when the original musicians didn't anticipate this? Any separation you make of the audio will be arbitrary, and will not actually "surround" players. It's like the stereo mod where some channels go to the left and others go to the right, but I bet that just sounds weird because the composers never used any actual effects making use of this separation.
The NES has five sound channels (except for a few games that use extra ones), which corresponds perfectly to the surround sound channels. I realize it may sound "weird", but some people may think it's cool for certain games. It was just a random idea I thought of, not something really meant to improve quality.
tokumaru wrote:We already have that no? Or do you mean better 3D glasses than the red/blue cellophane paper ones?
Already have it? We already have 3d glasses, but I am not aware of any NES emulator that is designed to be used with them. I mean implementing a graphics mode so that two copies of the screen are output in slightly different places but mostly overlapping, one tinted red and the other tinted blue. The virtual boy emulator vbjin does this, if you need an example of what I mean.

User avatar
tokumaru
Posts: 11519
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Making New NES emulator (NESFaCE) - need help

Post by tokumaru » Sun Feb 20, 2011 10:39 pm

6T4 wrote:The LLE method would be to have the emulator read all of the instructions in the game, figure out what has to be done to move the game forward (move the character, fight a boss, select a level, etc), and then do it.
I was trying to take you seriously, but now you are just sounding like this guy (your idea sounds just like the "bug catcher" feature in his NES game maker). There's absolutely no way your emulator can guess the game rules by examining the instructions. Not happening.
The HLE method would be to have the emulator react to graphics and sound just like a human player would.
You'd have to teach your emulator to read so that it can even get past the title screen. And even then there are title screens without any written instructions. You'd have to "press" the direction buttons and see which sprites (on-screen sprites, not OAM sprites because those change every frame) move as a consequence in order to identify the player character. Then you'd have to magically identify which background blocks are solid and which aren't. You'll have to watch sprites that move by themselves to identify enemies. Somehow you'll have to identify items. Somehow you'll have to detect deaths. Somehow adapt the AI to the physics model used in each game. This sounds slightly more possible, but still incredibly far-fetched.
The emulator should be able to figure out from the instructions if you die when you touch a certain sprite.
You see, it can't. The NES has no universal "KillCharacter" function. No universal "LivesLeft" variable. When a character dies, some variables (which you don't know the meaning of and there's no way for a program to guess) are moved around, some branches occur, and there's absolutely no way to tell that apart from, say, a character getting an upgrade, unless you have personally studied the architecture of the game and know what each variable means. This is not happening automatically, period.
The NES has five sound channels (except for a few games that use extra ones), which corresponds perfectly to the surround sound channels. I realize it may sound "weird", but some people may think it's cool for certain games.
You bet it will sound weird! All jump sounds will come from one speaker, all explosions from another, some effects will be split across different ones... None of that will make you feel any more immersed in the game.
I mean implementing a graphics mode so that two copies of the screen are output in slightly different places but mostly overlapping
You can't turn a 2D image into 3D if you don't know the depth of each object, just like you can't create surround sound from mono if you don't know where each sound is supposed to come from. The few NES games that have a 3D mode already switch between red and blue images (this is in the game code, not in the emulator), and you can play them with cellophane glasses.
The virtual boy emulator vbjin does this, if you need an example of what I mean.
All Virtual Boy games are 3D though, you have 2 different images to begin with. You can't generate a consisted 3D image out of a 2D one. You're 1 dimension short, and you can't get the information that's missing out of your ass. If it's not there, it's not there.

tepples
Posts: 21837
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Making New NES emulator (NESFaCE) - need help

Post by tepples » Mon Feb 21, 2011 6:08 am

tokumaru wrote:Programming mappers should be pretty easy actually, as their hardware is significantly simpler than the NES itself.
Even MMC5? Given what I've seen of the NES CPU die, how much space the PSGs take up, and how the MMC5 has two Famicom expansion sound channels, and how it supervises the PPU's operation, I'd wager that it's at least as complex as the CPU.
Since each game has its own architecture, there is no way for a generic AI player to know the difference between enemies and items, floors and bottomless pits, and so on. An auto-play feature like this would have to be implemented on a game-per-game basis
In Lua, possibly. Ideally, a game can peek CPU and PPU memory locations and respond with button presses. Sure, it'd require a memory map, but those are on datacrystal. Such a feature might even be useful for prototyping a computer AI to be implemented within the game itself. (But I bet FCEUX already dunnit.)

User avatar
koitsu
Posts: 4216
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Post by koitsu » Mon Feb 21, 2011 12:15 pm

I've added mention of this emulator to the never-ending list of NES emulators that people are developing.

User avatar
cpow
NESICIDE developer
Posts: 1092
Joined: Mon Oct 13, 2008 7:55 pm
Location: Minneapolis, MN
Contact:

Re: Making New NES emulator (NESFaCE) - need help

Post by cpow » Mon Feb 21, 2011 6:08 pm

tokumaru wrote:I was trying to take you seriously, but now you are just sounding like this guy (your idea sounds just like the "bug catcher" feature in his NES game maker). There's absolutely no way your emulator can guess the game rules by examining the instructions. Not happening.
I was hoping someone would drag that thread into this one...

User avatar
tokumaru
Posts: 11519
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Making New NES emulator (NESFaCE) - need help

Post by tokumaru » Mon Feb 21, 2011 8:10 pm

cpow wrote:I was hoping someone would drag that thread into this one...
So you also see the similarities then?

6T4
Posts: 48
Joined: Sat Feb 19, 2011 8:56 pm

Re: Making New NES emulator (NESFaCE) - need help

Post by 6T4 » Mon Feb 21, 2011 9:08 pm

I still need help with how to implement the mappers more than anything, so don't worry about any features too much yet. I hadn't really paid attention to where it says "if (byData <= wNumCHRROMPages)" in the mapper 3 dll, so I'm wondering if that is critical to what I'm trying to do. If anybody can help, it would be very much appreciated. On a side note, I have figured out how to implement support for 4 players. I suppose I should also respond to some criticism about the other features:
tokumaru wrote:You see, it can't. The NES has no universal "KillCharacter" function. No universal "LivesLeft" variable. When a character dies, some variables (which you don't know the meaning of and there's no way for a program to guess) are moved around, some branches occur, and there's absolutely no way to tell that apart from, say, a character getting an upgrade, unless you have personally studied the architecture of the game and know what each variable means. This is not happening automatically, period.
This is an example of something that would require a mix of both methods. While the NES has no universal "die" function, individual games do. After dying, the game almost always goes back to display a previous screen. So, the thing to do is find out what instruction (or combination of instructions) make this happen.
tokumaru wrote: You can't turn a 2D image into 3D if you don't know the depth of each object, just like you can't create surround sound from mono if you don't know where each sound is supposed to come from. The few NES games that have a 3D mode already switch between red and blue images (this is in the game code, not in the emulator), and you can play them with cellophane glasses.

All Virtual Boy games are 3D though, you have 2 different images to begin with. You can't generate a consisted 3D image out of a 2D one. You're 1 dimension short, and you can't get the information that's missing out of your ass. If it's not there, it's not there.
You can turn a 2d image into simulated 3d though by copying the image and displaying one copy for each eye. Think about that ViewMaster toy. It'll be like that, except for the different tints will be used with glasses instead of putting two tubes up to your eyes to view the two different pictures. Also, I believe Virtual Boy games just have one image, it's just displayed in two different locations on a grid.

User avatar
Dwedit
Posts: 4251
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Post by Dwedit » Mon Feb 21, 2011 9:09 pm

It's not simulated 3D unless you displace things in one eye vs the other eye.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!

User avatar
tokumaru
Posts: 11519
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Making New NES emulator (NESFaCE) - need help

Post by tokumaru » Mon Feb 21, 2011 10:02 pm

6T4 wrote:I hadn't really paid attention to where it says "if (byData <= wNumCHRROMPages)" in the mapper 3 dll, so I'm wondering if that is critical to what I'm trying to do.
This is actually wrong... Games can write values larger than the number of CHR pages the game has, what will happen is that the CHR data will appear to repeat. For example, if a game has 4 CHR pages and the number 7 is written to the register, page 3 wil be mapped in, because page 4 is the same as page 0, page 5 is the same as page 1, and so on. So I think you should get rid of that IF and use byData MOD wNumCHRROMPages as the actual page that will be mapped in.
While the NES has no universal "die" function, individual games do. After dying, the game almost always goes back to display a previous screen. So, the thing to do is find out what instruction (or combination of instructions) make this happen.
Maybe you can't understand what I'm saying to you now because you still have no idea how complex a NES game can be, and that automatically making sense of groups of instructions at such a low level is practically impossible, so I'll just stop talking about that and let you see for yourself when you try to implement this feature.
You can turn a 2d image into simulated 3d though by copying the image and displaying one copy for each eye.
Copying the image as a whole will do nothing, as all objects will be perceived at the same depth. For an image to look 3D, each object has to be displaced according to its depth (close objects are displaced more than distant ones), but your emulator can't tell the depth of the objects just from looking at the 2D image, so it can't displace the objects consistently.
It'll be like that, except for the different tints will be used with glasses instead of putting two tubes up to your eyes to view the two different pictures.
How are they different if you just said you will just clone the 2D image? Its just the same image twice, which will not result in a 3D effect.
Also, I believe Virtual Boy games just have one image, it's just displayed in two different locations on a grid.
I'm pretty sure this is not the case. Look at this screenshot:

Image
Pay attention to the lines on the floor. Note that they are angled very differently between the red and blue images. Notice that some objects are almost in the same position in both images, so they become purple, but others are so much apart that you can easily see the individual red/blue images. You can't create that effect from the flag 2D image the NES outputs.

Post Reply