It is currently Sat Dec 16, 2017 12:30 am

All times are UTC - 7 hours





Post new topic Reply to topic  [ 24 posts ]  Go to page 1, 2  Next
Author Message
PostPosted: Wed Sep 19, 2012 9:07 am 
Offline
User avatar

Joined: Thu Jan 03, 2008 1:48 pm
Posts: 544
I'd just like to bring to light something that has been bothering a lot of us over the years about No-Intro's commendable effort to database the known world of NES and Famicom (legit or unlicensed) cartridges.

We know that they are databased without any form of header often. And they believe that moving forward the NES/Famicom console format should be entirely headerless. It appears that they have put all their eggs into certain communities or emulator authors' unwilling baskets to support this overall effort; if you do something en masse it's up to the world as a whole to adapt to the change.

Since flash carts are becoming the enthusiasts option of choice, we know that a headerless format for the NES/Famicom would be entirely impossible unless processors/fpga-ish devices become faster/smaller/cooler and memory becomes cheaper. With current technology a NES/Famicom flash cart needs headers. Not only to this fact, but a header format takes the burden off emulator coders to database games themselves, keep up-to-date with good/bad dumps, allow dumps to be released without updating their code, allows not having to rely on a databasing community for updates to their database, and frankly allows a life outside of NES development. The NES/Famicom console ROM dump format and emulation could be much more intensive without the simple yet necessary innovation of the iNES header.

No-Intro could in fact be the agent to bring alive the NES 2.0 format or even beta-test the DotFami format. Instead it seems that they want to steer NES/Famicom into the dark.

As quoted by a No-Intro administrator in a thread by someone else attempting to make the effort to fix the NES/Famicom headers:

BigFred wrote:
I basically agree but the main goal was a completely headerless format. Just datting PRG/CHR and providing an external XML with header info accessible by emulators. I think it can be done with Nestopia or MESS.

I mean no disrespect to anyone, but my goal is to hopefully steer them away from this asynchronous step that is just doing more harm than good.


Top
 Profile  
 
PostPosted: Wed Sep 19, 2012 9:17 am 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19348
Location: NE Indiana, USA (NTSC)
What's wrong with this sort of son-of-ZapFC format? As I understand it, a ROM might look like this:
  • thwaite.prg: 16384 bytes
  • thwaite.chr: 8192 bytes
  • thwaite.board is an XML representation of the data in the iNES or NES 2.0 header
Is it that an 8-bit CPU has more trouble processing the XML representation than, say, NES 2.0? Or what else am I missing?

Anyway, you might want to mention that the maintainers of the NES 2.0 spec itself (kevtris and myself) are more easily contactable than Quietust. You might mention that the iNES format is in essence an archive format, just as zip and tar are archive formats. The header specifies whether certain predefined objects (PRG RAM trainer, PRG ROM, CHR ROM) are present and how big they are, and it's followed by the objects in that order.


Top
 Profile  
 
PostPosted: Wed Sep 19, 2012 11:15 am 
Offline
User avatar

Joined: Thu Jan 03, 2008 1:48 pm
Posts: 544
Well, opening this discussion has brought that to mind. I'm glad that you and kevtris are available for comment and aid. Quietust was a valuable source of information; and still makes a good emulator on the DL.

We have to consider that a flash cart will always be preferred to an emulator. Will a flash cart on older hardware be able to decompress 2 to 4 files and then look at an XML file, as in son-of-a-ZapFC to tell how the mapping logic and expansion chips function with given current technology and then be produced for a commercial market? Even so, that XML file is more or less a header but not existing at one end of the archived file. My conclusion is that something other than the PRG, CHR, and possibly sample ICs needs to exist to tell the emulator or hardware what to do with the file. And if it's a small header or XML file that exists in the archive, why not just keep headers and use the most useful databasing community as a vehicle to increase the capability and compatibility of that set of dumped images to aid the emulation and new commercial, NES/Famicom hardware.


Top
 Profile  
 
PostPosted: Wed Sep 19, 2012 11:35 am 
Offline
User avatar

Joined: Fri Nov 19, 2004 7:35 pm
Posts: 3969
Ultimately, you'll end up needing a PRG, CHR, and .verilog source for the mapper, and binary file to send to the FPGA inside the flash cartridge (built on a PC).

_________________
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!


Top
 Profile  
 
PostPosted: Wed Sep 19, 2012 12:37 pm 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 5899
Location: Canada
I am not sure you have anything to worry about. People spend a lot of time talking about formats, but its the availability of content that matters. Probably most people are going to stick with iNES 1.0 a long time yet.

What happened with ZapFC anyway?


Top
 Profile  
 
PostPosted: Wed Sep 19, 2012 9:36 pm 
Offline
User avatar

Joined: Thu Jan 03, 2008 1:48 pm
Posts: 544
Dwedit wrote:
Ultimately, you'll end up needing a PRG, CHR, and .verilog source for the mapper, and binary file to send to the FPGA inside the flash cartridge (built on a PC).

Exactly my point, but then that format wouldn't be very useful for PC/tablet/phone/console emulators. There needs to be one standardized format that just works. And as Kev and Quietust put it, NES 2.0 happens to also be "future-proof."

rainwarrior wrote:
I am not sure you have anything to worry about. People spend a lot of time talking about formats, but its the availability of content that matters. Probably most people are going to stick with iNES 1.0 a long time yet.


In the vein of "future-proof", I very much do have something to worry about. I'm also a NES/Famicom dumper with my CopyNES. And since I'm not an emulator author and I'm not entirely certain which mappers are available (if any) I am very reluctant to dump my stack of Brazil, Taiwan, Russia, etc. originals that would need another mapper number. Being that iNES 1.0 has grown too big for its britches, much of the new and interesting software cannot be dumped, coordinated with emulator authors, for mapper numbers.

I have to give props to NES 2.0 and DotFami. Both of them look pretty damned solid. Just considering the fact that ZapFC uses XML makes me wonder if a hardware flash cart would take kindly to it.


Top
 Profile  
 
PostPosted: Thu Sep 20, 2012 10:23 pm 
Offline
User avatar

Joined: Mon Feb 07, 2011 12:46 pm
Posts: 941
tepples wrote:
As I understand it, a ROM might look like this:
  • thwaite.prg: 16384 bytes
  • thwaite.chr: 8192 bytes
  • thwaite.board is an XML representation of the data in the iNES or NES 2.0 header
.....You might mention that the iNES format is in essence an archive format, just as zip and tar are archive formats....
In that case, could it be made 7-Zip to extract these three files (plus the trainer file if it has one) from a .NES file? (7-Zip can already extract many archive formats, including .msi, .iso, .rpm, and even Macintosh disk images, and more.)

Dwedit wrote:
Ultimately, you'll end up needing a PRG, CHR, and .verilog source for the mapper, and binary file to send to the FPGA inside the flash cartridge (built on a PC).
It may be possible to compile DotFami .cart binaries into Verilog, too. The ines.map file can then be used to convert iNES files into DotFami (or use the DotFami directly if it is already in that format), and then into Verilog and compiled to FPGA binaries.

B00daW wrote:
In the vein of "future-proof", I very much do have something to worry about. I'm also a NES/Famicom dumper with my CopyNES. And since I'm not an emulator author and I'm not entirely certain which mappers are available (if any) I am very reluctant to dump my stack of Brazil, Taiwan, Russia, etc. originals that would need another mapper number. Being that iNES 1.0 has grown too big for its britches, much of the new and interesting software cannot be dumped, coordinated with emulator authors, for mapper numbers.
If you have things that require another mapper number, there are two ways to proceed, either NES 2.0 first or DotFami first.

If you do NES 2.0 first:
  • Figure out the mapper.
  • Dump the PRG ROM and CHR ROM, as well as any others which might be necessary.
  • Write a document and/or DotFami .cart describing the mapper.
  • Post a message to ask for a mapper number.
  • Fix the NES 2.0 header with the specified mapper numbers.
  • If you want to, write a module to use with ines.map.

If you do DotFami first:
  • Figure out the mapper.
  • Dump the PRG ROM and CHR ROM, as well as any others which might be necessary.
  • Write a DotFami .cart describing the mapper (possibly using the Haskell library I plan to write for this purpose).
  • Combine them into a .fami file.
  • Optionally write a document describing the mapper.
  • If you want to, post a message to ask for a mapper number, and then write a module to use with ines.map. (Otherwise, someone will assign a mapper number later on if they want to.)

B00daW wrote:
I have to give props to NES 2.0 and DotFami. Both of them look pretty damned solid.
DotFami is incomplete, but I am working on it. Both are good formats.

B00daW wrote:
Just considering the fact that ZapFC uses XML makes me wonder if a hardware flash cart would take kindly to it.
Perhaps it is best to first use another computer to convert the format to whatever a specific flash cart needs.

_________________
.


Top
 Profile  
 
PostPosted: Fri Sep 21, 2012 11:34 am 
Offline
User avatar

Joined: Thu Jan 03, 2008 1:48 pm
Posts: 544
Well, with a little persuasion I have helped shed some light on the situation to No-Intro.

http://forums.no-intro.org/viewtopic.ph ... =10#p11812

The administrators are now aware of it understand that we need to reconsider the headerless idea and iNES 1.0.

They are in support of re-databasing the known collection of NES/Famicom games with the NES 2.0 format. This entails a lot off legwork but ensures to give emulator authors incentive to support NES 2.0. Since NES 2.0 is already proven to work and is more or less complete, I believe it's the format we should turn toward so we can continue dumping and emulating previously undiscovered NES/Famicom software without running out of mapper numbers. This way it could also make a lot of room for community custom mapper numbers to be assigned and developed under.


Top
 Profile  
 
PostPosted: Fri Sep 21, 2012 3:55 pm 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 5899
Location: Canada
I like NES 2.0, I was just saying I don't really see iNES 1.0 as obsolete yet. It works pretty well for most games, and emulators have workarounds for most of the ones that don't. We'll need new mapper IDs eventually, but there's still a few left for the time being.

Homebrew comes out slowly, but most (all?) homebrew is for existing mappers. I've seen a lot of custom mapper ideas being thrown around, but I haven't yet seen somebody actually make a game with one (if you know any, list them). We've got a slow stream of increasingly obscure games popping up with new mappers (esp. unlicensed chinese carts), but I don't imagine homebrew will be pushing the spec much.

DotFami is an interesting idea, but ultimately I kinda dislike the custom/generic mapper concept. I mean, I like the idea that it might account for new mapper configurations that we may need in the future, but I think it's just as likely that something new will come up requiring a spec revision and reimplementation. I think it would also be subject to homebrew abuse which may expose and complicate the internal implementation (compare: NSF multi-expansion). I much prefer the simplicity of an order-by-number mapper like you get with iNES.


Top
 Profile  
 
PostPosted: Fri Sep 21, 2012 4:12 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19348
Location: NE Indiana, USA (NTSC)
rainwarrior wrote:
Homebrew comes out slowly, but most (all?) homebrew is for existing mappers.

Just because a board uses an existing mapper doesn't mean it can be represented in original iNES. Size of PRG RAM and CHR RAM is the first big stumbling block. Original iNES supports SNROM and SUROM fine, but there's no widely supported way to specify the extra PRG RAM of SOROM and SXROM, and at least one homebrew release by Neil Baldwin needs the extra RAM of the SXROM board. An MMC3 board can be rewired to support large CHR RAM, but not in original iNES.


Top
 Profile  
 
PostPosted: Fri Sep 21, 2012 4:20 pm 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 5899
Location: Canada
Which Neil Baldwin release was that?

I'm not trying to poo-pooh efforts to move to a new format. I'm just saying that for most people iNES 1.0 isn't creating a lot of problems, and adoption of a new format is going to take a while yet. There are a some notable popular games that aren't properly handled (e.g. Startropics), but for the average user these problems are taken care of "under the hood" via CRC checks and whatnot. Not an ideal solution by any means, but I do think it slows the need for a format revision.

Do we have a list of games/releases somewhere that iNES is inadequate for? I guess I could go through FCEUX's CRC list, though I know many of those entries are just to correct improper iNES headers, rather than deal with iNES limitations...

By the way, why was MMC6 never given its own mapper number? I'm kinda curious about this. Is there some historical reason it doesn't get reassigned from iNES mapper 4?


Last edited by rainwarrior on Fri Sep 21, 2012 4:37 pm, edited 2 times in total.

Top
 Profile  
 
PostPosted: Fri Sep 21, 2012 4:28 pm 
Offline
User avatar

Joined: Wed Dec 06, 2006 8:18 pm
Posts: 2806
iNES and iNES 2.0 where the original fails is the way to go. There is no need for silly XMLs or other useless things. Considering the licensed/non-pirate collection of games is described quite well with iNES 1.0, iNES 2.0 helps with those that fell through the cracks and gets the job done. The whole idea of another format or a "headerless" format are never going to work. If you literally just had two files, one PRG and one CHR (or just PRG) you would have to hash it against an included database which is already a problem with iNES 1.0 for detecting games like Startropics or you have this XML file which is just a disguised header file.


Top
 Profile  
 
PostPosted: Fri Sep 21, 2012 4:56 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19348
Location: NE Indiana, USA (NTSC)
rainwarrior wrote:
Which Neil Baldwin release was that?

PR8


Top
 Profile  
 
PostPosted: Fri Sep 21, 2012 5:55 pm 
Offline
User avatar

Joined: Mon Feb 07, 2011 12:46 pm
Posts: 941
rainwarrior wrote:
I'm not trying to poo-pooh efforts to move to a new format. I'm just saying that for most people iNES 1.0 isn't creating a lot of problems, and adoption of a new format is going to take a while yet.
Unofficial MagicKit already supports NES 2.0 (but not by default; if it did, it would specify no RAM even if an older .asm file requires RAM on the cartridge), and I will soon add support for UNIF and DotFami as well.

Anyone making other assemblers (such as CA65 and so on) should consider support new formats too.

rainwarrior wrote:
There are a some notable popular games that aren't properly handled (e.g. Startropics), but for the average user these problems are taken care of "under the hood" via CRC checks and whatnot. Not an ideal solution by any means, but I do think it slows the need for a format revision.
I know, that is why we can have NES 2.0 submapper numbers, and is also the reason for the ines.map file (part of the DotFami specification).

MottZilla wrote:
There is no need for silly XMLs or other useless things. Considering the licensed/non-pirate collection of games is described quite well with iNES 1.0, iNES 2.0 helps with those that fell through the cracks and gets the job done. The whole idea of another format or a "headerless" format are never going to work. If you literally just had two files, one PRG and one CHR (or just PRG) you would have to hash it against an included database which is already a problem with iNES 1.0 for detecting games like Startropics or you have this XML file which is just a disguised header file.
OK. I think rips should be the PRG and CHR files and another file describing the other stuff, and then combine them together into the formats needed such as NES 2.0, UNIF, DotFami, and so on; you could uncombine them if necessary (although it might not be necessary). It just seems more logical to me to rip the ROMs separately and then combine them (including the header) afterward (which can be before they are published; when you need to send the files to anyone, only the combined files are necessary).

_________________
.


Top
 Profile  
 
PostPosted: Fri Sep 21, 2012 6:58 pm 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 5899
Location: Canada
zzo38 wrote:
Anyone making other assemblers (such as CA65 and so on) should consider support new formats too.


CC65 doesn't support any file formats directly, it just outputs binary files as you specify. You build an iNES ROM by creating a binary header in code, same for any other binary format.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 24 posts ]  Go to page 1, 2  Next

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group