It is currently Fri Dec 15, 2017 1:29 am

All times are UTC - 7 hours





Post new topic Reply to topic  [ 28 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: UNIF database?
PostPosted: Sun Jun 26, 2005 1:17 pm 
I'm guessing such a thing doesn't exist, otherwise there would be an actual iNES->UNIF convertor... But anyways, I have been wanting to make such a utility for ages but just don't have data for it to use. boardtable.txt is the only doc I've seen, but it only has the board name, no mirroring info and the like. Nestopia seems to have a good database packed in, but some of the board names seem ambiguous, for example some are simply called MMC1 rather than the specific board name.

I'm working on a FCEU fork, and I would really, REALLY like to remove iNES support completely and if one were to try and load one, it would bitch and force you to convert. I don't want to go on a rant about it, but the fact that iNES is still around just irritates me to no end.

I'm sure some will not agree with that, but some of the arguments I've heard that keep people sticking to iNES seem ridiculous to me. Like IPS patching not working with UNIF. WTF? Yes it's true that existing tools wouldn't work anymore, but an emu could easily remap the IPS and do a softpatch. Or if you like to do hard patching, creating a program to do so is trivial.

About the UNIF format itself, I of course love it. But I personally would like to see some additions to it. Supplemental info like publisher and year released would be nice. Something else I would really like to do is have an SRAM block and store your save in the same file. I of course would have the option to clear the block out. This wouldn't work if you were running from a read-only source, but I figured if that were the case, it would just create a local copy.

And finally (I know people will hate this idea), I think it would be cool if the UNIF container had blocks for holding images of the cart, box, and manual. Yes this would bloat the hell out of the filesize, but if you don't like that, you could always delete those blocks. I would really like it for manuals to be converted into vector type format. That would make them typically a lot smaller and much more useful. But I know this is unrealistic, wishful thinking. :)

But back on topic, I'm ready to make a conversion tool but I'm gonna need help on the database. I certainly don't want to be making guesses based off the iNES mapper number. I own ~200 carts so if needed I can have the fun job of opening them all up to get what I need :P Kevin Horton seems like he must have (or had access at one time) to damn near every cart. I wonder if he was recording such data for each game.


Top
  
 
 Post subject:
PostPosted: Sun Jun 26, 2005 2:01 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19346
Location: NE Indiana, USA (NTSC)
If I have a PRG file (from cc65's linker), a CHR file (from a bitmap converter or a tile editor), and a board spec (in some format), what is the best tool to make a UNIF file out of them? By contrast, the iNES format needs only 'copy /b'. Such a UNIF building tool would be needed to make homebrews, and it would reduce the scope of your app to creating the board spec.


Top
 Profile  
 
 Post subject:
PostPosted: Sun Jun 26, 2005 2:27 pm 
Offline
User avatar

Joined: Sun Sep 19, 2004 10:59 pm
Posts: 1393
There exists a "libunif" whose purpose was to simplify the creation and parsing of UNIF files, though it can be somewhat complex to deal with.
For creation of them, I opted for a simpler approach - I wrote a small program, named MakeUNIF, which takes a small textfile and creates a UNIF file based on the parameters inside. For .NES files, I made a similar utility named MakeINES.

_________________
Quietust, QMT Productions
P.S. If you don't get this note, let me know and I'll write you another.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jul 11, 2005 7:29 pm 
I've been throwing around a few ideas on the database specifics. I'm planning on using MySQL for the database. I would prefer to write the client interface as a standalone program as opposed to a web-based one. I use Windows, but if I stick with using strict Win32 API, Linux and such should be capable of running it without trouble (actually I would prefer to use MFC, but AFAIK that pretty much limits it to Windows only). Worst case, if one wanted to contribute, but couldn't use the program, they could pass the details along to me to add on their behalf. Also you could technically connect to the DB using any SQL client you please and add the data manually, but that would be a bit of a hassle.

As for the specifics of what the DB would contain, I've come up with this preliminary list. Some of it is not absolutely neccesary or even applicable in some cases but I think is useful nonetheless.

Before each entry, you will have to point to what ROM data your working on. It will allow you to load iNES, UNIF, or raw PRG+CHR. This is so it can calculate CRC data. If you load a UNIF file, it will also pull any other relevant data it can.

Game Name:
The full game name

Board Name:
The complete board name (ex. NES-AOROM-3, NES-NROM-256)

Hardware Onboard:
Chips on board besides the ROM/RAM/CIC (ex. MMCx,LS161)

Copyright holder/year:
Game publisher and year released (ex. Nintendo 1987)

PRG/CHR ROM sizes:
This will be auto-generated

PRG/CHR CRC:
Auto-gen'd

VRAM & WRAM Sizes:
Size of said if present, otherwise leave 0.

Battery:
Flagged if cart contains battery.

System: (See note later)
What system the cart is intended for. (NTSC,PAL,PC10,VS)

Country:
Country of origin.

Mirroring:
Mirroring on cart. (Horz, Vert, Mapper Ctrl'ed, 4 Screen, 1 Screen (there are 2 types of this?))

Catalog #:
The cat number for the cart (ex. NES-XX-USA)

ROM Names:
The ID stamped on the ROM(s) (ex. NES-XX-0 PRG)

Controller Type:
Controller used by game. This one is kinda iffy to me, it doesn't really have anything to do with the cart. Only reasoning for having this one is so an emu could auto-select the proper input device and also as a reference such as ("What games use the zapper?")

Cart Classification:
This is to ID the type of cart, as in "(Un)Licensed, PD, Proto, Pirate"
Any others?

The DB would also store your name/date submitted.

Something else I think would be a rather important item is a "Verified" flag. If all information is complete and you personally dumped the associated ROM data, you would check this off and this would lock the entry from future updates. This would also allow people with carts (and ability to open them) but no dumper, to add entries based on ROMs from elsewhere. Which 99% of the time should be good enough,unless the ROM associated with the entry is a bad/altered dump or a revision.

Back to the note about the system item. It's my understanding that PC10 games have something like an extra 8K? ROM that contains that menu program that runs on the upper screen. Also the VS Unisystem has the various dip switches. Also they have custom palette ROMs. Do you think there should be means to handle this info?

Sorry for the long post. To sum it all up, I'm asking for your insight on the interface idea, any items you think should be added/changed etc.


Top
  
 
 Post subject:
PostPosted: Mon Jul 11, 2005 9:57 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19346
Location: NE Indiana, USA (NTSC)
Game name: would this allow UTF-8, or would it require all titles to be transliterated into US-ASCII or into ISO Latin-1? I'd ask for both.

Hardware on board: NES-EVENT has a DIP switchbank. VS Unisystem has a color ROM. All licensed NES games have a CIC chip; unlicensed games have either a CIC clone (Tengen) or a -5V generator (Camerica, Color Dreams, AVE).

Copyright owner: What happens when a game is subject to multiple copyrights? Example is NBA Jam. Publisher listed on the box is Acclaim; characters and scenario are licensed from NBA Properties; game design is by Midway; program is by Iguana; some low level code libraries are licensed from Nintendo.

Cart classification would have to distinguish programs that have one or more of the following:
  • "Pirate" : title screen hacks to remove copyright or change the name of a game, often found on multicarts
  • "Graphics Hack" : Mario Baby would be both a pirate and a graphics hack
  • "Conversion" : any of the various fighting games based on the TMNT Tournament Fighters engine would be a pirate and a conversion
  • "Pirate Original" : game which has an original program but uses copyrighted character likenesses without permission, such as Kart Fighter or Somari or the various ports of Aladdin

Despite the prominence of pdroms.de, I'm rawther not fond of the designation "PD", as its expansion "public domain" implies that all copyright in a work has been abandoned. Some authors give a blanket license to noncommercial distribution of verbatim copies but reserve all other rights under copyright. I'd say "freely redistributable proto" instead.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jul 12, 2005 7:28 am 
Offline
Formerly Fx3
User avatar

Joined: Fri Nov 12, 2004 4:59 pm
Posts: 3076
Location: Brazil
Well... if you have some patience... go ahead. :)

_________________
Zepper
RockNES developer


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jul 12, 2005 9:56 am 
Offline

Joined: Sun Jul 10, 2005 10:40 am
Posts: 8
Wouldn't a flat file method be better than a MySQL database? It would be much more cross platform friendly and could easily be merged it an app. The number of games and the amount of data is hardly enough to worry about performance in a flat file. Also, if it is in MySQL, doesn't a MySQL instance have to be running on the machine. If it is going to be server based, than a web interface to a MySQL server would work great. If it is a local DB, than a flat file is the only way to go.

Alas, don't count on ANYTHING that has Win32 APIs to run well under Linux. Wine is still lagging in most areas. Standard libs and cross over libs are the only way to keep the fustration to a min.

Also, if you are going to force users to use something different, then you must at least provide a conversion tool.

A great idea though. I am thinking of integrating such a (simpler) DB into my emulator.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jul 12, 2005 10:52 am 
tepples wrote:
Game name: would this allow UTF-8, or would it require all titles to be transliterated into US-ASCII or into ISO Latin-1? I'd ask for both.

I was going to use solely UTF-8, but your right, an ASCII version should probably be required too.

tepples wrote:
Copyright owner: What happens when a game is subject to multiple copyrights? Example is NBA Jam. Publisher listed on the box is Acclaim; characters and scenario are licensed from NBA Properties; game design is by Midway; program is by Iguana; some low level code libraries are licensed from Nintendo.

I was considering it to be filled as the publisher only. Or in the case in wasn't a published game, then it would be the creator. I should probably have it labeled as publisher.

tepples wrote:
Cart classification would have to distinguish programs that have one or more of the following:
  • "Pirate" : title screen hacks to remove copyright or change the name of a game, often found on multicarts
  • "Graphics Hack" : Mario Baby would be both a pirate and a graphics hack
  • "Conversion" : any of the various fighting games based on the TMNT Tournament Fighters engine would be a pirate and a conversion
  • "Pirate Original" : game which has an original program but uses copyrighted character likenesses without permission, such as Kart Fighter or Somari or the various ports of Aladdin

No problem.

tepples wrote:
Despite the prominence of pdroms.de, I'm rawther not fond of the designation "PD", as its expansion "public domain" implies that all copyright in a work has been abandoned. Some authors give a blanket license to noncommercial distribution of verbatim copies but reserve all other rights under copyright. I'd say "freely redistributable proto" instead.
I see your point, makes no difference to me. We can go by popular vote on that I guess :)

Omega wrote:
Wouldn't a flat file method be better than a MySQL database? It would be much more cross platform friendly and could easily be merged it an app. The number of games and the amount of data is hardly enough to worry about performance in a flat file. Also, if it is in MySQL, doesn't a MySQL instance have to be running on the machine. If it is going to be server based, than a web interface to a MySQL server would work great. If it is a local DB, than a flat file is the only way to go.

Well I actually started writing this with a flat-file in mind. In fact I pretty much had the basic DB system done. I had initially planned on having this work by each submitter would basically be running this program, and create entries that would have to be sent to me or wherever to be merged into a master DB. But I basically I decided it was going to be easier and much more streamlined to use SQL. It's not a matter of performance, just convience all around. As for the DB server, I was intending on just having it run on a local machine off my own connection, since this isn't something that going to put much load on it. The reasons I'm leary of the web interface: While I want it to be as hassle free as possible, I don't want it to be so easy that any old meathead can come and punch in a bunch of garbage. Not that I suspuct that would be much of a problem. Secondly, I'm not a big VB fan, and the only other way I could go about this would be a VB ActiveX control. I'm sure it's possible with PHP and the like, but I am unfamilar with that. Also this way would require me to be running a webserver as well.

Omega wrote:
Alas, don't count on ANYTHING that has Win32 APIs to run well under Linux. Wine is still lagging in most areas. Standard libs and cross over libs are the only way to keep the fustration to a min.

Yeah I don't use Linux, so I'm not really sure about that. Friends have said they rarely have problems as long as it's Win32 API only. I could of course go to a console based system, but I figured it would be easier for everyone to use a GUI based interface. I'm open to any suggestions.

Omega wrote:
Also, if you are going to force users to use something different, then you must at least provide a conversion tool.

Absolutely. In fact that is the main reason I'm doing this :)


Top
  
 
 Post subject:
PostPosted: Tue Jul 12, 2005 12:23 pm 
Offline

Joined: Sun Jul 10, 2005 10:40 am
Posts: 8
PHP is VERY easy, and having a webserver (WIN or Linux) is easy and free with Apache. I could provide you with any PHP code you needed to connect to ANY database you want.

I am very interested in this project and I would like to help out. I am very familiar with MySQL, PHP, HTML, javascript, Linux, Apache, Windows, DB design and interfacing. So if you have any questions, let me know.

I do think that you need to make interfacing as easy as possible though, regardless of possible data pollution. DB cleanup is going to happen no matter what. Oh, and don't forget about the Mac and UNIX users out there with that WIN32 API :)


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jul 12, 2005 12:44 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19346
Location: NE Indiana, USA (NTSC)
Omega wrote:
PHP is VERY easy, and having a webserver (WIN or Linux) is easy and free with Apache.

Caution: Most residential Internet access terms of service do not permit running a web server, and even if it does you can't turn off your computer or boot it into a different operating system, so you'll probably have to use some sort of shared hosting plan. This costs per year, as the free web hosting packages generally do not support PHP and MySQL.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jul 12, 2005 12:52 pm 
Offline

Joined: Sun Jul 10, 2005 10:40 am
Posts: 8
I have used PHP and MySQL on several web-server rental sites for pretty cheap ($8.00 - 15.00/mounth). IPowerWeb being one of them. I have also found it to be pretty easy to get web service support from ISPs. As far as not shuting down the computer, well, that's just part of being a server.

Good point though, if you can't run the service, then what do you do.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jul 12, 2005 12:54 pm 
Offline
Site Admin
User avatar

Joined: Mon Sep 20, 2004 6:04 am
Posts: 3487
Location: Indianapolis
Yeah, the label "PD" has been misused way too much where "freeware" would be more appropriate. It can be a major annoyance for anyone who releases a game for people to play for free (since people can and do sell collections of "PD" roms).

It also depreciates programs that actually are PD. If you know the saying, when you hear the clock strike 13 times, it also casts doubt on all the strikes that came before, heheh.


Top
 Profile  
 
 Post subject: "1300 hours"
PostPosted: Tue Jul 12, 2005 1:03 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19346
Location: NE Indiana, USA (NTSC)
Memblers wrote:
Yeah, the label "PD" has been misused way too much where "freeware" would be more appropriate. It can be a major annoyance for anyone who releases a game for people to play for free (since people can and do sell collections of "PD" roms).

I generally don't mind if people sell copies of my games, as long as they include the source code. Heck, if you can port one of my productions from one of the emulators to NES hardware, and you want to stick it on a multicart next to pirated Balloon Fight and all those other pirated NROM-128 classics, go right ahead and put the source code on a CD. It shouldn't make the packaging too much bigger.

Quote:
when you hear the clock strike 13 times, it also casts doubt on all the strikes that came before, heheh.

Unless you're in the military of course.

Anyway, what's a proper designation for games such as Elite that have been released on a cartridge and then "freed" by the author?


Top
 Profile  
 
 Post subject:
PostPosted: Sun Jul 17, 2005 8:52 am 
Offline

Joined: Wed Jul 13, 2005 3:14 pm
Posts: 356
About mirroring, according to the UNIF spec, these are the legal values:

* $00 - Horizontal Mirroring (Hard Wired)
* $01 - Vertical Mirroring (Hard Wired)
* $02 - Mirror All Pages From $2000 (Hard Wired)
* $03 - Mirror All Pages From $2400 (Hard Wired)
* $04 - Four Screens of VRAM (Hard Wired)
* $05 - Mirroring Controlled By Mapper Hardware

I'm wondering though, other than boards that have the H/V select pads, mirroring can always be implied by the board name can't it? If so, it seems unnecessary to make that distinction and the [MIRR] block would only be necessary if the board has the solder pads and only have a value for H and V.


Top
 Profile  
 
 Post subject:
PostPosted: Sun Jul 17, 2005 9:14 am 
Offline
User avatar

Joined: Sun Sep 19, 2004 10:59 pm
Posts: 1393
That is correct - the UNIF "MIRR" blocks is ONLY relevant if the board (described in the "MAPR" block) has solder pads on it to select hardwired mirroring. Many [non-Nintendo] cartridges are simply hardwired to horizontal or vertical mirroring (in which case the board name would imply it), while there are also a few Nintendo boards (MMC3, possibly MMC1 as well) which have solder pads to select between Horizontal, Vertical, and Mapper-controlled.

The fact that there are two different "hard-wired single-screen" settings in UNIF is rather stupid, since if it's hard-wired you'll NEVER be able to tell which half you're using.

_________________
Quietust, QMT Productions
P.S. If you don't get this note, let me know and I'll write you another.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 28 posts ]  Go to page 1, 2  Next

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group