Oversize FDS disk images

Discuss hardware-related topics, such as development cartridges, CopyNES, PowerPak, EPROMs, or whatever.

Moderator: Moderators

NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: Oversize FDS disk images

Post by NewRisingSun »

I am not going to continue discussing a matter from another thread here. If you are of the opinion that I misbehaved in another thread and and would like to discuss that, post to the thread in question.
lidnariq wrote:You opened the entire thread only stating the two-byte form, which is the thing Loopy objected to,
But the full form was already included in the quotation to which he responded, and according to your following comment, the length of the string makes no difference anyway, which I do not agree with.
lidnariq wrote:which I see you have now edited out from the first post to provide no evidence of your misspeaking.
Actually, I have edited in the full form, not edited out. The only thing I have edited out was some blather about why I asked the question even though nobody would implement the application, which I later considered beside the point.
lidnariq wrote:Searching for magic numbers within a file is fragile. Doesn't matter if it's two bytes or many.
The "magic number" in question is not some far-fetched heuristic, but a defining characteristic of the beginning of any disk image. For the strategy to fail, an oversize disk would need to have the exact fifteen-byte string in the oversize area (not in the initial 65500 bytes of the side's data), something that has not occurred in almost three thousand floppy disk images, and that is not conceivable in any real-life case, as the disks in question contain images of data that was originally in a cartridge, which has no reason to contain that string. You would be correct however to the extent that the strategy would not be robust to somebody deliberately contriving an image in this manner.

In any case, you have made your point on the oversized disk side question, and I will see what I make of it.
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: Oversize FDS disk images

Post by Fiskbit »

I know little about how FDS files are handled, but for games with save features, don't these disks contain user data that could potentially match your magic string?
NewRisingSun
Posts: 1510
Joined: Thu May 19, 2005 11:30 am

Re: Oversize FDS disk images

Post by NewRisingSun »

I do not see how that could happen, because only normal FDS games write user data to disk, and these games are by definition not oversized. Only Game Doctor et al. games use oversized disks, and being cartridge games transferred to disks, they do not save any data to disk, and could not do that even if somebody tried to hack them to do that, because the FDS BIOS is no longer accessible on most models once the RAM cartridge has been activated until the next power cycle. Even if it did happen, it would only be a problem if that user data, that happened to contain the \x01*NINTENDO-HVC* string, were saved in the oversized area, meaning after the original 65500 bytes of a side.

(Yes, this also means that games that originally had battery-backed WRAM effectively lose their saving ability when transferred to a Game Doctor et al. disk. For this reason, to compensate for that, later RAM cartridges included a "real-time save" functionality to compensate for that, similar to that of software emulators.)
User avatar
loopy
Posts: 405
Joined: Sun Sep 19, 2004 10:52 pm
Location: UT

Re: Oversize FDS disk images

Post by loopy »

Even if you're right, why make a format that relies on a bunch of assumptions? Simpler solutions exist, I don't know why you're doubling down on this point.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Oversize FDS disk images

Post by tepples »

Just use an archive format like Zip, Tar, or GBFS. If it must be extractable on an NES flash cart, use GBFS.
User avatar
Bregalad
Posts: 8056
Joined: Fri Nov 12, 2004 2:49 pm
Location: Divonne-les-bains, France

Re: Oversize FDS disk images

Post by Bregalad »

NewRisingSun wrote:No, I come to the forum seeking direct answers to specific questions. I asked a very specific question: "How to represent such a multi-disk set using the headered "fwNES" FDS file format", and got no answer on that question.[...]

I'll take the answers, such as they were, as "It cannot be represented at all, so use a new format",
Seems like you answered your own question. The ".fds" disk sides have a maximum of 65500 bytes, which actually is much higher than the real FDS disk capacity. I'm not sure what you're refering too, but your disks are not FDS disks so they cannot be stored in the .fds format and you have to come up with something else.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Oversize FDS disk images

Post by koitsu »

If you want a history lesson about fWNES, the FDS format (sort of), alternative Famicom Disk System file formats, the author/inventor FanWen YANG, that kinda thing, just ask -- I'm familiar with it a little bit because I'm one of the few (maybe the only?) people from the late 90s NES emulation/dev scene that spoke Mandarin. I did a history write-up but felt it wouldn't matter, but does go into Famicom disk formats prior to FDS (there are at least 3 I know of off the top of my head). Instead let's cut to it:

My view is the exact same as others stated here: what you have are technically disk images from a Game Doctor/Magicard unit from Bung. There are several Game Doctor-compatible copiers/units, all of which behave a little differently, no? Maybe they all use the same format, I don't know -- I don't have these devices. I do know that there are other companies that Bung who made such copiers (that's essentially what they are, particularly the later models), for example I think Venus Corp made one and strongly suspect Front Fareast had one. But there are also other systems that were used for dumping too -- if Donald Moore / MindRape was still with us, he'd love to talk about this -- like a particular model of Brother computer that could supposedly read Famicom disks.

The FDS file format is very clear: the per-disk-side size is 65500 bytes. And while bytes 5-15 of the 16-byte FDS header itself are all zero, they were intended to be used for mainly emulation purposes, e.g. "oh, we learned a new magical quirk about the FDS that happens only on some models, we need to use byte 5 for that purpose". If there IS something new/unique about the FDS itself that requires this data (i.e. is not Game Doctor/Magicard specific), then that's a different matter.

The FDS format is conceptually identical to the original NES format by Marat Fayzullin: much like the the 16KB PRG and 8KB CHR bank sizes of the NES format, this is another case of where the information available at the time (dumps, RE, pioneering efforts, etc.) made the best decision they could for that file format (FDS), and thus the 65500-byte-per-side definition. I know, lots of folks wish these formats had been more flexible size-wise, but they weren't. We must be pragmatic and err on the side of caution (read: ensuring full compatibility).

If that's not enough for you, here's a point to consider as a comparison:

There are many different SNES copiers from many companies, *all* of which use their own separate file format (sometimes not just a header!) and approaches, despite the ROM of the backed up game (obviously) being part of the image. You cannot take, for example, a .SMC/SWC file from a Super Wildcard and load it on a Game Doctor by simply changing the extension to .078. Conversion between these formats -- and there are *MANY* -- is commonly done with UCON (I haven't used this version myself, but the early 90s DOS version was a godsend). And we've seen people come along and try to introduce new file formats due (only in part) to those annoyances, as well as only loading ROMs with those headers removed entirely (i.e. pure ROM images) -- nobody but the most pedantic of the pedants care, what came first (or thereabouts) "won the war".

Likewise, I strongly doubt you can take a Game Doctor/Magicard dump that's "oversized" like this and put it onto another device/copier/system (particularly one not from Bung) and have it work. You'd need a file conversion utility, or at least something that strips off the bytes between 65500 and whatever the byte before 0x01 + *NINTENDO-HVC* was, to achieve a "raw" disk format. Heck, do you even know what those "oversized" bytes are for? Game Doctor units apparently had all sorts of neat features (especially the later models); could it be data the unit itself reads/benefits from? If so, well then there you have it, that's a file format specific to those unit(s) and should be retained that way. So the fact remains that with a disk side larger than 65500 bytes means they're not compatible with the FDS file format as designed in the late 90s.

I would urge you to make your own file format with a different extension (maybe you can find out what Bung preferred these be named, extension-wise?), document that, and/or make a conversion utility that can strip off the unwanted bytes to convert to FDS (or maybe you could talk to the UCON folks and have them do it, considering they already support a boatload of NES-related file formats, including one of the older Famicom disk formats I hinted at in my first paragraph).
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: Oversize FDS disk images

Post by lidnariq »

Bregalad wrote:which actually is much higher than the real FDS disk capacity.
[...]your disks are not FDS disks
NRS said:

* They are real Quick Disks
* In a real FDS drive (HVC-022)
* Connected to a real FDS base station (HVC-023)

The only differences are
1- Some of these disks are overformatted to hold, evidently, about 4-6 KB more on them
and
2- There's an extra piece of hardware between the Famicom and the HVC-023 that does simple bankswitching
koitsu wrote:maybe you can find out what Bung preferred these be named, extension-wise?
Loopy earlier stated that these were originally distributed as raw disk images with single letters as extensions to specify side number. NRS objected on the grounds that with at least one system, the subsequent sides cannot be parsed without the first side.
Pokun
Posts: 2681
Joined: Tue May 28, 2013 5:49 am
Location: Hokkaido, Japan

Re: Oversize FDS disk images

Post by Pokun »

So the fwNES FDS format can't handle it and I guess the more generic QD format has the same problem. You have either the native Game Doctor format or invent a new format that's going to introduce all kinds of new problems like iNES or whatever.

Using the existing format makes most sense to me. If multiple files really is a problem then combine them somehow (like Tepple's suggestions). File extensions are just a guide and shouldn't be relied on, I don't see how that can be a problem (use ".fds", ".qd", ".dsk", ".flp", ".img" or whatever).
User avatar
B00daW
Posts: 586
Joined: Thu Jan 03, 2008 1:48 pm

Re: Oversize FDS disk images

Post by B00daW »

Correct me if I'm wrong, but it doesn't seem that the Game Doctor images are even being talked about. Have we gone through the process of converting them into a current, working format or are we going to butt heads and masturbate egos?

Are they FDS or Famicom games?

(At any rate it's pretty interesting that you found over-sized images. It's just a bit silly that we have petty arguments and forget the points. I see you guys do this time to time; and unfortunately it seems like some of the smartest people here have some of the most fragile egos.)
User avatar
loopy
Posts: 405
Joined: Sun Sep 19, 2004 10:52 pm
Location: UT

Re: Oversize FDS disk images

Post by loopy »

They are just cartridge games ported/hacked to run from disk (that was the purpose of Game Doctor after all). I'm not aware of any original games made specifically for game doctor hardware. There's nothing special about the disks themselves (aside from the size), they follow the standard FDS disk format. I'm not sure that storing them as .FDS files makes sense - or if you go that route, I think you should indicate somehow that they require Game Doctor hardware to run. Nah, IDGAF

What NewRisingSun originally proposed is actually really close to my ideal format, I just think the way it's defined is sloppy.
User avatar
B00daW
Posts: 586
Joined: Thu Jan 03, 2008 1:48 pm

Re: Oversize FDS disk images

Post by B00daW »

Theoretically couldn't Famicom Disk System be added to NES 2.0 under Extended Console Type with a few reused redefinitions of the standard "PRG-ROM" area as "Total Size of Disk(s and Sides)"? Mapper 20 could be used.

There could be another reused area for number of disks and size per side. That way we're all just focusing on making NES 2.0 the all-encompassing 8-bit Nintendo format.
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: Oversize FDS disk images

Post by lidnariq »

I would really strongly prefer that NES2.0 PRG and CHR ROMs only contain parallel random access memories. It just makes everything a horrible mess when the interpretation of the bytes varies more and more.
Pokun
Posts: 2681
Joined: Tue May 28, 2013 5:49 am
Location: Hokkaido, Japan

Re: Oversize FDS disk images

Post by Pokun »

And then we can include cassette tapes. Just kidding, I think it's more clean to have separate file formats for separate media images. And yeah NES 2.0 is already confusing as it is.
I'm not sure how things like Datach are handled though.
Great Hierophant
Posts: 780
Joined: Tue Nov 23, 2004 9:35 pm

Re: Oversize FDS disk images

Post by Great Hierophant »

We have a NES 2.0, why not an FDS 2.0? Here is how it could be implemented.

The original fwNES FDS defined a 16-byte header, but only the first five bytes are used. The use of the first five bytes would not change in FDS 2.0. But the remaining bytes would. Here is how a FDS 2.0 header might look like

46 44 53 1A 03 47 44 4D xx xx xx xx xx xx xx xx

Bytes 1-5 are the ASCII for FDS, the DOS end of File Character $1A and the Number of Disk Sides
Bytes 6-8 are the ASCII for GDM, which is intended to stand for Game Doctor/Magicard.
Bytes 9-10 represent the additional amount of storage space required by Disk Side 1
Bytes 11-12 represent the additional amount of storage space required by Disk Side 2
Bytes 13-14 represent the additional amount of storage space required by Disk Side 3
Bytes 15-16 represent the additional amount of storage space required by Disk Side 4

Now you might observe that this limits you to four disk sides whereas the Game Doctor can use at least eight. What do you do then? You tack on another header after the end of the data on Disk Side 4 and continue with the data. The second header is identical to the first until Bytes 9-10, which represent the additional storage space required by Disk Side 5.

I have never seen an FDS 1.0 image ever take up more than four Disk Sides, so compatibility should not be effected. Moreover, FDS 1.0 images will have a file size that is easily determined (65500 x # of Disk Sides + 16 byte header (optional)). But if it is then the user must do a second insert of the next image with a compatible emulator.
Last edited by Great Hierophant on Thu Feb 14, 2019 9:45 am, edited 1 time in total.
Post Reply