.fds format: Can checksums be heuristically detected?

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems.

Moderator: Moderators

anikom15
Posts: 22
Joined: Mon Nov 30, 2020 2:41 am

Re: .fds format: Can checksums be heuristically detected?

Post by anikom15 » Thu Dec 10, 2020 12:20 am

Well that’s not something I’m advocating for, but I don’t really see any harm in a universal format either.

It’s not uncommon for extensions to mean fuckall anyway. The fact Windows uses extensions to decide what program to launch is stupid in the first place.

tepples
Posts: 22288
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: .fds format: Can checksums be heuristically detected?

Post by tepples » Thu Dec 10, 2020 8:34 am

anikom15 wrote:
Thu Dec 10, 2020 12:20 am
It’s not uncommon for extensions to mean fuckall anyway. The fact Windows uses extensions to decide what program to launch is [careless] in the first place.
I'd be interested to discuss a replacement for the file extension (or file name suffix) in this topic.

Pokun
Posts: 1761
Joined: Tue May 28, 2013 5:49 am
Location: Hokkaido, Japan

Re: .fds format: Can checksums be heuristically detected?

Post by Pokun » Thu Dec 10, 2020 9:04 am

anikom15 wrote:
Thu Dec 10, 2020 12:20 am
Well that’s not something I’m advocating for, but I don’t really see any harm in a universal format either.
I agree with you that one universal quickdisk format would be ideal. But it sounds like it wouldn't be possible in any satisfying way, and it would probably do a lot of harm. This isn't about the file extension, but the actual format of the data.

I fully agree that VC games are official releases. They are in Nintendo's QD format, but patched to work in the VC emulator which ignores CRCs. They comes with some metadata (including a digital manual) and are packed in a WAD file. Any emulator that wants to run them must be able to detect them as VC games and adjust accordingly to support them. The Dolphin emulator already do support them in their native WAD files. They are already fully preserved as they are 100% digital files to begin with. Any other format would make little sense since the are made for the Wii. Besides the QD file can be easily extracted from the WAD using existing tools.

The QD/RDA format is the format Nintendo stores their master disk files in, and the format the developers used when they created the games, so they should also be 100% preserved (for better or for worse, all master disk files have leaked out to the public on the internet). Nintendulator NRS supports the QD format, so it's possible to rely on bad CRCs if you use this format and that emulator for development.
Converting an RDA (Q.D. RAM-disk format) file to QD format is as easy as merging all RDA and RDB files for all disk sides sequentally into a single file using a hex editor or shell script, so they are essentially the same format.

The main problem would be Game Doctor disks or any other system that uses quickdisk drives such as MSX and those Roland keyboards. NewRisingSun already have Game Doctor disks covered. His tool also covers all files above (except the Wii WAD package files of course) and can losslessly convert between them. That's more than what Nintendo's official tool "rdafds.exe" can, as it can't reinsert the CRC when converting from FDS to QD (all CRCs will be 0). The CRCs are calculated from the file data, so it's not like the FDS format doesn't have the necessary information to reproduce the CRCs, given that the FDS file isn't corrupt.

The other problem is that gaps seemingly can't be digitally preserved to 100%, and would be different on different disks anyway. The QD/RDA format doesn't have them because they are byproducts of the disk writing process if I understands it correctly. I'm not sure why the FDS format is made to require standardized gaps filled with zeroes, as they seem to serve no point.

tepples
Posts: 22288
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: .fds format: Can checksums be heuristically detected?

Post by tepples » Thu Dec 10, 2020 9:28 am

I imagine "Can I use an FDS transport to dump a DataDisk for my Smith-Corona Personal Word Processor?" isn't part of the scope of this. (Unless someone wants to make it so.)

NewRisingSun
Posts: 1287
Joined: Thu May 19, 2005 11:30 am

Re: .fds format: Can checksums be heuristically detected?

Post by NewRisingSun » Thu Dec 10, 2020 10:35 am

I have never seen the low-level format of Quickdisks for other applications. But I would expect it to be entirely different from Nintendo's format. The fact that some Quickdisks are reported as storing 256 KiB per side indicates that the recording density must be different, which in turn may or may not mean that the coercivity of the physical disk differs as well.

User avatar
loopy
Posts: 399
Joined: Sun Sep 19, 2004 10:52 pm
Location: UT

Re: .fds format: Can checksums be heuristically detected?

Post by loopy » Thu Dec 10, 2020 10:49 am

I have a stack of Roland sample disks, if you want to play around with them. Density is exactly the same as FDS. I've used FDSStick to read them out before, but haven't tried analyzing the format.

Charizard700
Posts: 11
Joined: Thu Nov 26, 2020 1:15 am

Re: .fds format: Can checksums be heuristically detected?

Post by Charizard700 » Sat Dec 19, 2020 11:42 am

There seems to be a QDF format that includes gaps and checksums. It seems that there are other RAW formats (QDK, FDK).
VirtuaQD - FDS/QD emulator -
http://www.2a03.jp/~norix/virtuaqd/index.html

Details can be downloaded from "イメージファイル変換ツール(Image File Conversion Tool)".

It seems that the capacity of Famicom Disk and other systems is different due to the difference in bit rate.
Famicom Disk Approx. 95.8Kbps, Actual capacity approx. 80KB
Other system Quick disk approx. 101Kbps, Actual capacity approx. 70KB

Post Reply