Ethics of repairing FDS headers?

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

Post Reply
Founder of higan project
Posts: 1550
Joined: Mon Mar 27, 2006 5:23 pm

Ethics of repairing FDS headers?

Post by Near » Fri Apr 05, 2019 9:41 pm

I came across this in most of my No-Intro FDS images:

I've been told that they've since redumped most of these images, but the bad images are still apparently floating around.

It seems that all sides of the images have block 1 0x1f-0x37 zeroed out, and all sides after the first contain a tool string.
So basically, it's DiskDude!, but this time in an actual part of the game itself that games can read if they wanted to.
I'm most concerned about the actual side byte that is zeroed out by this. The dates could be displayed onscreen in a game, and the values would no longer be valid BCD values.

higan operates in two modes: one is that the original file is parsed, I add back the gaps and checksums if they're missing, and I write out each side into per-game folders. The other is that I hold the transformed data in memory. Either way, the original files are never modified.

So, I'm asking here to get people's input: is it a good idea or a bad idea to try and repair the headers via an opt-in setting in my emulator?

The idea would be to collect a set of game hashes + known-good headers into a database, by scanning against the latest No-Intro set. When I find a match and see the "hCON" string, I'll restore the header with the copy from the database.

The tricky part is, how to compute a hash that's valid for both images. Because I can't know for certain which blocks are save data and may have been modified. I could probably match based on header bytes 0x00-0x1e alone?

The trickier part is, what if there's no match in the database? I hacked together a quick simple repair that will at least get us valid dates and disk side information, on the off chance a game tries to read that data and possibly display it on screen.

But the ethical concern here is that there's an infinitesimal chance someone actually uses higan, and then an even smaller one they use it for FDS emulation, and then an even smaller one that they try and reseed these "repaired" images, and it may not be immediately obvious that a repair was attempted, and may fool someone into thinking it's a clean image whereas the "hCON" version is obviously tampered with.

If the majority view here is that I shouldn't do this, I'll gut the code out.

So, thoughts?

Code: Select all

  for(uint index : range(size() / 65500)) {
    auto side = &data[index * 65500];
    if(memory::compare(&side[0x20], "hCON by hal9999", 15)) vandalized = true;
  if(!vandalized) return;

  //attempt to rebuild block 1 header for all disk sides
  for(uint index : range(size() / 65500)) {
    auto side = &data[index * 65500];

    //zero out the vandalism
    for(uint index = 0x20; index <= 0x37; index++) side[index] = 0x00;

    //repair damage as best we can: the values won't be correct, but at least they won't be invalid
    side[0x1f] = 0x61;  //1986 (Shouwa 61)
    side[0x20] = 0x01;  //January
    side[0x21] = 0x01;  //1st
    side[0x22] = 0x49;  //country code: Japan
    side[0x23] = 0x61;  //unknown (constant?)
    side[0x26] = 0x02;  //unknown (constant?)
    side[0x2c] = side[0x1f];  //rewritten date
    side[0x2d] = side[0x20];  //same as manufactured date if never rewritten
    side[0x2e] = side[0x21];
    side[0x2f] = 0xff;
    side[0x30] = 0xff;
    side[0x31] = 0xff;
    side[0x32] = 0xff;
    side[0x33] = 0xff;
    side[0x35] = index & 1;   //disk side (0x00 = side A, 0x01 = side B)

User avatar
Posts: 7824
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada

Re: Ethics of repairing FDS headers?

Post by rainwarrior » Fri Apr 05, 2019 11:33 pm

byuu wrote:but this time in an actual part of the game itself that games can read if they wanted to.
Well, it's a little bit different that corrupting part of a ROM that's going to be continually present in memory. Corrupted areas of a ROM can be casually read by accident. Data like this off the disk has an extremely low probability of being read by accident, especially if you're talking about the disk info block and not data in a file block that the BIOS load routines will actually copy into RAM for the game program.

So, in the ideal sense, yeah it was bad, but in practice, it's not that bad, maybe? I think the information lost is a whole lot more relevant (and worthwhile) for game archaeology / history than it is for emulation accuracy.

Posts: 218
Joined: Sun Mar 27, 2016 7:56 pm

Re: Ethics of repairing FDS headers?

Post by Nicole » Sat Apr 06, 2019 12:00 am

I wouldn't try repairing the header. Especially considering the information is mostly important for game preservation, I wouldn't risk having "repaired" bad dumps floating around.

Just slap a "Bad FDS header" warning message in the corner of the screen for a couple seconds. For those who are bothered by it, they'll no doubt track down a good dump instead, which means fewer copies of the bad dump hanging around. For those who don't care and just want to play the game, they'll just ignore it.

Founder of higan project
Posts: 1550
Joined: Mon Mar 27, 2006 5:23 pm

Re: Ethics of repairing FDS headers?

Post by Near » Sat Apr 06, 2019 2:35 am

Yeah, I had the same fear. I'll drop the heuristic repair attempt, and only potentially repair if I know I can transform it directly to a known verified dump, eg the header database idea. I like the idea of detecting this and displaying a 'known bad dump' warning, too.

Post Reply