Broken FDS emulation: Kiki Kaikai - Dotou Hen

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

User avatar
thefox
Posts: 3134
Joined: Mon Jan 03, 2005 10:36 am
Location: 🇫🇮
Contact:

Post by thefox »

I tried manually patching the correct amount of files in the file amount block (wrote a small Python tool to calculate the actual number of files), and guess what, now it works:

Code: Select all

Comparing files Kiki Kaikai - Dotou Hen (1987)(Taito Corp.).fds and KIKI KAIKAI
- DOTOU HEN (1987)(TAITO CORP.) - PATCHED.FDS
00000049: 08 09
00010025: 0C 0D
The original question of course still stands... is this just a bad disk or is there something that can be done to improve FDS emulation. Patching the number of files on load time doesn't seem like the Right Thing(TM) to do. :)
User avatar
BMF54123
Posts: 410
Joined: Mon Aug 28, 2006 2:52 am
Contact:

Post by BMF54123 »

Weird. Was that some form of copy protection back in the day (specifying the wrong number of files so disk copying tools would fail to copy everything)? Does an actual FDS even look at that value?
3gengames
Formerly 65024U
Posts: 2284
Joined: Sat Mar 27, 2010 12:57 pm

Post by 3gengames »

BMF54123 wrote:This "throw chunks of mystery code around and see if they stick" mentality to fixing emulation problems is kinda disappointing.
*Spitfire_NES* wrote:I wonder what that all means....lol :p
Your enthusiasm is commendable, but...if you don't actually know what the code is doing, perhaps you shouldn't be trying to "fix" the problem, lest you introduce even more somewhere down the road.
I thought they were trying to pinpoint the problematic code.

Hmmm weird thing...
User avatar
plasturion
Posts: 63
Joined: Thu Jun 02, 2011 2:05 am
Contact:

Post by plasturion »

thefox wrote:I tried manually patching the correct amount of files in the file amount block (wrote a small Python tool to calculate the actual number of files), and guess what, now it works:

Code: Select all

Comparing files Kiki Kaikai - Dotou Hen (1987)(Taito Corp.).fds and KIKI KAIKAI
- DOTOU HEN (1987)(TAITO CORP.) - PATCHED.FDS
00000049: 08 09
00010025: 0C 0D
The original question of course still stands... is this just a bad disk or is there something that can be done to improve FDS emulation. Patching the number of files on load time doesn't seem like the Right Thing(TM) to do. :)
Good Job!!!
[Here] is two of Nestopia dir log files - original and patched kiki image, the additional files appeared with patched form. So it's strange that number of existing files is written inside the real image FDS, or the image format? Why recalculating this value by emulation is incorrect (like any other dir command in file system), because the hardware can't do it, ...maybe it can?
User avatar
*Spitfire_NES*
Posts: 306
Joined: Fri May 21, 2010 4:10 pm

Post by *Spitfire_NES* »

Great job fox!
Last edited by *Spitfire_NES* on Mon Jul 25, 2011 8:19 am, edited 1 time in total.
User avatar
plasturion
Posts: 63
Joined: Thu Jun 02, 2011 2:05 am
Contact:

Post by plasturion »

...i wish i could have a [x] button somewhere. =]
Last edited by plasturion on Mon Jul 25, 2011 1:35 pm, edited 3 times in total.
User avatar
*Spitfire_NES*
Posts: 306
Joined: Fri May 21, 2010 4:10 pm

Post by *Spitfire_NES* »

........
Last edited by *Spitfire_NES* on Mon Jul 25, 2011 8:19 am, edited 1 time in total.
User avatar
plasturion
Posts: 63
Joined: Thu Jun 02, 2011 2:05 am
Contact:

Post by plasturion »

BMF54123 wrote:Weird. Was that some form of copy protection back in the day (specifying the wrong number of files so disk copying tools would fail to copy everything)? Does an actual FDS even look at that value?
Yes it looks like it has something with copy protection...
Apparently, some FDS disks implement a very simple copy protection scheme, which the game relies on in order for the game to refuse to work on the copied disk. Normally, the number of files that exist on an FDS disk is stored in the second block recorded on it. However, some games maintain "invisible" files, which are basically files that exist beyond what the file count number in the file count block indicates. This poses somewhat of a problem for copy software like FDSLOADR, since these tools rely on the file count block, and don't assume that there is any valid data past the last file found on the disk. This means that when these types of disks are copied, the invisible files will be lost, and when the game loads the files that do exist, the game's going to give the user heat about there being a file missing or somthing, gumming up the works. However in practice, when an FDS disk is programmed, the unused end of the disk is usually completely zeroed out, and this makes detecting the end of the disk simple: just wait to find a GAP period of extreme length. Except in rare cases, this model for detecting the true end of an FDS disk should generally provide the best results for copying the complete contents for all types of FDS disks.
If the file count block stores info about number of common files, how the real FDS is able to know about of hidden files? Is this realize somehow by BIOS? Maybe it counts while scanning disc surface.
User avatar
thefox
Posts: 3134
Joined: Mon Jan 03, 2005 10:36 am
Location: 🇫🇮
Contact:

Post by thefox »

plasturion wrote:If the file count block stores info about number of common files, how the real FDS is able to know about of hidden files? Is this realize somehow by BIOS? Maybe it counts while scanning disc surface.
The "problem" is emulators require that very same BIOS! If it does any counting, it should count on emulators as well.

I'm wondering if there can be any condition that when dumping this game the file count would have accidentally be bumped down by 1, but the dumper didn't notice because the game seems to work fine in the beginning.

I think the only way to figure this one out is to see if the game does anything funky in its loading routines that would allow it to work correctly on the actual hardware, unless somebody actually can copy the game back to a FDS disk and verify that it works on hardware. Personally I've lost interest in this issue, too many open questions.
User avatar
plasturion
Posts: 63
Joined: Thu Jun 02, 2011 2:05 am
Contact:

Post by plasturion »

I think we can treat this issue rather like a bad dump, and we no need to know if this image works on the real FDS hardware.

Let's see some changelog part of T.Takeda when he add this autopatching feature to Nester.
2001/03/22
code fixed 20
tested 20

# support 'VS Super Skater (Bootleg)'
# support 'Yuushi no Monshou (Deep Dungeon)'.

# change disk save file format (same as fds format / can load old files).
# change disk state save file format (can not load old files).
# calcurate number of files in fds image when load.
Yuushi no Monshou (Deep Dungeon) is Deep Dungeon II and can be found in earlier TOSEC romset pack as Yuushi no Monshou (1987)(Humming Bird Soft).zip, and as another file know as Deep Dungeon II - Yuushi no Monshou (Japan) (v1.1) .fds and both are the same.
I confirmed that this image is the very same issue like Kiki kaikai. I used nestopia log to see all the file list with both sides, the last files are:

Fds: Disk 1 Side A: 44k in 12 files, 17b trailing data..
Fds: file: "KYODAKU-", id: 0, size: 224, index: 0, address: 0x2800, type: NMT
...
Fds: file: "OPEN6/25", id: 0, size: 8736, index: 11, address: 0xA000, type: PRG
-------------------------------------------------------------------
Fds: Disk 1 Side B: 47k in 13 files, 18b trailing data..
Fds: file: "FLOOR12F", id: 1, size: 5960, index: 0, address: 0xC800, type: PRG
...
Fds: file: "MAP0PRO", id: 0, size: 2752, index: 12, address: 0xC800, type: PRG



and this is fixable in the same way like Kiki (by incrementation number of files by one), so patched rom show as additional (probably hidden) files that the common emulation has now access:
Fds: Disk 1 Side A: 44k in 13 files..
Fds: file: "KYODAKU-", id: 0, size: 224, index: 0, address: 0x2800, type: NMT
...
Fds: file: "OPEN6/25", id: 0, size: 8736, index: 11, address: 0xA000, type: PRG
Fds: file: "CHARA", id: 96, size: 144, index: 12, address: 0xB300, type: PRG
-------------------------------------------------------------------
Fds: Disk 1 Side B: 47k in 14 files..
Fds: file: "FLOOR12F", id: 1, size: 5960, index: 0, address: 0xC800, type: PRG
...
Fds: file: "MAP0PRO", id: 0, size: 2752, index: 12, address: 0xC800, type: PRG
Fds: file: "DUMMY", id: 170, size: 1, index: 13, address: 0xDFF0, type: PRG



there's also "correct" dump of previous version of this game know as Deep Dungeon II - Yuushi no Monshou (Japan) (v1.0)
Fds: Disk 1 Side A: 44k in 13 files..
Fds: file: "KYODAKU-", id: 0, size: 224, index: 0, address: 0x2800, type: NMT
...
Fds: file: "OPEN5/12", id: 0, size: 8736, index: 11, address: 0xA000, type: PRG
Fds: file: "CHARACTE", id: 96, size: 144, index: 12, address: 0xB300, type: PRG
-------------------------------------------------------------------
Fds: Disk 1 Side B: 47k in 13 files, 18b trailing data..
Fds: file: "FLOOR12F", id: 1, size: 5960, index: 0, address: 0xC800, type: PRG
...
Fds: file: "MAP0COM", id: 0, size: 1568, index: 11, address: 0xD800, type: PRG
Fds: file: "MAP0PRO", id: 0, size: 2752, index: 12, address: 0xC800, type: PRG

...and the same way while changing amount files of second disk side to 14 like in previous example we receive the DUMMY file, 1st side seems ok. I only wonder is this file has impact to code execute and is it used in english translation. (Maybe CHARA / CHARACTE is only save file, cause deleting this file relay only the same value we are talking about. In that case v1.1 has deleted save file)

In the same way (incrementation amount files by 1) I've tried to found dumps of other games that could has similar issue but I didn't found, even correct Doki Doki that someone confirmed on board that has copy protection, all the list was complete with visible DUMMY file. So even the hardware can work with hidden files, ripping tool probably check is there hidden files and recalculate this value makes all of them visible to avoid emulation problem, (and maybe there's some condition that this "bad" dumps occur (noticed that when I incremented amount files, sometimes SAVE files was appeared, and usually is placed as last additional file, maybe ripping tool decrease files number to hide unneeded or previous deleted SAVE file, but is not perfect)). If is true I think there's no need to improve emulation, even "bad dump" means "real dump"

In that case T.Takeda could know about weak point of dumping tool and wanted to enhance support with this kind of bad dumped FDS images. Lower value amount files of Kiki Kaikai, could be also result of bug code while playing, before was dumped. Last question is, does other emulator need this autopatching solution of T.Takeda to auto-fix bad dumps and enhance support of current romset receiving all deleted needed or not files every single game? (It could generate problems with unneeded, last deleted files).

[Here] is requested patch - Kiki Kaikai fixed.
User avatar
*Spitfire_NES*
Posts: 306
Joined: Fri May 21, 2010 4:10 pm

Post by *Spitfire_NES* »

I extensively tested this game with the new fix on this game today and was able to fight 2 of the 4 bosses before it goes back to freezing on a boss room. Would incrementing more invisible files perhaps prevent this. Just wanted to report my findings.
User avatar
plasturion
Posts: 63
Joined: Thu Jun 02, 2011 2:05 am
Contact:

Post by plasturion »

OK... so you confirm that this patch is working now. I think that's obvious that this value should be correct (all the necessary files must be visible (except some unnecessary save or extra files deleted before was dump)). If you increase more invisible files you make more visible files deleted or in other words "decrease amount of visible files with necessary data", and game cannot be able to access to this data, because this value during loading files tells that they not exist (that's the reason why kiki cannot load boss files into $7364 while loading second stage).

... i'm afraid you didn't found anything, this was told before thousand times, check that post before you said "i wonder what all of this mean", and please try to understand.
User avatar
*Spitfire_NES*
Posts: 306
Joined: Fri May 21, 2010 4:10 pm

Post by *Spitfire_NES* »

I did check. I started a new game and it was patched correctly. I fought the first 2 bosses successfully and when I got to the third it froze.

I may not completely understand how the fds works but im not so ignorant that I don't know if the game froze. Ill try again and see what happens. I was just reporting my findings here. :)
User avatar
plasturion
Posts: 63
Joined: Thu Jun 02, 2011 2:05 am
Contact:

Post by plasturion »

Did you patched game with the same name like ips patch, and with header that contains "FDS" at the beginning? After applying patch into image - do you have the same file list (look at last files in nestopia log)?
Fds: Disk 1 Side A: 51k in 9 files..
Fds: file: "KYODAKU-", id: 0, size: 224, index: 0, address: 0x2800, type: NMT
...
Fds: file: "end", id: 30, size: 8124, index: 7, address: 0x6000, type: PRG
Fds: file: "WORKSS", id: 80, size: 33, index: 8, address: 0xDC10, type: PRG
-----------------------------------------------------
Fds: Disk 1 Side B: 48k in 13 files..
Fds: file: "mchar", id: 16, size: 6348, index: 0, address: 0x6000, type: PRG
...
Fds: file: "kitune", id: 27, size: 1180, index: 11, address: 0x7364, type: PRG
Fds: file: "jyzsoh", id: 28, size: 1114, index: 12, address: 0x7364, type: PRG
Did you played the same game before you applied this patch? You must to know that nestopia create some overriding patch image to avoid changing real file (write to disk is common, so it's a safety solution).
After all of this try to copy patched game into different folder or rename it.

IMPORTANT!!! You can't use previous created nestopia save state (created before you patched game) - they for sure contains wrong (old) amount files value.

...or do you suggest that during playing something happen and occur that game freeze in boss room like before? Please report more details.

[Try this patch]
It has deleted (hidden) save file ("WORKSS"), and visible boss file ("jyzsoh"). I think it's more correct, we don't need this previous (dumper) save file, maybe it's corrupted or loading this makes amount files of side B to wrong value. Anyway when you die, and choose first option this user save file will be created (overwritten).
I've played little more and beaten 5 different bosses or sub-bosses (don't know which one are you talking about, which one are third) I fought some fox, fatman, longman, even some tanuki that dressed up my cloths, and boss room never freezed. Maybe try to make some screen or describe where did you found this freeze rooms.
User avatar
*Spitfire_NES*
Posts: 306
Joined: Fri May 21, 2010 4:10 pm

Post by *Spitfire_NES* »

thanks plast. i will try it again and see if produces the same results. :)
Post Reply