N64 16Kbit Eeprom saves and padding. (Solved)

Discussion of development of software for any "obsolete" computer or video game system.
Post Reply
BOBdotEXE
Posts: 4
Joined: Mon Jan 09, 2017 1:38 pm

N64 16Kbit Eeprom saves and padding. (Solved)

Post by BOBdotEXE » Sat Mar 14, 2020 2:13 pm

Hey, so long story short, I'm trying to find out how important padding is for N64 eeprom saves,


I've been following an modified version of the Gameshark+DexDrive method, so all I have is the raw save data to work with.

I'm actually working on making a video guide the shows the same process, but using a more modern adapter.

But from messing around with emulators, it looks like some n64 games that use 16Kbit EEPROM produce padded saves.
Tons of '00' lines at the beginning and end of the fine, with the save in the middle.
(example DK64)

However If I remove everything, and just input my gameshark dumped data,(no padding) the game seems to load fine,
After running, the emulator seems to add padding. But only to the end of the file.
I've also found .eep saves online that only have padding at the beginning, and not the end.


So I'm beginning to question if the padding actually serves a purpose, do older emulators still need it?
or Is it required for transferring using a dedicated cart dumper?

I already know how to manually overwrite the save data while keeping the emulators padding, but I'm just wondering if it's worth the trouble.
The game seems to load fine either way.

If someone has a way to see how the cartridge actually stores the data, that may be helpful to me.

For reference I've included three save files,
DONKEY KONG 64.eep - Save found online (only padded at start)
DONKEY KONG 64_2.eep - Same save as before, but run though mupen (padding at end added by mupen)
Donkey Kong 64 (U) [!].eep - Raw gameshark dump run through mupen, (padding at end added by mupen)
Attachments
dk64_save.zip
save files referenced in post.
(1.6 KiB) Downloaded 53 times
Last edited by BOBdotEXE on Sat Mar 14, 2020 9:17 pm, edited 2 times in total.

lidnariq
Posts: 9706
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: N64 16Kbit Eeprom saves and padding. (help)

Post by lidnariq » Sat Mar 14, 2020 2:26 pm

The physical device on the cartridge holds exactly 16 Kibit, arranged as 256 planes of 8 bytes=64 bits each. By removing the padding, you just add undefined behavior as to what to do with an undersize file. Different emulators or methods of putting data in an EEPROM will do different things, some of which may not matter, but some of which might cause the game to object.

Also, things that look like padding compress extremely well - you'll notice that both "DONKEY KONG 64.eep" and "DONKEY KONG 64_2.eep" both compress to 277 bytes, despite the former having its padding removed.
BOBdotEXE wrote:
Sat Mar 14, 2020 2:13 pm
If someone has a way to see how the cartridge actually stores the data, that may be helpful to me.
The file is literally just the 256 8-byte entries in the EEPROM in order, lowest numbered plane first, first byte on the JOYBUS first.


A 4 Kibit EEPROM is almost the same, but only 64 planes instead.

BOBdotEXE
Posts: 4
Joined: Mon Jan 09, 2017 1:38 pm

Re: N64 16Kbit Eeprom saves and padding. (help)

Post by BOBdotEXE » Sat Mar 14, 2020 2:38 pm

lidnariq wrote:
Sat Mar 14, 2020 2:26 pm
The physical device on the cartridge holds exactly 16 Kibit, arranged as 256 planes of 8 bytes=64 bits each. By removing the padding, you just add undefined behavior as to what to do with an undersize file. Different emulators or methods of putting data in an EEPROM will do different things, some of which may not matter, but some of which might cause the game to object.
Ah, okay, thanks for the info!

So I assume it's always best to keep the padding if possible.

So it sounds like the emulator may just start reading save data where ever it finds it.
(with or without padding)
I've tested a few 16Kb games, and they've all seemed to work without the padding,

Is it working because of something the emulator is doing, or do games normally start reading save data at the first non-null byte?
(sorry if I explain this poorly, I'm kinda new to this stuff)

In short, I'm wondering if the games (on real hardware) always start reading at a fixed offset,
or if they just look for any readable data, and start there.

---

On a side note, do you happen to know if anyone's figured out how to decompress gameshark sram backups?

I think I found one really old guide, that required encoding conversion, but it did not work for me.

lidnariq
Posts: 9706
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: N64 16Kbit Eeprom saves and padding. (help)

Post by lidnariq » Sat Mar 14, 2020 2:57 pm

BOBdotEXE wrote:
Sat Mar 14, 2020 2:38 pm
So I assume it's always best to keep the padding if possible.
There's certainly no advantage to trimming them. Removing the padding won't make it compress better, won't make it take less space on disk, and is such a tiny amount of data there's no advantage in terms of network bandwidth.
So it sounds like the emulator may just start reading save data where ever it finds it.
(with or without padding)
I'd assume the emulator just tries to read 2KiB from the disk, and if it doesn't get a full 2KiB the end is either padded or random garbage. Either way, not really any benefit.
I've tested a few 16Kb games, and they've all seemed to work without the padding,
It's likely that any game that doesn't write new values to those addresses won't ever access them, and so won't care if there's padding. Nonetheless, the physical device still held something in those bytes.
Is it working because of something the emulator is doing, or do games normally start reading save data at the first non-null byte?
(sorry if I explain this poorly, I'm kinda new to this stuff)
In short, I'm wondering if the games (on real hardware) always start reading at a fixed offset, or if they just look for any readable data, and start there.
I'd be shocked if any games worked with padding at the beginning removed. The game has to do something like "Give me the data for plane N" and the EEPROM replies with 8 bytes.
On a side note, do you happen to know if anyone's figured out how to decompress gameshark sram backups?
I know nothing, sorry.

BOBdotEXE
Posts: 4
Joined: Mon Jan 09, 2017 1:38 pm

Re: N64 16Kbit Eeprom saves and padding. (help)

Post by BOBdotEXE » Sat Mar 14, 2020 4:27 pm

lidnariq wrote:
Sat Mar 14, 2020 2:57 pm
Ok, thanks again for the info, it was all very helpful!

Edit: Ok, I think I figured out what was confusing me.


Most games don't seem to be like this, but it looks like dk64 reads whatever it sees first as data for the first save slot,

When slot 1 is empty (or deleted), 0-18f Will also be empty. (Just full of '00')

I assumed it was extra padding when I generated a new save, but turns out, it was just an empty save slot 1.

lidnariq
Posts: 9706
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: N64 16Kbit Eeprom saves and padding. (help)

Post by lidnariq » Sat Mar 14, 2020 8:03 pm

Are there 5 save slots? If so, I'll point out that 16kbit = 2048 bytes = 400*5+48 and 0x18F+1 = 400

BOBdotEXE
Posts: 4
Joined: Mon Jan 09, 2017 1:38 pm

Re: N64 16Kbit Eeprom saves and padding. (help)

Post by BOBdotEXE » Sat Mar 14, 2020 9:16 pm

lidnariq wrote:
Sat Mar 14, 2020 8:03 pm
Are there 5 save slots? If so, I'll point out that 16kbit = 2048 bytes = 400*5+48 and 0x18F+1 = 400
There are only 3, But I think there may be extra data for 'bonus unlocks' that are not tied to a specific save.

Luckily, I'm not working with save editing, so I don't really need to track down exactly what slot is stored where,

I just found it kinda odd that in dk64, deleting save 1 leaves the start of the save 'empty'.
most other 16k games I've tested a least have some data at 0x0. (But again, not a problem for me.)

Post Reply