Reverse-engineering DLDI specs for NDS

Discussion of development of software for any "obsolete" computer or video game system. See the WSdev wiki and ObscureDev wiki for more information on certain platforms.
nocash
Posts: 1405
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Reverse-engineering DLDI specs for NDS

Post by nocash »

Forward: Don't read unless you want to see me making a fool of myself.
This thread is about rev-engineering other people's open-source code, doing that is a bit foolish, isn't it? More so when doing it on a PC that doesn't support open-source.

---

Things I never could make sense of:

DLDI

As far as I understand that's some sort of driver for using different flashcarts with homebrew NDS titles, and it's extremely popular in most of the homebrew scene, and there's no technical documentation about how it's working. There's some open source code available, including a "template" for the driver structure, but it's all very abstract, not really providing info on which bytes are to be stored at which address for which purpose. Or does anybody know how to make sense of it?

For DSi (and unlaunch), it might be useful to supply a DLDI driver that allows old NDS homebrews to read from the DSi's SD/MMC slot instead from flashcart. Is there already such a DLDI driver?
The two possible problems would be: A few titles might want to access both flashcart (via DLDI) and SD/MMC slot (via 40048xxh), and might get confused if DLDI redirects to 40048xxh, too. And, NDS can access flashcarts via both ARM9 or ARM7, whilst DSi can access SD/MMC via ARM7 only (or would need a way to forward data from ARM7 to ARM9).
tepples wrote:Does "Writing a DLDI Driver" at Chishm's page answer any of your questions?
Not so much, the page itself doesn't answer anything, and the links on the page only bring up more questions. The only tech specification I could find is the part about "32KiB are allocated for DLDI patches", and, well, I already knew that (it's the standard answer when asking anybody "What the heck is that DLDI thing that you are talking about?").

Looking through some driver source files...

default.zip contains a DLDI header, and some nonfunctional dummy functions. And some comments like ".byte 0x0F @32KiB @ Log [base-2] of the size of this driver in bytes." - I am reading that "use your floating point maths to compute logarithm of value base minus 2 then publish your result on @socialmedia!!!" - or it might try to say "8000h = 32Kbytes = 1 SHL 15".
dlms.zip I can see it containing two files with program code, but can't see any DLDI header in there.
gmtf.zip Contains a DLDI header, and also actual code, that's potentially useful for further research.

Some of the files contain something called "dldi.ld" is it safe to ignore that, or is it important?
The files on the chism webpage are quite old. A while ago I have also seen newer (?) files with Rumble support aside from the sector reading/writing. I don't remember where I have seen that. And, when writing DLDI specs, should that include features like Rumble, too?

Then for the patching tools, there's https://github.com/devkitPro/dstools/tr ... c/dlditool on github - which doesn't allow me to download that source code file(s). Could somebody upload a copy of that package here?
Last edited by nocash on Thu Nov 15, 2018 12:19 am, edited 4 times in total.
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Reverse-engineering DLDI specs for NDS

Post by tepples »

"ld" is the linker in GNU Binutils. My first guess is that the .ld file is a linker config file, analogous to the file specified with -C when calling ld65.

Is there anyone left on http://forum.gbadev.org to answer DLDI questions?

What error do you get when clicking "Raw" on the web view of dlditool.c? And what error do you get when using Git to download the entire devkitPro/dstools repository or download it as a zipfile?
nocash
Posts: 1405
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Re: Reverse-engineering DLDI specs for NDS

Post by nocash »

Last time I had posted something on gbadev.org didn't work out too well.

I've managed to preview some of the github installer code on a tablet. Looks as if it contains a function named "quick find" and nobody has realized that it's slow as hell. Maybe I've misread the source code, but it looks as if it were scanning the whole file in 4-steps for finding the DLDI chunk - that would be about the slowest possible "quick find" method that I could think of (except maybe using 1-bit steps for non.byte-aligned addresses).
I guess the original idea was to install the driver once, and then keep the patched file unchanged for the rest of your life. But as far as I know flashcards have adopted the idea as auto-install the driver everytime when booting a game, in worst case that's meaning scanning a whole 128Mbyte file - only to find out that it doesn't contain a slot for DLDI driver at all.
Some possible workarounds might be scanning only the ARM9 and ARM7 sections of the file, hoping that the data isn't elsewhere. Even then, that would mean scanning up to several megabytes. Doing that on an ARM cpu might work via LDMIA r0-r9 registers, and then check r0-r9 for the first word of the ID via one CMP and nine CMPNE opcodes. And even then, the pure idea makes me like having smeared myself with someone else's dirt.

Ideally, the DLDI stuff should have been stored at fixed file location, or at a fixed location within ARM9 or ARM7 areas (eg. in their last 32Kbyte). But I guess the HLL people didn't know how to store data at a fixed location : /
I can't think of a way for fixing the design. One could scan the file once and then store the DLDI address in the carthdr. But that would mean modifying the carthdr - even for retail games (ie. putting a No DLDI flag in there).
Other idea would be scanning files once, and building a database, with info like <filename> was already scanned, and DLDI <does/doesn't> exist, located at <address>, plus a copy of the <TITLE> string to make sure <filename> really contains the expected file. Damn, no, the title is always "HOMEBREW", so maybe some CRC on some small but characteristic cart snippets.
Of course, if the database grows too large, then it might become faster to scan the .nds file than loading the database. Unless the database contains a quick look-up table with sorted entries.

Github is just doing the usual what-you-are-on-win98 stuff

Code: Select all

Unable to complete secure transaction
You tried to access the address https://github.com/devkitPro/dstools/archive/master.zip
Well, I am firmly resisting to update my OS just because they don't want me to download their high-security master.zip file via http.
Anyways, I'll have a look at the gmtf.zip file for now. The master.zip would be also interesting because I had spotted some stuff related to "adjusting addresses" in there.
nocash
Posts: 1405
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Re: Reverse-engineering DLDI specs for NDS

Post by nocash »

I have looked through the two source files in gmtf.zip, deleted the tab stops and other irrelevant stuff, and this is what I came up with. Understanding the __addresses would probably require more research via hex editors, but below should give at least some rough idea about the DLDI header.

I don't know where I got that files from, but some files on my SD card (BOOT_FC.NDS, for example) are throwing a "fatInitDefault failed!" message when starting them with unlaunch. I suspect that the error might be DLDI related, and if yes, should I cart about that and try to solve the issue in unlaunch?

Code: Select all

DS Cart DLDI Driver
-------------------

DLDI (Dynamically Linked Device Interface for libfat) is a popular yet
undocumented flashcart driver for homebrew NDS software. Below was
reverse-engineered 11/2018 from "gmtf.zip" source code.

file.dldi --> driver file (can be small like 1.5Kbyte, or max 32Kbyte)
file.nds --> ROM image (must contain 32Kbyte space with DLDI ID for patching)

Driver patch file standard header -- 16 bytes
  00h 4  DLDI ID Word    (BF8DA5EDh)
  04h 8  DLDI ID String  (20h,"Chishm",00h)
  0Ch 1  Version Number  (must be 01h, patching tools will refuse other values)
  0Dh 1  Size of .dldi file (rounded up to 1 SHL N bytes) (max 0Fh=32Kbytes)
  0Eh 1  Sections to fix (see FIX_xxx)
  0Fh 1  Space in .nds file (1 SHL N bytes) (0Fh in .nds, can be 0 in .dldi)
Text identifier - can be anything up to 47 chars + terminating null -- 48 bytes
  10h 48 Name of current driver (ASCII, max 47 chars, plus zero padding)
Offsets to important sections within the data -- 32 bytes
  40h 4  __text_start    @ data start
  44h 4  __data_end      @ data end
  48h 4  __glue_start    @ Interworking glue start    -- Needs address fixing
  4Ch 4  __glue_end      @ Interworking glue end
  50h 4  __got_start     @ GOT start                  -- Needs address fixing
  54h 4  __got_end       @ GOT end
  58h 4  __bss_start     @ bss start                  -- Needs setting to zero
  5Ch 4  __bss_end       @ bss end
IO_INTERFACE data -- 32 bytes
  60h 4  Driver ioType ID (eg. ASCII "GMTF" meaning Datel Games 'n' Music?)
  64h 4  Features (see FEATURE_xxx)
  68h 4  Function Address startup()                           ;0=fail, 1=okay
  6Ch 4  Function Address isInserted()                 ;0=no/fail, 1=yes/okay
  70h 4  Function Address readSectors(sector,numSectors,buf)  ;0=fail, 1=okay
  74h 4  Function Address writeSectors(sector,numSectors,buf) ;0=fail, 1=okay
  78h 4  Function Address clearStatus()                       ;0=fail, 1=okay
  7Ch 4  Function Address shutdown()                          ;0=fail, 1=okay
  80h .. Driver Code (max 7F80h bytes)
Constants:
  FIX_ALL                            01h   ;um, so 01h+02h means... what?
  FIX_GLUE                           02h
  FIX_GOT                            04h
  FIX_BSS                            08h
  FEATURE_MEDIUM_CANREAD             00000001h
  FEATURE_MEDIUM_CANWRITE            00000002h
  FEATURE_SLOT_GBA                   00000010h
  FEATURE_SLOT_NDS                   00000020h
startup, isInserted, clearStatus, shutdown can be dummy functions that nothing (other than returning "true" which is supposedly meaning r0=1).
Alternately startup/shutdown can initialize or power down the hardware (if the hardware requires such things), clearStatus is meant to be some sort of SD card soft reset, and isInserted is allowing to test if the SD card is inserted & working.

read/write sectors are reading/writing one or more sectors. Sector size is 200h bytes, sector numbers is 0=First 200h bytes, 1=Next 200h bytes, and so on.
The driver functions can support SD and SDHC (or the flashcart manufacturer might release driver updates if SDHC wasn't supported).
Higher level FAT functions must be contained in the .nds file (so a driver update won't help if the .nds file lacks FAT32, for example).

FIX_xxx does maybe relate to address adjustments made by the dldi installer. Unknown if/how that's working. Would require https download from github for details.

SLOT_GBA/NDS seem to relate to GBA and NDS slots, the driver can probably have only one of the SLOT bits set (the functions don't allow to select which slot to use).
Purpose is unclear to me, maybe just telling the .nds file that the flashcart is in the given slot (and thereby shouldn't expect other hardware in that slot). Or maybe telling telling the installer which hardware the driver is supposed for.

Some DLDI variants seem to support extra features like Rumble, unknown if/how that's working.
tepples wrote:"ld" is the linker in GNU Binutils. My first guess is that the .ld file is a linker config file, analogous to the file specified with -C when calling ld65.
Might be so. 7zip opens it with windows notepad on PC then showing an endless ASCII string without linebreaks. My tablet is telling me that I should join googlemail because the Play Store might have an application for viewing the .ld file content.
Well, I guess it's not worth reading the file (or learning how to configure linkers in general). Unless it's containing something important like forcing relocate-able code for DLDI...?
Last edited by nocash on Thu Nov 08, 2018 1:34 pm, edited 3 times in total.
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Reverse-engineering DLDI specs for NDS

Post by tepples »

nocash wrote:Last time I had posted something on gbadev.org didn't work out too well.
When was that?
nocash wrote:Looks as if it contains a function named "quick find" and nobody has realized that it's slow as hell. Maybe I've misread the source code, but it looks as if it were scanning the whole file in 4-steps for finding the DLDI chunk
I was under the impression that the version in flash card firmware looked only in the ARM9 and ARM7 parts, as you suggest. That's why when I designed GBFS, which searches GBA address space at runtime, I required the space for the first archive to be 256 byte aligned to save on search time.
nocash wrote:But I guess the HLL people didn't know how to store data at a fixed location : /
Correct. If more than one thing ends up requiring the same fixed location, they end up conflicting.
nocash wrote:Github is just doing the usual what-you-are-on-win98 stuff

Code: Select all

Unable to complete secure transaction
You tried to access the address https://github.com/devkitPro/dstools/archive/master.zip
Well, I am firmly resisting to update my OS just because they don't want me to download their high-security master.zip file via http.
I guess you could run a browser that uses a TLS library other than the OS's built-in SChannel. Might there be a build of Lynx, w3m, Links, or even Wget or cURL that works?

With KernelEx, Windows 98 users can run Firefox 3.6 stably and kinda-sorta run Firefox 9.
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Reverse-engineering DLDI specs for NDS

Post by tepples »

You edited in something about Windows Notepad "showing an endless ASCII string without linebreaks". That happens most often when a text file uses UNIX newlines (0A), not CP/M newlines (0D 0A). From the README file of my NES NROM template:
IF THIS FILE HAS NO LINE BREAKS: View it in a web browser.
Windows Notepad is a very popular text editor that comes with the Windows operating system, but it doesn't recognize line breaks in text files made on Linux or any other UNIX-like operating system.
Text files with UNIX line breaks display correctly in WordPad, Notepad++, Programmer's Notepad, Gedit, or a web browser, just not in Windows Notepad prior to the Windows 10 October 2018 Update.
What does WordPad show? What about versions of Emacs or Vim from the Windows 98 era? Or Notepad++ and Programmers Notepad before they dropped Windows 98 support?
nocash
Posts: 1405
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Re: Reverse-engineering DLDI specs for NDS

Post by nocash »

tepples wrote:What error do you get when clicking "Raw" on the web view of dlditool.c?
That link worked, indirectly, it's in google web cache, not the Raw version, but the one with line numbers is good enough.
Also here is a slightly older clean-text version http://www.smspower.org/maxim/forumstuff/dlditool.c in plain http
nocash
Posts: 1405
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Re: Reverse-engineering DLDI specs for NDS

Post by nocash »

Yeah, I have a text editor that can handle 0Dh or 0Ah instead of 0Dh,0Ah. I just wasn't in mood for unpacking the .zip and loading the .ld file into a normal text editor (sorry if I have confused everybody about what I wasn't doing).

---

I've tried to search for DLDI with Rumble. All I could find was the http://www.chishm.com/DLDI/index.html webpage mentioning a "SuperCard Rumble (SD Card)", but it's without source code. However there's source for a "SuperCard Lite (SD Card)"...

The source code contains two rumble functions, "scms_rumble_flash_Erase" and "scms_rumble_flash_write".
Now I am confused, I was thinking that rumble would refer to vibration functions.
But now it does look more as if it were referring to a memory technology called "rumble flash"?
Probably not.

---

Looking through dlditool.c, the stuff about loading the .dldi file is probably not so important, except maybe mentioning the DLDIPATH environment variable. But that's more on the users side, not related to the binary file format.
nocash
Posts: 1405
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Re: Reverse-engineering DLDI specs for NDS

Post by nocash »

The main part of the dlditool.c seems to be this
(with the little-endian 32bit readAddr/writeAddr functions abbreviated to rd and wr, and with addrIter abbreviated to i)
(and some blank lines & close brackets & error messages removed, too)

Code: Select all

    patchOffset = quickFind (appFileData, dldiMagicString, appFileSize, sizeof(dldiMagicString)/sizeof(char));

    pDH = dldiFileData;
    pAH = &appFileData[patchOffset];

    // Make sure the DLDI file is valid and usable
    if (strcmp ((char*)dldiMagicString, (char*)&pDH[DO_magicString]) != 0) { err
    if (pDH[DO_version] != DLDI_VERSION) { err
    if (pDH[DO_driverSize] > pAH[DO_allocatedSpace]) { err

    memOffset = rd(pAH, DO_text_start);
    if (memOffset == 0) {                                        <--- ???
                    memOffset = rd(pAH, DO_startup) - DO_code;   <--- ????????

    ddmemOffset = rd(pDH, DO_text_start);
    relocationOffset = memOffset - ddmemOffset;

    printf ("Old driver:          %s\n",     &pAH[DO_friendlyName]);
    printf ("New driver:          %s\n",     &pDH[DO_friendlyName]);
    printf ("Position in file:    0x%08X\n", patchOffset);       eg. 0x000062C0
    printf ("Position in memory:  0x%08X\n", memOffset);         eg. 0x02000000
    printf ("Patch base address:  0x%08X\n", ddmemOffset);       eg. 0xBF800000
    printf ("Relocation offset:   0x%08X\n", relocationOffset);  eg. 0x428060C0

    ddmemStart = rd(pDH, DO_text_start);
    ddmemSize = (1 << pDH[DO_driverSize]);                      <--- !!!
    ddmemEnd = ddmemStart + ddmemSize;                          <--- ?????

    // Remember how much space is actually reserved
    pDH[DO_allocatedSpace] = pAH[DO_allocatedSpace];     <---- IMPORTANT !!!
    // Copy the DLDI patch into the application
    memcpy (pAH, pDH, dldiFileSize);

    // Fix the section pointers in the header
    wr(pAH, DO_text_start,   rd(pAH, DO_text_start)  + relocationOffset);
    wr(pAH, DO_data_end,     rd(pAH, DO_data_end)    + relocationOffset);
    wr(pAH, DO_glue_start,   rd(pAH, DO_glue_start)  + relocationOffset);
    wr(pAH, DO_glue_end,     rd(pAH, DO_glue_end)    + relocationOffset);
    wr(pAH, DO_got_start,    rd(pAH, DO_got_start)   + relocationOffset);
    wr(pAH, DO_got_end,      rd(pAH, DO_got_end)     + relocationOffset);
    wr(pAH, DO_bss_start,    rd(pAH, DO_bss_start)   + relocationOffset);
    wr(pAH, DO_bss_end,      rd(pAH, DO_bss_end)     + relocationOffset);
    // Fix the function pointers in the header
    wr(pAH, DO_startup,      rd(pAH, DO_startup)     + relocationOffset);
    wr(pAH, DO_isInserted,   rd(pAH, DO_isInserted)  + relocationOffset);
    wr(pAH, DO_readSectors,  rd(pAH, DO_readSectors) + relocationOffset);
    wr(pAH, DO_writeSectors, rd(pAH, DO_writeSectors)+ relocationOffset);
    wr(pAH, DO_clearStatus,  rd(pAH, DO_clearStatus) + relocationOffset);
    wr(pAH, DO_shutdown,     rd(pAH, DO_shutdown)    + relocationOffset);

    if (pDH[DO_fixSections] & FIX_ALL) {       <--- who or what is an "ALL" ????
      // Search through and fix pointers within the data section of the file
      for (i = (rd(pDH, DO_text_start) - ddmemStart); i < (rd(pDH, DO_data_end) - ddmemStart); i++) {
        if ((ddmemStart <= rd(pAH, i)) && (rd(pAH, i) < ddmemEnd)) {  <--- ?@#!!
          wr(pAH, i, rd(pAH, i) + relocationOffset);                  <--- !!!!!

    if (pDH[DO_fixSections] & FIX_GLUE) {               <--- is that HLL stuff?
      // Search through and fix pointers within the glue section of the file
      for (i = (rd(pDH, DO_glue_start) - ddmemStart); i < (rd(pDH, DO_glue_end) - ddmemStart); i++) {
        if ((ddmemStart <= rd(pAH, i)) && (rd(pAH, i) < ddmemEnd)) {  <--- ?@#!!
          wr(pAH, i, rd(pAH, i) + relocationOffset);                  <--- !!!!!

    if (pDH[DO_fixSections] & FIX_GOT) {                <--- maybe HLL related?
      // Search through and fix pointers within the Global Offset Table section of the file
      for (i = (rd(pDH, DO_got_start) - ddmemStart); i < (rd(pDH, DO_got_end) - ddmemStart); i++) {
        if ((ddmemStart <= rd(pAH, i)) && (rd(pAH, i) < ddmemEnd)) {  <--- ?@#!!
          wr(pAH, i, rd(pAH, i) + relocationOffset);                  <--- !!!!!

    if (pDH[DO_fixSections] & FIX_BSS) {
      // Initialise the BSS to 0      <--- needed if the .dldi filesize didn't include BSS area?
      memset (&pAH[rd(pDH, DO_bss_start) - ddmemStart] , 0, rd(pDH, DO_bss_end) - rd(pDH, DO_bss_start));
Is that normal coding practice for NDS homebrew code, or is it totally wrecked, or both?
The "ALL" section seems to be "text..data", that is all CODE and DATA? And any constants or opcodes in that area are forcefully modified by "+relocationOffset" if their 32bit value happens to fall within the ddmemStart..End range?
Or is it even applied in 1-byte steps, meaning that the last byte of an opcode, and the first three bytes of the next opcode could be modified by "+relocationOffset"?
I believe that I begin to understand why nobody has documented that functionality yet.
Or did I download the wrong file from google webcache?
Or maybe I am just not too open-minded today, maybe I can make more sense of it tomorrow.
Last edited by nocash on Thu Nov 08, 2018 11:08 am, edited 1 time in total.
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Reverse-engineering DLDI specs for NDS

Post by tepples »

"scms_rumble_flash_write" looks like it stands for "flash write for SuperCard Mini/MicroSD Rumble". The DLDI doesn't support rumble by itself, but it does support block device access on an SD adapter that also happens to have rumble.
nocash
Posts: 1405
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Re: Reverse-engineering DLDI specs for NDS

Post by nocash »

I've been wondering if it were possible to use the DLDI driver - without using the bugged official DLDI installer.

Flashcarts with auto-patching might be already able to do that - they could probably repair the rom image in most cases, even if the official installer had damaged it previously.
For additional safety, the flashcarts could be shipped with encrypted DLDI drivers - so the official installer won't know how to install that patches. Hahaha, you didn't expect that, dlditool, didn't you!

Or simply change "Version (Must be 01h, else it won't work with the official installer),
to "Version (Must be anything OTHER than 01h, else it WILL work with the bugged official installer).

But the easiest workaround would be to create only DLDI drivers that are always having the FIX_xxx bits all set to zero - that should disabled the bugs, and then one could probably even use the official installer without running into troubles.
With that workaround, this whole stuff can be removed from the DLDI specs:

Code: Select all

    if (pDH[DO_fixSections] & FIX_ALL) {       <--- who or what is an "ALL" ????
      // Search through and fix pointers within the data section of the file
      for (i = (rd(pDH, DO_text_start) - ddmemStart); i < (rd(pDH, DO_data_end) - ddmemStart); i++) {
        if ((ddmemStart <= rd(pAH, i)) && (rd(pAH, i) < ddmemEnd)) {  <--- ?@#!!
          wr(pAH, i, rd(pAH, i) + relocationOffset);                  <--- !!!!!

    if (pDH[DO_fixSections] & FIX_GLUE) {               <--- is that HLL stuff?
      // Search through and fix pointers within the glue section of the file
      for (i = (rd(pDH, DO_glue_start) - ddmemStart); i < (rd(pDH, DO_glue_end) - ddmemStart); i++) {
        if ((ddmemStart <= rd(pAH, i)) && (rd(pAH, i) < ddmemEnd)) {  <--- ?@#!!
          wr(pAH, i, rd(pAH, i) + relocationOffset);                  <--- !!!!!

    if (pDH[DO_fixSections] & FIX_GOT) {                <--- maybe HLL related?
      // Search through and fix pointers within the Global Offset Table section of the file
      for (i = (rd(pDH, DO_got_start) - ddmemStart); i < (rd(pDH, DO_got_end) - ddmemStart); i++) {
        if ((ddmemStart <= rd(pAH, i)) && (rd(pAH, i) < ddmemEnd)) {  <--- ?@#!!
          wr(pAH, i, rd(pAH, i) + relocationOffset);                  <--- !!!!!
Because there is no way that that code could work for installing DLDI drivers. Or, well, maybe future generations will find a way to make DLDI drivers that are working with that code. As for now, I would treat it as an interesting unsolved mathematical problem...

As far as I understand, ddmemXxx are virtual addresses (usually at BF80xxxxh, which could, say, conflict with zzBF80yyh opcodes), so the compiler could automatically change that virtual addresses to other regions (but take care that they won't conflict with other opcodes - or with things like the "Chishm" string in DLDI header itself). Or the compiler could replace opcodes like "MOV r3,0" by "SUB r3,r9,r9" where needed. A lossy compiler could also try to replace constants like timeout=10000 by timeout=9999, or use two constants like timeout=4000+6000 if that helps on avoiding conflicts. And of course the compiler could insert NOPs or other dummy opcodes/constants where needed.
Also encrypted code might help: Then one would only need to make sure that the header & decryption code are installed without conflicts, and the remaining code could be stored as "code XOR random", and the compiler could brute-force different random values until finding a setting that works without conflicts. Or storing all bytes (xxh) as halfwords (FxFxh), and making sure that ddmem doesn't cover any addresses at FxFxFxFxh.
The interesting part is that "+relocationOffset" is an unknown constant (it depends on the location of the DLDI space in different .nds file(s), and the compiler can't know that location(s) in advance). So, if the installer had patched a word at offset 100h..103h, then the values at 101h,102h,103h could contain unknown new values (with patching applied on random basis). A possible workaround would be inserting dummy words like FFFFFFFFh (or similar values in opcode space) after each and every word, so 101h..103h would become FFxxxxxxh, FFFFxxxxh, FFFFFFxxh (which might be all outside of the selected ddmem space).

With the above three bugged FIX_xxx's removed, one could also remove this from DLDI specs:

Code: Select all

    if (pDH[DO_fixSections] & FIX_BSS) {
      // Initialise the BSS to 0      <--- needed if the .dldi filesize didn't include BSS area?
      memset (&pAH[rd(pDH, DO_bss_start) - ddmemStart] , 0, rd(pDH, DO_bss_end) - rd(pDH, DO_bss_start));
That part isn't strictly bugged, but it's pretty useless (assuming that driver creators would know how to zeropad their files on their own (though I might over-estimate their skills)).
Last edited by nocash on Thu Nov 08, 2018 10:48 am, edited 5 times in total.
nocash
Posts: 1405
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Re: Reverse-engineering DLDI specs for NDS

Post by nocash »

tepples wrote:"scms_rumble_flash_write" looks like it stands for "flash write for SuperCard Mini/MicroSD Rumble". The DLDI doesn't support rumble by itself, but it does support block device access on an SD adapter that also happens to have rumble.
Yeah, doing an image search, those SuperCards with Rumble seem to be all GBA slot devices (probably requiring a passme adaptor in the NDS slot).
For the GBA slot, there are three different official rumble methods http://problemkaputt.de/gbatek.htm#dscartrumblepak maybe the SuperCards are emulating one (or more) of those methods, or maybe they come up with a fourth method. But, yeah, it's apparently done via direct I/O, not via the DLDI driver.

Speaking of GBA memory. The GBA could map ROM to memory address 8000000h and up, and it could execute code in there.
But DLDI is for NDS, and that requires to transfer code from ROM to RAM before executing it.
At least that's so when reading from NDS slot (and in practice, it's almost always also done that way when reading NDS code from GBA slot).

The RAM addresses for NDS code are stored in NDS cart header - but the DLDI installer doesn't look at the NDS cart header at all. So the installer's buggy address adjustments seem to be yet incomplete...
Maybe it does require further address adjustments to be carried out by the code in the .nds file before using the DLDI functions?
Or maybe DLDI requires the ARM9 code always being located at, say, 2000000h?
Some DLDI drivers are said to work (or don't so) for ARM7, so then ARM7 might be also required to be located at a certain fixed RAM address?

Though the installer adjusts addresses relative to the ROM address, so to come up with fixed RAM addresses, one would also need fixed ROM address in the first place. For ARM9 one could say that it must be always located at ROM offset 4000h or the like, but I don't see how one could do the same for ARM7 (except by always zeropadding the ARM9 area to it's maximum size).
Vaporeon
Posts: 2
Joined: Sat Sep 26, 2015 5:15 am

Re: Reverse-engineering DLDI specs for NDS

Post by Vaporeon »

Metapad supports UNIX line endings and Windows 98 still.
http://liquidninja.com/metapad/
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Reverse-engineering DLDI specs for NDS

Post by tepples »

nocash wrote:Speaking of GBA memory. The GBA could map ROM to memory address 8000000h and up, and it could execute code in there.
But DLDI is for NDS, and that requires to transfer code from ROM to RAM before executing it.
At least that's so when reading from NDS slot (and in practice, it's almost always also done that way when reading NDS code from GBA slot).
Which means that if the firmware is waiting for the NAND flash (CF, SD, etc.) to have a sector ready, it can search for the DLDI signature in the previous sector(s) in order to patch the .nds executable as it streams in. This interleaving might be part of why you don't notice a wait when loading an application using DLDI.

In order to help you further, I might need to understand what went wrong last time you asked on forum.gbadev.org.
nocash
Posts: 1405
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Re: Reverse-engineering DLDI specs for NDS

Post by nocash »

I have dumped the BOOT_FC.NDS file to see how DLDI would look inside of actual .nds files:

Code: Select all

BOOT_FC.NDS
 000000: 2E 00 00 EA 00 00 00 00  00 00 00 00 23 23 23 23   ............####
 000010: 00 00 00 00 01 00 00 00  00 00 00 00 00 00 00 04   ................
 000020: 00 02 00 00 00 08 00 02  00 00 00 02 78 A2 03 00   ............x...
 000030: 00 A6 03 00 00 00 38 02  00 00 38 02 CC 24 00 00   ......8...8..$..
 ...
 000F70: 3C BC 90 46 99 46 A2 46  AB 46 F0 BD 48 67 02 02   <..F.F.F.F..Hg..
 000F80: EE A5 8D BF 00 76 03 02  44 4C 44 49 F8 B5 DE 46   .....v..DLDI...F
 000F90: 4E 46 45 46 57 46 E0 B5  07 00 01 20 98 46 0A AB   NFEFWF..... .F..
 ...
 001220: 02 20 E3 E7 A0 EF FF FF  E0 65 02 02 18 11 02 02   . .......e......
 001230: FF FF FF FF FF FF FF FF  FF FF FF FF FF FF FF FF   ................
 001240: ED A5 8D BF 20 43 68 69  73 68 6D 00 01 0E 00 0E   .... Chishm.....
 001250: 44 65 66 61 75 6C 74 20  28 4E 6F 20 69 6E 74 65   Default (No inte
 001260: 72 66 61 63 65 29 00 00  00 00 A0 E1 00 00 A0 E1   rface)..........
 001270: 00 00 A0 E1 00 00 A0 E1  00 00 A0 E1 00 00 A0 E1   ................
 001280: 40 10 00 02 40 50 00 02  00 00 00 00 00 00 00 00   @...@P..........
 001290: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................
 0012A0: 44 4C 44 49 00 00 00 00  C0 10 00 02 C0 10 00 02   DLDI............
 0012B0: C0 10 00 02 C0 10 00 02  C0 10 00 02 C0 10 00 02   ................
 0012C0: 00 00 A0 E3 1E FF 2F E1  00 00 00 00 00 00 00 00   ....../.........
 0012D0: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................
 0012E0: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................
 0012F0: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................
 ...
 0267B0: E9 52 00 06 00 00 00 00  ED A5 8D BF 20 43 68 69   .R.......... Chi
 0267C0: 73 68 6D 00 ED A5 8D BF  20 43 68 69 73 68 6D 00   shm..... Chishm.
 0267D0: 42 4F 4F 54 2E 4E 44 53  00 FF FF FF B8 54 00 06   BOOT.NDS.....T..
 0267E0: C8 54 00 00 62 6F 6F 74  73 74 75 62 A8 00 00 00   .T..bootstub....
 ...
 026940: 04 FE FF 02 60 01 00 00  EE A5 8D BF 20 43 68 69   ....`....... Chi
 026950: 73 68 6D 00 00 00 00 00  00 00 00 00 00 00 00 00   shm.............
 026960: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................
 ...
 03D540: 44 00 53 00 69 00 4D 00  65 00 6E 00 75 00 2B 00   D.S.i.M.e.n.u.+.
 03D550: 2B 00 0A 00 52 00 6F 00  62 00 7A 00 38 00 00 00   +...R.o.b.z.8...
 ...
Ah, okay, the BOOT_FC.NDS is DSiMenu++ by Robz8!
Though the carthdr[012h] unitcode says that it's a NDS title without DSi support... I guess it requires some bootstub hack to load that file in DSi mode.
For some reason there are four DLDI headers, at offset 1240h, 267B8h, 267C4h, and 26948h. The installer will only process one of them (the first one found in memory, ie. at 1240h), so the other three headers won't get patched (and the last one has the EDh byte replaced by EEh so it doesn't fully match the required ID anyways).

Now if you thought that everybody knew that DLDI was supporting 32Kbytes patches, look at 124Dh. That's only max 16Kbytes allocated (1 SHL 14).
I guess there might be also .nds files that have 32Kbyte allocated, but one can't trust on that. Maybe there are even some with less than 16Kbyte out there?

Looking at the carthdr ARM9 area:
- ARM9 ROM offset 200h
- ARM9 RAM address 2000000h
So the stuff at 1240h in ROM will appear at 2001040h in RAM.
I guess DLDI simply requires RAM to be located at ROM+1FFFE00h.
So if the carthdr were 4000h bytes in size and ARM9 code would be at ROM offset 4000h, then RAM load address would have to be 2003E00h, right?
Or alternately one would need the kind of bootstub hack that appears to be used for loading the above file, then one could use files with 200h-byte headers, as they were apparently generated by some old NDS compilers.

Looking at the Function addresses in the DLDI header, for example the init function at ROM 12A8h:
That's already set to 20010C0h (the correct function address in RAM, when using the RAM=ROM+1FFFE00h formula).
So there's apparently no further address adjustment needed to be carried out by the code in the .nds file.
Post Reply