3DS reverse engineering

Discussion of development of software for any "obsolete" computer or video game system. See the WSdev wiki and ObscureDev wiki for more information on certain platforms.
nocash
Posts: 1405
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Re: 3DS reverse engineering

Post by nocash »

I've learned a bit more about Imprecise Data Aborts, and experimented with writing wrong LGYFB scale values.
But I couldn't reproduce a relation betweeen the two things: I am getting Data Aborts on wrong LGYFB writes... but that does happen regardless of the "Imprecise Data Abort Disable" bit in CPSR.bit8 (ie. no matter if clearing it via CPSIE A or setting it via CPSID A).

Imprecise Data Aborts
That's described in ARM11 reference. As far as I understand that aborts could be from external hardware, or from internally delayed operations (like write buffer), in either case they are "imprecise" in so far that they don't belong to the "current opcode".
The exception seems to trigger the normal "Data Abort" exception vector (and there's probably some state info somewhere indicating what had caused it, I didn't look into that).
The disable bit in CPSR.bit8 is mainly intended to prevent nested calls to the abort handler (eg. LDR triggering an data abort, while an older (write buffered) STR write is also about to trigger an abort).

Wrong LGYFB scale values
The abort happens immediately when writing the scaling values (even when ARM7 is still stopped, and even when LGYFB transfers aren't started). Despite of the abort, the register is updated (with bit15 being written to all sign bits). For example, [10110340h]=00008234h, does trigger data abort, and expands the sign from 8000h to FC00h, and the new register value is then 0000FE30h (the lower 4bit (and upper 16bit) are always stripped, regardless of aborts).
homepage - patreon - you can think of a bit as a bottle that is either half full or half empty
profi200
Posts: 66
Joined: Fri May 10, 2019 4:48 am

Re: 3DS reverse engineering

Post by profi200 »

nocash wrote: Sun May 30, 2021 12:50 pm Good to know. I was aware of the "CPSIE A" opcode, but I didn't know what "A" is meant to do.
Basically, it's external aborts (eg. from DMA's), as opposed to the internal data/prefetch aborts (from CPU itself)?
As far as I understand, "CPSIE A" does clear CPSR bit8 to enable those imprecise aborts...
I couldn't find out what happens then... does it trigger one of the standard exception/interrupt vectors? With some status flag to indicate that the vector was triggered due to imprecise abort?
Yes, as far as i understand it's external and delayed (CPU) data aborts. It does trigger the data abort exception. You want to look at the "Data Fault Status Register" (DFSR) i think. That should tell you where the abort came from.
nocash wrote: Sun May 30, 2021 12:50 pm To me, it does look as if they are equally visible horizontally and vertically (when using the low resolution 400x240 pixel mode).
But it can depend on the source data (if the source contains many horizontal or vertical lines with high contrast).
And, it can also depend on how the source pixels are aligned to the scale pattern, especially when using two sharp pixels and one blurred pixel (eg. 4000h, 4000h, 2000h+2000h), and not so much when slightly blurring all pixels (eg. 3000h+FFFh, FFFh+3000h, 3000h+FFFh).
Dunno, i meant on normal usage you will notice the horizontal artefacts less than the vertical ones. If you look for them you will see both easily. These LCDs are also clearly not designed to be used in this orientation because the sub pixels are laid out in such a way that it appears as if there are horizontal lines/gaps when the opposite is the case. There are vertical gaps but they don't show as much.
nocash wrote: Sun May 30, 2021 12:50 pm In your lgyfb source code,

Code: Select all

#define REG_LGYFB_TOP_DITHPATT0 ((vu32*)(LGYFB_TOP_REGS_BASE + 0x100)) // 2 u32 regs with 4x2 pattern bits (mask 0xCCCC) each.
#define REG_LGYFB_TOP_DITHPATT1 ((vu32*)(LGYFB_TOP_REGS_BASE + 0x108)) // 2 u32 regs with 4x2 pattern bits (mask 0xCCCC) each.
I am quite sure that it should be:

Code: Select all

#define REG_LGYFB_TOP_DITHER0 ((vu32*)(LGYFB_TOP_REGS_BASE + 0x100)) // u32 reg with 4x2 pattern bits (mask 0xCCCC)
#define REG_LGYFB_TOP_DITHER1 ((vu32*)(LGYFB_TOP_REGS_BASE + 0x108)) // u32 reg with 4x2 pattern bits (mask 0xCCCC)
#define REG_LGYFB_TOP_DITHER2 ((vu32*)(LGYFB_TOP_REGS_BASE + 0x110)) // u32 reg with 4x2 pattern bits (mask 0xCCCC)
#define REG_LGYFB_TOP_DITHER3 ((vu32*)(LGYFB_TOP_REGS_BASE + 0x118)) // u32 reg with 4x2 pattern bits (mask 0xCCCC)
There are four dithering registers, at 8-byte aligned addresses (and same for bottom screen LGYFB_BOT registers, too).
There are 2 kinds of dithering. Temporal and spatial. Each of these dithering modes has 2 u32 regs if i recall correctly (notice how its a pointer). That's why i named them like this. I hate the dithering and it can't be fully turned off even with RGB8 and RGBA8 (but it's not visible with these).
nocash wrote: Mon May 31, 2021 11:34 am I've learned a bit more about Imprecise Data Aborts, and experimented with writing wrong LGYFB scale values.
But I couldn't reproduce a relation betweeen the two things: I am getting Data Aborts on wrong LGYFB writes... but that does happen regardless of the "Imprecise Data Abort Disable" bit in CPSR.bit8 (ie. no matter if clearing it via CPSIE A or setting it via CPSID A).

Imprecise Data Aborts
That's described in ARM11 reference. As far as I understand that aborts could be from external hardware, or from internally delayed operations (like write buffer), in either case they are "imprecise" in so far that they don't belong to the "current opcode".
The exception seems to trigger the normal "Data Abort" exception vector (and there's probably some state info somewhere indicating what had caused it, I didn't look into that).
The disable bit in CPSR.bit8 is mainly intended to prevent nested calls to the abort handler (eg. LDR triggering an data abort, while an older (write buffered) STR write is also about to trigger an abort).

Wrong LGYFB scale values
The abort happens immediately when writing the scaling values (even when ARM7 is still stopped, and even when LGYFB transfers aren't started). Despite of the abort, the register is updated (with bit15 being written to all sign bits). For example, [10110340h]=00008234h, does trigger data abort, and expands the sign from 8000h to FC00h, and the new register value is then 0000FE30h (the lower 4bit (and upper 16bit) are always stripped, regardless of aborts).
Could be that it works differently with the MMU off. I always have it on and the instruction pointed to with the exceptions i got were not right. DFSR is set to 0x1C06 on the abort triggered by the scaling matrix reg writes.

The DFSR should contain the abort source as stated above.

edit:
I tested this yesterday and indeed, it's an imprecise abort. If i write out of bounds values to the scaling matrix and disable imprecise aborts i don't get any exception (until i re-enable imprecise aborts).
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: 3DS reverse engineering

Post by tepples »

I found the rationale underlying this "interlacing-like behavior" in Game Boy systems' displays (and other LCDs for that matter). From the README file of 144p Test Suite for Game Boy and 160p Test Suite for Game Boy Advance (view on GitHub or download on NESdev):
To keep positive and negative voltages balanced, an LCD inverts each pixel's phase on alternate frames. Game Boy displays alternate this phase by row. Thus slight level differences between a pixel and its inverted counterpart can look like interlace. (See "LCD monitor technology and tests" by W. Andrew Steer.)
A few tests in the Suite, such as Full screen stripes, Motion blur, and Color bleed (the last SGB/GBC and GBA only), are good at demonstrating this behavior. (Lacking a No$gmb license as of yet, I can't thoroughly test GBC behavior in the emulator, only on hardware.)
nocash
Posts: 1405
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Re: 3DS reverse engineering

Post by nocash »

Yes, I think that is related to the VCOM voltage, too. As far as I understand, one could output pixels with postive or negarive voltages, for example, using VCOM+2V or VCOM-2V should both produce the same color (swapping between positive and negative helps against picture burn-in).
To get a perfect picture, VCOM must be calibrated to the middle between GND and Supply voltage, otherwise positive and negative will be slightly darker/brighter producing some flickering (and if the source pixels are also flickering or scrolled, that will produce the interlace-like effect).
I have checked some of my notes, and GBA and NDS seems to be all having VCOM potentiometers that could be used for calibration (dunno if or how DSi can do that, too).

A calibration tool could be as simple as displayin green/blue lines on left screen half, and blue/green ones on right half in even frames, and vice-versa in odd frames, and prompting the user to turn the potentiometer until both halves are looking identical.
Somebody in a 3DS forum mentioned that the "lines effect" could change appearance when the console has warmed up though, if that's true, then perfect calibration won't work.
homepage - patreon - you can think of a bit as a bottle that is either half full or half empty
profi200
Posts: 66
Joined: Fri May 10, 2019 4:48 am

Re: 3DS reverse engineering

Post by profi200 »

I knew about the pixels switching polarity quickly to not get stuck but i didn't know there are LCDs doing that on whole lines. That reminds me that IPS panels are prone to temporary "burn in". The IPS panels on New 3DS XL are no exception and there are a number of reports of people having this effect after playing a game with rapidly flickering graphics like the map in F-Zero. That probably interferes with the LCDs polarity switching causing this.

And yeah, i checked with my GBA SP AGS-001 and the LCD is indeed fast enough to clearly show the flickering. Dunno about AGS-101. I heard that LCD has quite bad pixel response times. The LCDs on DS lite are also quite bad if you are used to modern LCD panels.
nocash
Posts: 1405
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Re: 3DS reverse engineering

Post by nocash »

Yeah, flickering would cause only "positive" or only "negative" pixels, I didn't knew that a little flickering it could actually damage the hardware. At least it's only temporary burn-in in that case (Sony's Pocketstation seems to suffer from permanent burn-in, it's somehow having permanently "watermarked" the LCD screen with the Sony logo, even when power is switched off).
profi200 wrote: Sat May 22, 2021 7:26 am SD cards are complex beasts these days as the flash is getting cheaper but also increasingly shitty. They try to cram more and more bits into each flash cell which reduces the lifetime drastically. The workaround is controllers with complex flash management algorithms to distribute all writes equally across the whole flash...
This is several years old but still relevant and a good read: https://www.bunniestudios.com/blog/?p=3554
I knew error correction only from cdroms and dotcodes. Until reading the blog article, I wasn't aware that it's also used in flash chips. But yeah, it makes sense if the chip has a few bad bits and bytes (as opposed to completely broken worn out sectors).

I am wondering how that would relate to the sector size... for example, if the logical size is 200h bytes, and error corrections does take, say, 60h bytes, then the physical sector size would be 260h bytes... I don't how they would best arrange that in memory (or if the manufacturing process for sequential flash chips is even bound to power of 2 block sizes).

Anyways, something similar might be also going on in NDS/DSi/3DS game cartridges (at least those with write-able NAND memory). In the NDS cart thread, http://forums.nesdev.com/viewtopic.php? ... 45#p274136 the Samsung DSi NAND chip does have a dump-able reserved area, which does contain the cart firmware (ARM/THUMB code and blowfish keys), but reading that area includes unstable data bits and the code blocks aren't aligned to 200h-byte boundaries... it looks a bit as if the cart is returning "raw" data from the reserved area, without applying error correction.

I don't know if the 3DS NAND carts (aka CARD2 carts) do have similar dump-able reserved areas. If yes, that might be an easy way to get keys/seeds for RC4 and SNOW encryption. The 3DS NAND seems to have a different maker ID though (unlike DSi chips, they are apparently not made by Samsung) (and the 3DS NAND transfer protocol seems to be still unknown anyways).
homepage - patreon - you can think of a bit as a bottle that is either half full or half empty
profi200
Posts: 66
Joined: Fri May 10, 2019 4:48 am

Re: 3DS reverse engineering

Post by profi200 »

nocash wrote: Wed Jun 16, 2021 2:48 pm Yeah, flickering would cause only "positive" or only "negative" pixels, I didn't knew that a little flickering it could actually damage the hardware. At least it's only temporary burn-in in that case (Sony's Pocketstation seems to suffer from permanent burn-in, it's somehow having permanently "watermarked" the LCD screen with the Sony logo, even when power is switched off).
Yes, it usually goes away after a few hours. Would not call it damaging the hardware in that case but it's certainly not good. As for the Pocketstation i don't know. Never saw or owned one.
nocash wrote: Wed Jun 16, 2021 2:48 pm I am wondering how that would relate to the sector size... for example, if the logical size is 200h bytes, and error corrections does take, say, 60h bytes, then the physical sector size would be 260h bytes... I don't how they would best arrange that in memory (or if the manufacturing process for sequential flash chips is even bound to power of 2 block sizes).
No idea to be honest. The eMMC and SD card specs mention ECC but what the controller actually does for error correction is highly vendor specific. Something that is for sure is there are some reserve flash pages that get swapped in when one of the mapped pages goes bad. SSDs use the same technique.
nocash wrote: Wed Jun 16, 2021 2:48 pm Anyways, something similar might be also going on in NDS/DSi/3DS game cartridges (at least those with write-able NAND memory). In the NDS cart thread, http://forums.nesdev.com/viewtopic.php? ... 45#p274136 the Samsung DSi NAND chip does have a dump-able reserved area, which does contain the cart firmware (ARM/THUMB code and blowfish keys), but reading that area includes unstable data bits and the code blocks aren't aligned to 200h-byte boundaries... it looks a bit as if the cart is returning "raw" data from the reserved area, without applying error correction.

I don't know if the 3DS NAND carts (aka CARD2 carts) do have similar dump-able reserved areas. If yes, that might be an easy way to get keys/seeds for RC4 and SNOW encryption. The 3DS NAND seems to have a different maker ID though (unlike DSi chips, they are apparently not made by Samsung) (and the 3DS NAND transfer protocol seems to be still unknown anyways).
Actually all "ROM" chips are NAND flash these days because it's cheaper to make universal one-time flashable NAND flash chips than a mask ROM. The later becomes even more expensive when there are multiple revisions of the game with updates/fixes.
There is also cmd 0xC5 for 3DS games which is sent on init and every 10000 reads. That might be some kind of refresh cmd to make sure all flash pages are good. Would be annoying if they die after a few years of usage. Dunno if DS games have such a cmd too.

CARD2 save area writing has been figured out mostly. See https://github.com/PSI-Rockin/Corgi3DS/ ... e.cpp#L243 and the following cmds. Be warned. The Action Replay Power Saves which can write CARD2 saves has been known to brick these games in rare cases (game doesn't boot anymore or crashes). The reason for the bricks is unknown but probably something going wrong at save writing. So unless you want to sacrifice a game don't go too wild. "The Legend of Zelda - Tri Force Heroes" or "Animal Crossing - New Leaf" might be the cheapest games to get. Otherwise i only know from Pokémon X/Y and Omega Ruby/Alpha Sapphire to be CARD2 but expect higher prices even for used copies because apparently everything with Pokémon printed on it is worth more including toilet paper (sarcasm).

That reminds me you mentioned how the graphics don't look much better on 3DS games than on DS. That's because you tried the wrong games :wink:
nocash
Posts: 1405
Joined: Fri Feb 24, 2012 12:09 pm
Contact:

Re: 3DS reverse engineering

Post by nocash »

profi200 wrote: Wed Jun 23, 2021 7:34 am That reminds me you mentioned how the graphics don't look much better on 3DS games than on DS. That's because you tried the wrong games
Probably. I've bought only five 3ds cartridges yet (including two supposed "quality" titles). I've tried hard to figure out which titles might be interesting to have, but the 3DS software library doesn't seem to be too large (especially when focusing on adult or dark themed action / adventure titles, I guess Kirby games could be quite cool either, I would rather try to avoid P-Letter and Mario games though).
Which games would you suggest for impressive graphics?

I've more or less got loading NDS/DSi titles on 3DS working in wifiboot (good enough to boot the old Eragon demo and some of my own homebrew code). I still need to initialize lots of values in RAM at 2FFxxxxh and in DSi AES keyslots (basically porting the DSi Launcher or Unlaunch to 3DS, and executing that code either before or after switching from 3DS mode to DSi mode).
And, I've some problems with the NDS/DSi hardware emulation:

NDS RTC time/date/status/alarm can be set via Port 10147100h-10147166h. That's working... but the time doesn't increment. Is there a CLK enable bit somewhere? Or does one need to increment it manually once per second via timer interrupts on ARM11 side??

LGYFB does occassionally show garbage in the upper scanlines, then followed by the actual picture. It's somehow related to when the LGYFB hardware gets enabled: I am doing the LGYFB enable on ARM11, and the 3DS-to-DSi mode switch on ARM9, and it seems that one must somehow synchronize that two steps with each other. Otherwise the upper 12 scanlines (or, sometimes, more than hundred scanlines) contain junk that should be usually displayed at the bottom of screen.
It looks as if a scanline counter or old FIFO content isn't automatically cleared at vblank. I don't know if there's a way to do that manually... otherwise I'll just have to sync the initial Mode switch and LGYFB enable.

Touchscreen doesn't work so far. Theoretically one needs to switch the TSC from 3DS mode to DSi mode (which I don't really know what needs to be done there). And thereafter, for NDS games, one would need to switch the TSC from DSi mode to NDS mode (I know how to do that, at least on real DSi).
homepage - patreon - you can think of a bit as a bottle that is either half full or half empty
asie
Posts: 14
Joined: Sun Sep 22, 2019 10:41 pm

Re: 3DS reverse engineering

Post by asie »

nocash wrote: Sun Jul 04, 2021 3:49 pm Touchscreen doesn't work so far. Theoretically one needs to switch the TSC from 3DS mode to DSi mode (which I don't really know what needs to be done there).
I've asked Sono about it, and apparently that's controlled by TSC[67h:25h] bit 6. 0 = DSi mode, 1 = 3DS mode.

According to Sono, this only remaps the touch screen; the Circle Pad runs off a "completely independent and time-sharing ADC".
Post Reply