It is currently Thu Oct 17, 2019 4:17 pm

All times are UTC - 7 hours





Post new topic Reply to topic  [ 11 posts ] 
Author Message
 Post subject: Detecting MBR vs VBR
PostPosted: Sat Sep 14, 2019 6:46 am 
Offline

Joined: Fri Feb 24, 2012 12:09 pm
Posts: 1025
The first sector of a disk can contain a MBR (partition table) or a VBR (FAT header).
The MBR in first sector is common for harddisks and SD cards.
The VBR iin first sector s common for floppy disks, but it does reportedly also exist on some SD cards.

Does somebody know a good/tested/reliable/working method for detecting wheter the disk starts with MBR or VBR?

They are unfortunately both using the same ID bytes (55h,AAh in last two bytes), so that isn't helpful. I could only think of using heuristic guessing, like checking that the MBR has partition start smaller than partition end, and checking that (common) VBRs have 2^n bytes per sector, usually 2 fat copies, usually type F8h, commonly ascii characters in volume label, and so on. The probability guessing should work even with some small errors, but might fail when relying or some incorrect "must be always this value" assumptions.

Anyways, there must be several operating systems and other programs already having solved that issue... if there is something that is not too complicated and more or less well tested then I could just use the same method.

_________________
homepage - patreon


Top
 Profile  
 
 Post subject: Re: Detecting MBR vs VBR
PostPosted: Sat Sep 14, 2019 8:19 am 
Offline
User avatar

Joined: Fri Nov 19, 2004 7:35 pm
Posts: 4222
I remember Libfat solved this once. I think it tried to do checks for a valid FAT filesystem, and if it failed, assume there is an MBR.

Edit: Rechecked the code from libfat, and it looks like a leftover FAT boot sector as sector 0 (with partition table added) could throw it off. It's just using signature checks.

Edit 2: I guess you could sanity check the location and size of the cluster chains, the location of the root directory, and possibly some of the root directory entries. Maybe check for invalid numbers in the first sector of the cluster chains as well.

_________________
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!


Top
 Profile  
 
 Post subject: Re: Detecting MBR vs VBR
PostPosted: Sun Sep 15, 2019 6:30 am 
Offline

Joined: Fri Feb 24, 2012 12:09 pm
Posts: 1025
Dwedit wrote:
Good idea to look there. Hmmmm, that is confusing. It seems to have several detection functions, each calling isValidMBR (or doing equivalent checks without using isValidMBR), and the isValidMBR function is actually checking if it is a valid VBR (not a MBR).

Apart from that confusion, it's mainly checking if there is a "FAT" string (aka "F","A","T" byte array). That's probably working in more than 99% cases. On the other hand, as far as I remember, that string isn't required to be present and detection shouldn't rely on its presence. It might be better to use a scoring scheme, like +10 for "FAT" string, and +1 for things like FatCopies=2.

Does anyone have links for other MBR/VBR detection functions? Like open source code from linux, or embedded devices, or the like.

Dwedit wrote:
it looks like a leftover FAT boot sector as sector 0 (with partition table added) could throw it off.
Leftover FATs??? I hadn't thought about that case. Is that a common real-life problem? I guess that could happen if a very crappy partition editor would try to edit a non-existent partition table... but I would consider that as trashed beyond repair (the Leftover FAT might still point to a Leftover Root directory, so different OSes could do perfectly "legit" writes to different/overlapping disk sections).

Dwedit wrote:
I guess you could sanity check the location and size of the cluster chains, the location of the root directory, and possibly some of the root directory entries. Maybe check for invalid numbers in the first sector of the cluster chains as well.
The location of cluster chains and root does vary for FAT16 vs FAT32, in FAT16 they appear just after the VBR without specific location entries (apart from the Sectors per VBR entry, which could be 1 or bigger). One could only guess if the values are looking common, and perhaps insist on some values not to exceed the disk/partition size.

Looking at further other sectors... it would be certainly easier to check only one sector... but yeah, if the probability to have a VBR is somewhere in a grayzone at 20..80% then it might help to do further checks on other sectors (theoretically MBR bootcode could contain bytes that resemble common VBR values, but I doubt that that is really happening in practice).

SD Cards
At the moment I am working on SD cards. There, the SD card's CSD register might help for sector number vs disk size checks. Unfortunately that isn't so easy:
- SD/MMC cards have a 12bit C_SIZE entry in CSD
- SDHC/SDXC high capactity cards have that replaced by an expanded 22bit C_SIZE entry in CSD
- rare MMC high capactity cards have 12bit C_SIZE=FFFh, and actual size in EXT_CSD register
- the above 12bit C_SIZE additionally relies on READ_BL_LEN and C_SIZE_MULT
- future SD/MMC cards might do that yet differently
Hmmm, getting that wrong might just introduce more problems : /

SD/MMC cards do also have FILE_FORMAT and FILE_FORMAT_GRP entries in the CSD register, that bits should indicate if the card has a MBR or not. The bits are R/W, so a formatting/partition tools could theoretically update that bits when creating/removing partition tables.
But I rather doubt that all tools are really properly updating that bits (and that FILE_FORMAT bits were removed in SDHC/SDXC, probably because they didn't work very well).

_________________
homepage - patreon


Top
 Profile  
 
 Post subject: Re: Detecting MBR vs VBR
PostPosted: Sun Sep 15, 2019 10:05 am 
Online

Joined: Sun Apr 13, 2008 11:12 am
Posts: 8613
Location: Seattle
nocash wrote:
It seems to have several detection functions, each calling isValidMBR (or doing equivalent checks without using isValidMBR), and the isValidMBR function is actually checking if it is a valid VBR (not a MBR).
Here's dosfstools's VBR validation: https://github.com/dosfstools/dosfstool ... oot.c#L450
There's some ancient cruft in there you assuredly don't have to support (e.g. ATARI DOS, or FAT12)

The one time I wrote my own "superfloppy" vs MBR detection, I checked for a valid MBR first (signature, valid partition type, plausible size) and then proceeded to a much more rigorous set of tests about whether the VBR each pointed at was valid (finally reinterpreting the first sector as a VBR).


Top
 Profile  
 
 Post subject: Re: Detecting MBR vs VBR
PostPosted: Sun Sep 15, 2019 11:45 am 
Offline
User avatar

Joined: Fri Nov 19, 2004 7:35 pm
Posts: 4222
Fat12 is legitimately found on very small memory cards, such as 16MB CompactFlash cards, and its maximum size is 32MB, so it could even be on 32MB cards.

_________________
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!


Top
 Profile  
 
 Post subject: Re: Detecting MBR vs VBR
PostPosted: Sun Sep 15, 2019 12:15 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 21635
Location: NE Indiana, USA (NTSC)
However, anything bigger than 4.1 MiB (and smaller than 2 GiB) can be reformatted to FAT16.

_________________
Pin Eight | Twitter | GitHub | Patreon


Top
 Profile  
 
 Post subject: Re: Detecting MBR vs VBR
PostPosted: Sun Sep 15, 2019 1:04 pm 
Offline

Joined: Fri Feb 24, 2012 12:09 pm
Posts: 1025
lidnariq wrote:
nocash wrote:
It seems to have several detection functions, each calling isValidMBR (or doing equivalent checks without using isValidMBR), and the isValidMBR function is actually checking if it is a valid VBR (not a MBR).
Here's dosfstools's VBR validation: https://github.com/dosfstools/dosfstool ... oot.c#L450
Thanks! I'll check there later.

Dwedit wrote:
Fat12 is legitimately found on very small memory cards, such as 16MB CompactFlash cards, and its maximum size is 32MB, so it could even be on 32MB cards.
Yes, the DSi also has a FAT12 partition with 32.7Mbyte for the DSi camera photos (admittedly I didn't support booting titles from that photo partition in unlaunch).
Why only 32MByte? I thought the limit would be (a bit less than) 4096*cluster_size. As far as I know FAT12/FAT16 don't even have an official limit for cluster_size (though there might be an inofficial "MSDOS v3.0 did crash when" limit?). FAT32 officially allows max 32Kbyte as cluster size (which is also what I am currently supporting as max cluster_size).

tepples wrote:
However, anything bigger than 4.1 MiB (and smaller than 2 GiB) can be reformatted to FAT16.
Yeah, could be reformatted, but one should support FAT12 nonetheless. Especially because SD cards should have the cluster size matched to physical sector size, so reformatting to FAT16 with more-but-smaller clusters could decrease the card's performance & lifetime.

_________________
homepage - patreon


Last edited by nocash on Sun Sep 15, 2019 1:14 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Detecting MBR vs VBR
PostPosted: Sun Sep 15, 2019 1:13 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 21635
Location: NE Indiana, USA (NTSC)
In theory, yes, one ought to implement all five of FAT12, FAT16, FAT32, UDF, and (now that Microsoft has opened the spec) exFAT. In practice:

  • It's trickier to make a read-write FAT12 implementation that is resilient to loss of power when rewriting a cluster number that straddles a sector boundary.
  • Just because the FAT12 or FAT16 clusters are the same size as the underlying flash sectors doesn't mean they were given the appropriate alignment when the volume was formatted.
  • It's not quite as much of a problem on DSi, but implementing more file systems on a microcontroller or 8-bit microprocessor reduces the ROM space available to implement other features.

_________________
Pin Eight | Twitter | GitHub | Patreon


Top
 Profile  
 
 Post subject: Re: Detecting MBR vs VBR
PostPosted: Sun Sep 15, 2019 1:52 pm 
Online

Joined: Sun Apr 13, 2008 11:12 am
Posts: 8613
Location: Seattle
nocash wrote:
FAT32 officially allows max 32Kbyte as cluster size (which is also what I am currently supporting as max cluster_size).
Every modern OS (at least since Win98SE - I just verified, see attachment) supports 128 sectors/cluster. I haven't heard of anyone using FAT32 on 4Kn disks (although I have found a bunch of people asking about it) but those should support 512KB clusters.

Wikipedia additionally says:
Windows 98, SE and ME also supported reading and writing [volumes formatted using 128 sectors/cluster], but its disk utilities did not work with it and some FCB services are not available for such volumes. This contributes to a confusing compatibility situation.

Note that FCBs don't work on FAT32 volumes at all, so it's not clear whether that's a constraint.


Attachments:
win98se-supports-128sectors-per-cluster.png
win98se-supports-128sectors-per-cluster.png [ 18.85 KiB | Viewed 4157 times ]
Top
 Profile  
 
 Post subject: Re: Detecting MBR vs VBR
PostPosted: Mon Sep 16, 2019 1:34 pm 
Offline

Joined: Fri Feb 24, 2012 12:09 pm
Posts: 1025
Just checked the "dosfstools" code. I didn't see anything related to MBRs in there. But the VBR code is interesting, it looks as if the "die" function does completely reject various uncommon values. Theoretically one could have up to 255 fat copies, but the dosfstools does simply reject anything with more than 2 fat copies. If that radical reject anything dubious approach is working okay, then there is little risk of getting false positives.

Yes, 12bit fat entries crossing sector boundaries are nasty, it's uncomfortable even when just reading data.

I didn't knew that microsoft supports 64K clusters. Or, well, I once heard of somebody using that formatting, so there must be an OS supporting it. I wonder if microsoft has meanwhile changed the specs to support it officially (or if they do have formatting tools that create such formats). As of December 2000, their FAT32 File System Specification did state that:

"Note however, that a value should never be used that results in a “bytes per cluster” value (BPB_BytsPerSec * BPB_SecPerClus) greater than 32K (32 * 1024). There is a misconception that values greater than this are OK. Values that cause a cluster size greater than 32K bytes do not work properly; do not try to define one. Some versions of some systems allow 64K bytes per cluster value. Many application setup programs will not work correctly on such a FAT volume."

That larger values are nasty when squeezing the cluster buffer into some small memory region, which could work only when adding support for reading only half clusters at a time (or even smaller snippets).

_________________
homepage - patreon


Top
 Profile  
 
 Post subject: Re: Detecting MBR vs VBR
PostPosted: Mon Sep 16, 2019 2:24 pm 
Online

Joined: Sun Apr 13, 2008 11:12 am
Posts: 8613
Location: Seattle
Remember that Microsoft is trying to maintain excruciating compliance with every program that ever ran on their systems; all that means is that some ancient version of some installer - maybe written by themselves, maybe written by someone else - used a uint16 to hold bytes per cluster. But "X random program doesn't run on x86 on windows when $value is out of range" doesn't mean that the on-disk format is ill-defined, or unused, or even all that rare.

The Windows XP and Windows 8.1 FORMAT.EXE explicitly supports specifying the number of sectors per cluster:
/A:size
Overrides the default allocation unit size. Default settings are strongly recommended for general use.
[...]
FAT supports 512, 1024, 2048, 4096, 8192, 16K, 32K, 64K, (128K, 256K for sector size > 512 bytes).
FAT32 supports 512, 1024, 2048, 4096, 8192, 16K, 32K, 64K, (128K, 256K for sector size > 512 bytes).

Microsoft seem have shaved down FAT32 from its on-disk maximum of 228 clusters down to 222 by their command-line format tool, or 220 in their GUI format tool.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 11 posts ] 

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group