Tagged with allocation - Personal View Talks https://personal-view.com/talks/discussions/tagged/allocation/feed.rss Fri, 22 Nov 24 09:05:30 +0000 Tagged with allocation - Personal View Talks en-CA What's so special about the GH2's preferred 128kB allocation unit size? https://personal-view.com/talks/discussion/3501/whats-so-special-about-the-gh2s-preferred-128kb-allocation-unit-size Thu, 07 Jun 2012 19:08:27 +0000 karl 3501@/talks/discussions I wanted to share some findings with you that I found while experimenting with different methods to format a SDXC card before use in a GH2.

While I cannot see yet an immediate beneficial use of my results, I thought that they may inspire other experiments or just add to the knowledge of the GH2 internals.

At first I wanted to know what precisely a GH2 writes to my 64GB SDXC card when doing the usual "Format" before use. The answer:

The GH2 creates a DOS-style partition table containing one primary partition that starts at an offset of exactly 32768 sectors (of 512 byte, each), or 16MB. This seems like a reasonable choice to me, as it makes sure the start of the filesystem is aligned to a 2^N byte offset that is almost certain to be a multiple of the erase block size of the MLC NAND flash memory that SD cards commonly consist of. The type ID of the primary partition is set to 7, a value used by Windows for NTFS as well as exFAT (and also for HPFS with OS/2, which may rest in peace). (Notice that with SD and SDHC cards, partition type 0xb is used, which is common for use with FAT file systems.) The unused 16MB before the first partition start may also help (when never having been written to) the SD card internal Flash controller and its wear level management to prepare completely empty erase blocks for writing. (NAND Flash can be read and written to when empty in small blocks such as 512 byte, but erasing NAND Flash memory is possible only in much larger "erase blocks" - such as 512k to 4MB - so the controllers in SD cards, USB sticks or SSDs need to employ some sector allocation management of their own to make sure they can erase blocks of that size before new data can be written.)

The GH2 writes an exFAT filesystem to the primary partition with the following characteristics:

  • Volume serial number: A seemingly random number that changes with each format
  • exFat filesystem version: 1.0
  • Sector size: 512
  • Cluster size: 131072
  • First sector: 32768 (consistent with the start of the partition)
  • FAT first sector: 16384
  • FAT sectors count: 16384
  • First cluster sector: 32768
  • Root directory cluster: 4
  • Volume state: 0x0000
  • FATs count: 1
  • Drive number: 0x80
  • Label: (none, empty)

This is the same filesystem layout that results if you format the partition using e.g. the following Windows command line (replacing X: with your actual drive letter):

format X: /FS:exFAT /Q /A:128k

The GH2 writes a directory structure and some files to the initialized exFAT filesystem:

size name
     0 DCIM/
     0 MISC/
     0 PRIVATE/
     0 PRIVATE/AVCHD/
     0 PRIVATE/AVCHD/BDMV/
     0 PRIVATE/AVCHD/BDMV/PLAYLIST/
     0 PRIVATE/AVCHD/BDMV/CLIPINF/
     0 PRIVATE/AVCHD/BDMV/STREAM/
 504 PRIVATE/AVCHD/BDMV/INDEX.BDM
   66 PRIVATE/AVCHD/BDMV/MOVIEOBJ.BDM
     0 PRIVATE/AVCHD/AVCHDTN/
   72 PRIVATE/AVCHD/AVCHDTN/THUMB.TID
     0 PRIVATE/AVCHD/AVCHDTN/THUMB.TDT
     0 PRIVATE/AVCHD/IISVPL/
607776 PRIVATE/FILEINFO.TBL
   76 PRIVATE/FILEMNG.DAT
1672 PRIVATE/cache.dat

If you archive this directory structure on a computer of yours, you can re-create a "blank" SDXC card for use with the GH2 at any time, just by creating the same filesystem type and copying these files to it. I tested that indeed, the GH2 behaves completely normal if you use a card prepared such, indistinguishable from an "original in-camera formatting".

Now I wanted to know whether changing any aspects of this "formatting" would yield interesting results, and indeed, it does.

First, I checked whether the offset of the first partition does make a difference to the GH2 when changed. Doubling or halving this offset didn't change anything, the GH2 seemed not to take notice of the difference.

Next, I checked what would happen if I used a different allocation unit size, e.g. 256kB or 64kB. Such change causes the GH2 to display a warning when inserting the SDXC card, saying (as far as I can remember the wording): "This card has not been formatted with this camera. It will not be suitable for recording video." After displaying this message for a few seconds, the camera behaves normally regarding the taking of still pictures. When starting to shoot video, it first seems that all is normal, too, but whatever allocation unit size different from 128kB I tried, video recording will stop after a few seconds with the well known error message that "the cards write speed is insufficient". This happenes even when recording low-bandwidth video, so I wonder what special meaning the 128kB allocation unit size has for the GH2 that it is working so badly with other sizes.

Notice that when I benchmark writing to the same card from my PC with allocation unit sizes other than 128kB (within reasonable limits), I do not measure significantly different performance. But of course, the SD card interface of the GH2 may be very different from the Kingston USB 3.0 card reader that I use with my PC.

Well, that's about it with my experiments on this, so far.

BTW: The "SD Formatter" offered at https://www.sdcard.org/downloads/formatter_3/ claims that it has one mode of operation where it pre-erases the Flash memory on the card, which would plausibly allow for faster writing, but only until all the blocks have been written to at least once. I also wonder under which circumstances this tool is able to issue the ERASE_WR_BLK_START/_END etc. SD protocol messages to a SDXC card that is connected via a card reader, the documentation of the formatter indicates that the tool falls back to just overwriting the blocks if it cannot erase them - but just overwriting blocks would not leave the card with empty erase blocks, so subsequent writes would require erase operations by the cards internal controller, resulting in no write performance benefit.

]]>