Re: [gentoo-user] [OT]: Optimal formatting a SDcard (64GB) with partions of diffent sizes and filesystems?

Fernando Rodriguez <frodriguez.developer@×××××××.com> [15-08-08 05:43]:
> On Friday, August 07, 2015 9:44:50 PM Meino.Cramer@×××.de wrote:
> > Mick <michaelkintzios@×××××.com> [15-08-07 20:04]:
> > > On Friday 07 Aug 2015 04:27:15 Fernando Rodriguez wrote:
> > > > On Thursday, August 06, 2015 6:18:59 PM Meino.Cramer@×××.de wrote:
> > > > > Hi,
> > > > >
> > > > > for my tablet PC I used a used 32GB FAT32 formatted SDcard. The
> > > > > formatting was already done by the manufacturer.
> > > > > Then I screwed it up and had to do the partioning and formatting
> > > > > myself again. "No big deal", I thought -- and was wrong.
> > > > > Yes, the "thing" I got could be read and written. But it was
> > > > > DAMN slow in comparison to the original formatting.
> > > > >
> > > > > I googled and found a description, which described exactly,
> > > > > what I wanted: An optimal formatting for one big FAT32 partion.
> > > > > I did it again ;) and: TADA! The speed was back.
> > > > > LINK:http://zero1-st.blogspot.de/2012/05/formatting-fat32-volumes-larger-> > than.html
> > > > >
> > > > > Now I need the something identical but explained in a way
> > > > > that it can be successfully applied to any partion layout
> > > > > and any SDcard size.
> > > > > Currently the new SDcard has 64GB (yes, the tablet eats that size
> > > > > well :) and needs at least two partions: One FAT32 and one ext4.
> > > > > May be that I need a different layout later.
> > > > >
> > > > > To what aspect and "logic" do I have to keep my eyes on, when
> > > > > it comes partioning/formatting any SDcard size with any partion
> > > > > layout and any filesystem?
> > > > >
> > > > > Thank you very much in advance for any help!
> > > > > Best regards,
> > > > > Meino
> > > >
> > > > I wrote a long reply to this and it appears to have been swallowed by
> > > > /dev/null.
> > > >
> > > > SD cards don't have 128K blocks. Except for the very early ones
> (standard
> > > > capacity), they are divided in allocation units (AU) that are 1MB to 4MB
> > > > for SDHC and even larger for SDXC. The only way to get that value is by
> > > > reading a register in the card (so you can't do it in usermode on
> linux).
> > > >
> > > > The AUs are divided into Recording Units (RUs). The size of these can be
> > > > deduced from the card speed class (that's the number inside the C on the
> > > > label), and the card capacity. For class 2 and 4 if the card is less
> than
> > > > 1GB it's 16KB, otherwise it's 32KB. For class 6 it is 64KB, and for
> class
> > > > 10 it's 512KB.
> > > >
> > > > After an AU is erased you can write to any of the free RUs in any order
> in
> > > > blocks of 512 bytes sequentially (the block size is configurable by the
> > > > driver but 512 is the most common). But if you write to a nonfree RU
> then
> > > > all non- free RU get copied to a new AU. So the performance hit depends
> on
> > > > how many non-free RUs are in the AU when this happens.
> > > >
> > > > So to get the best performance you need to align the first FAT cluster on
> > > > an AU boundary and that the RUs used by the reserved sectors after the
> FAT
> > > > are free. This is not so easy from usermode because you can't get the AU
> > > > size and you can't erase the AU to make sure reserved sectors are free.
> > > > The Windows 7 and later format utility will do it if you don't partition
> > > > the card. The next best thing is to align it to an RU which should be
> > > > pretty easy.
> > > >
> > > > You could guess the AU size by writting blocks of RU size from the start
> of
> > > > the card and timing it. Every time you hit the AU boundary there will be
> a
> > > > longer delay.
> > > >
> > > > For more details see the SD specification (chapter 4.13).
> > > >
> > > > https://www.sdcard.org/downloads/pls/
> > > >
> > > > They also have formatter tools for Windows and OSX. I tried the Windows
> > > > version years ago but had problems with it (can't remember what).
> > >
> > > Excellent information Fernando, thank you!
> > >
> > > So there is no tool for me to use to read the AU/RU on the chip?
> > >
> >
> >
> > Hi,
> >
> > sorry for being a little late...was too busy and my sdcard is still
> > not formatted... ;)
> >
> > Thank you very much for the help and all the informations. Currently I
> > start to understand the problems and solutions in formatting ONE
> > partition with a FAT32 filesystem on a sdcard the correct way, but
> > when it comes to more the one partition and filesystems for example
> > like ext4fs I still dont know how to...
> >
> > Just a few minutes before I found this:
> > http://www.bradfordembedded.com/2014/05/flashbenching/
> > http://wiki.laptop.org/go/How_to_Damage_a_FLASH_Storage_Device
> > https://github.com/bradfa/flashbench
> > http://wiki.laptop.org/go/SDCard_Testing
> > https://lists.linaro.org/pipermail/flashbench-results/
> > https://blogofterje.wordpress.com/2012/01/14/optimizing-fs-on-sd-card/
> >
> > I am still in the process of reading and hopefully understanding
> > this...
> >
> > Best regards,
> > Meino
>
> For partitioning just account for the offset of the 1st partition when
> calculating the number of reserved sectors and make the FAT partition the first
> partition. If the MBR pushes the FAT beyond the number of AUs that it would
> otherwise occupy I would reconsider using an unpartitioned filesystem.
>
> I'm not very familiar with the structure of ext4 so I'm not sure what you can
> do to optimize it but SD cards are designed with FAT32 in mind. A journaling
> filesystem (or a FAT partition outside the first partition) for example will put
> a lot of stress on areas that are not meant to support it.
>
> --
> Fernando Rodriguez
>

Hi Fernando,

thanks a lot again for your thoughts! :)

A little background to what I am trying to acchieve: I have an
Android tablet, which supports FAT32 for the external flash (the
sdcard) in its initial state.
It has an internel flash, which is ext4 formatted (as far as I know).
That means: It is possible to use ext4 on the sdcard also, because
the Android kernel knows this fs.
As Android is a SELinux, it only installs applications (with a few
exceptions) on internal flash.
BUT:  :)
There are applications for Android (I hate to say "Apps" ... it sounds
like one would say "eds" for "editors" and "compils" for "compilers"),
which make it possible to move applications from internal to external
flash -- the sdcard, if one has a formatted the sdcard with a second
partition with an ext4fs on it.

Additionally I found this:
https://wiki.gentoo.org/wiki/SDCard

And: One could switch off journaling on ext4fs to minimize writes
to the flash. Further options are available too.

Best regards,
Meino