Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel?

[Laurence Perkins]

On March 30, 2021 10:11:56 AM PDT, Dr Rainer Woitok <rainer.woitok@×××××.com> wrote:
>On Saturday, 2020-12-05 19:07:51 +0100, I myself wrote:
>
>("> >" refers to Michael <confabulate@××××××××.com>)
>
>> Michael,
>>
>> On Friday, 2020-11-27 19:07:17 +0000, you wrote:
>>
>> > ...
>> > A 4k block size is recommended for ntfs-3g which is the default
>sector created
>> > by fdisk and friends on Linux these days.  This will align your
>partition
>> > optimally.  In addition, mkfs.ntfs will use 4096 bytes as the
>default cluster
>> > size, so you should be good in that respect.
>> >
>> > Another setting you may want to try is mounting the USB with
>'big_writes' -
>> > check the man page.  This should help particularly with large
>files, which
>> > will use larger blocks up to 128KB when copying data to the NTFS.
>>
>> Both, the VeraCrypt command line (--fs-options=big_writes) and the
>Vera-
>> Crypt GUI  (under "Settings  --> Preferences")  allow setting this
>mount
>> option.  But
>>
>>    $ mount | grep veracrypt
>>
>> never shows it,  initially causing me  to erroneously believe  it
>wasn't
>> set and to try finding  on the web another way  of setting it.   By
>pure
>> chance I finally found out that
>>
>>    $ ps -ef | grep veracrypt
>>
>> lists a  "/usr/sbin/mount.ntfs" task  which shows the  options really
>in
>> effect.  However,  I haven't yet had the time to test the effect of
>this
>> option when writing  plenty of really big files.   I will report on
>that
>> later.
>
>Well,  it's been quite a while,  due to my being almost permanently
>con-
>fronted with more pressing tasks ... :-(
>
>To sum up my experience with my new 128 GB Philips USB 3.0 sticks:
>while
>the Philips sticks  are significantly faster for reading operations
>than
>my old 64 GB Verbatim ones (probably USB 2.0), writing operations to
>the
>Philips sticks  are unbearably slow,  regardless of whether  I created
>a
>normal unencrypted NTFS filesystem on them or an encrypted NTFS
>filesys-
>tem using VeraCrypt.   Writing to  the USB stick  while at the same
>time
>reading from it in a different terminal window caused commands like
>"cd"
>or "ls" to simply stall.  Thus while running
>
>   $ cp --preserve=timestamps -ru $source_dir .
>
>in one terminal window, I ran
>
>   $ while true
>   > do n=$(ps -ef|g 'cp --preserve'|g -v grep)
>   >    if [[ "$n" = "${o-}" ]]
>   >    then sleep 10
>   >    else o="$n"
>   >         echo "$n"
>   >    fi
>   > done
>
>in another, to get the  wall clock times  when copying a new file
>began.
>That way I found that copying a 30 MB file took about 40 minutes.
>
>So what are my options?
>
>   - Stay away from Philips USB 3.0 sticks?
>
>   - Stay away from Philips USB sticks in general?
>
>   - Stay away from USB 3.0 sticks in general?
>
>  - Stay away from Filesystem in User Space  using a non-stable 5.10 or
>     5.11 kernel (currently I'm using stable 5.4.97)?
>
>   - Stay away from Gentoo?
>
>  - Stay away from Linux in general  and go back to OTOS  (aka the Only
>     True Operating System aka Windoze)?
>
>   - ...?
>
>Any ideas and comments welcome ...
>
>Sincerely,
>  Rainer

There are a number of things which might be going on here.
To start with, you can get the kernel, user, and wall clock run times for commands by prefixing it with "time".  So:

time cp <large file> <USB mountpoint>

Will get you more precise answers with much less effort.

As for the performance of the USB drive in question, there are a few things that might be tripping it up.

Firstly, writing flash memory is significantly slower than reading it.  Some drives deal with this by having some kind of internal cache mechanism.  Many deal with it by using a pile of smaller chips instead of one big one and striping the writes.  If the Phillips drive just used a few large chips instead, then it's just slow to write to and there isn't much you can do about it.  I've seen a lot of cheaper drives that are that way.

Double check that the alignment and block size are correct for the drive's internal structure.  That can cause some pretty massive performance hits if it's incorrect.

You can also check the output of the dmesg command for any errors the system is encountering with regard to the drive.

I don't know of any reason to stay away from usb 3.0 on Linux, but if you have USB 3 devices on both ends and try to hook them together with a USB 2.0 or 1.1 rated cable that could easily cause some problems...  I assume you're plugging the drive straight into the machine's socket.  If you're using the front panel though try one of the ones on the back.  There may be something up with the case wiring.

I've never had a Phillips USB stick, so maybe do some tests with another brand of stick and see if it has the same problem.  Kingston or SanDisk or something.  One of the ones where memory is their primary focus.  

You could definitely check performance on a different OS.  There may be driver-related performance issues on this model of drive or even this specific drive.

Instead of NTFS you could also try UDF.  It's supported on both Linux and Windows Vista and newer.  (And even XP I think, but only an older revision of the filesystem.)  Being an open standard it is quite a bit more stable on Linux than NTFS, doesn't need a userspace driver, and still supports larger drives and files.  You can find Linux and Windows formatting instructions on the web pretty easily.  Big thing is that on Windows it's available via the format command, but not in the GUI.

That's all I can think of at the moment.  Hope it helps.

LMP