From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 7AD88158042 for ; Sat, 16 Nov 2024 19:47:29 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id D5E0BE0869; Sat, 16 Nov 2024 19:47:22 +0000 (UTC) Received: from cornsilk.ash.relay.mailchannels.net (cornsilk.ash.relay.mailchannels.net [23.83.222.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 3F2BBE0833 for ; Sat, 16 Nov 2024 19:47:21 +0000 (UTC) X-Sender-Id: thundermail|x-authsender|confabulate@kintzios.com Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id AFC148445ED for ; Sat, 16 Nov 2024 19:47:20 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1731786440; a=rsa-sha256; cv=none; b=zUiX0cWsgfve9lXI8FJPmtCYW9IIaXwHXA8xrHd92Vzi8Yfx2JXFjTJfF9kNrrlFmw9GGE VnwsepK12FH2Ec8InqznlIOQj58oOth8CouYkfcwrQvSKa4WFXvWs/3IxVHYvuERWZ3l0F AXUqJBFTrx5RePweWGueSZCXoq7nT5zUMpXP4h8gX9bTepQwjxBn60X3pjgJfuoD0z5UHr YvYCQcTc/tLeUDwLiWIirQRHFzZ4EanNsLXDE8YZw26G/rv/G3db3iz9yJCHGlh0c46LJ8 23JUzMUiaSBJKTEjfTzwLL2MKGaPqW0qjkGWd5C1SrMYv1qgp/ulzx/BcRbq9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1731786440; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references; bh=YI0F2X1kt/CvFHmIrLC+5HdVMobnnXnlTJzuwws8o60=; b=NH4wf2oyCif9mXaKSBWvZ13xR34ZftbmkN0oh+ug8BpNohtUl2PiFNaer8y8J1vEzTiqmS iWvo2+vL3OikpJF2UzASnhxMtCjw8SHT4ox/CHbTc6EQId2VmMEQdRruxEeS5AFKVWJqZ9 twvK+yLpxEDJqWoyPmFhoh1MIBszEKcFDdPOMTWqBrBoDpySysAfU/EYkl3GvlqGcaLiE0 eoUyfhRMW4OgkZMSXRNVpjM+FKvBR6S9gA7WPUEh/V/l+wK3Wd3RLePhuqv/8EHbZwcuty mWzs3PU3mbSMFgFH/Jhexo/mw05cxggD62x8T0liq19Yz86qpa2Ct1uEn2mhnQ== ARC-Authentication-Results: i=1; rspamd-645676964-6k2wn; auth=pass smtp.auth=thundermail smtp.mailfrom=confabulate@kintzios.com X-Sender-Id: thundermail|x-authsender|confabulate@kintzios.com X-MC-Relay: Neutral X-MailChannels-SenderId: thundermail|x-authsender|confabulate@kintzios.com X-MailChannels-Auth-Id: thundermail X-Inform-Battle: 6bb798fc705946be_1731786440297_660358107 X-MC-Loop-Signature: 1731786440297:611079872 X-MC-Ingress-Time: 1731786440297 Received: from mailclean11.thundermail.uk (mailclean11.thundermail.uk [149.255.60.66]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.113.72.180 (trex/7.0.2); Sat, 16 Nov 2024 19:47:20 +0000 Received: from cloud238.thundercloud.uk (cloud238.thundercloud.uk [149.255.62.116]) by mailclean11.thundermail.uk (Postfix) with ESMTPS id 0C53D1E0004 for ; Sat, 16 Nov 2024 19:47:16 +0000 (GMT) Authentication-Results: cloud238.thundercloud.uk; spf=pass (sender IP is 217.169.3.230) smtp.mailfrom=confabulate@kintzios.com smtp.helo=rogueboard.localnet Received-SPF: pass (cloud238.thundercloud.uk: connection is authenticated) From: Michael To: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] Seagate hard drives with dual actuators. Date: Sat, 16 Nov 2024 19:47:02 +0000 Message-ID: <1836185.3VsfAaAtOV@rogueboard> In-Reply-To: References: <2633393.Lt9SDvczpP@rogueboard> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart3291666.AJdgDx1Vlc"; micalg="pgp-sha256"; protocol="application/pgp-signature" X-PPP-Message-ID: <173178643622.3407369.8893125343446684823@cloud238.thundercloud.uk> X-PPP-Vhost: kintzios.com X-Spamd-Result: default: False [-1.51 / 999.00]; SIGNED_PGP(-2.00)[]; MID_RHS_NOT_FQDN(0.50)[]; ONCE_RECEIVED(0.20)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[]; DMARC_POLICY_ALLOW(0.00)[kintzios.com,none]; FROM_HAS_DN(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; REPLYTO_DOM_NEQ_TO_DOM(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MISSING_XM_UA(0.00)[]; ASN(0.00)[asn:34931, ipnet:149.255.60.0/22, country:GB]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_DN_NONE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[gentoo-user@lists.gentoo.org]; R_DKIM_NA(0.00)[]; NEURAL_HAM(-0.00)[-0.999]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(0.00)[+mx]; HAS_REPLYTO(0.00)[confabulate@kintzios.com] X-Rspamd-Queue-Id: 0C53D1E0004 X-Rspamd-Action: no action X-Rspamd-Server: mailclean11 X-Archives-Salt: d9336452-3dd7-42a0-be25-28316d317e46 X-Archives-Hash: fe8a744dd85ed9dba25eab843a826dd7 --nextPart3291666.AJdgDx1Vlc Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8"; protected-headers="v1" From: Michael To: gentoo-user@lists.gentoo.org Reply-To: confabulate@kintzios.com Subject: Re: [gentoo-user] Seagate hard drives with dual actuators. Date: Sat, 16 Nov 2024 19:47:02 +0000 Message-ID: <1836185.3VsfAaAtOV@rogueboard> MIME-Version: 1.0 On Saturday 16 November 2024 14:36:02 GMT Rich Freeman wrote: > On Sat, Nov 16, 2024 at 6:02=E2=80=AFAM Michael wrote: > > I assume (simplistically) with DM-SMRs the > > discard-garbage collection is managed wholly by the onboard drive > > controller, while with HM-SMRs the OS will signal the drive to start > > trimming when the workload is low in order to distribute the timing > > overheads to the system's idle time. >=20 > I'll admit I haven't looked into the details as I have no need for SMR > and there aren't any good FOSS solutions for using it that I'm aware > of (just a few that might be slightly less terrible). However, this > doesn't seem correct for two reasons: >=20 > First, I'm not sure why HM-SMR would even need a discard function. > The discard command is used to tell a drive that a block is safe to > overwrite without preservation. A host-managed SMR drive doesn't need > to know what data is disposable and what data is not. It simply needs > to write data when the host instructs it to do so, destroying other > data in the process, and it is the host's job to not destroy anything > it cares about. If a write requires a prior read, then the host needs > to first do the read, then adjust the written data appropriately so > that nothing is lost. As I understand it from reading various articles, the constraint of having = to=20 write sequentially a whole band when a random block changes within a band=20 works the same on both HM-SMR and the more common DM-SMR. What differs wit= h=20 HM-SMR instructions is the host is meant to take over the management of ran= dom=20 writes and submit these as sequential whole band streams to the drive to be= =20 committed without a read-modify-write penalty. I suppose for the host to h= ave=20 to read the whole band first from the drive, modify it and then submit it t= o=20 the drive to write it as a whole band will be faster than letting the drive= =20 manage this operation internally and getting its internal cache full. This= =20 will not absolve the drive firmware from having to manage its own trim=20 operations and the impact metadata changes could have on the drive, but som= e=20 timing optimisation is perhaps reasonable. I can't recall where I read thi= s=20 bit - perhaps some presentation on XFS or ext4 - not sure. > Second, there is no reason that any drive of any kind (SMR or SSD) > NEEDS to do discard/trim operations when the drive is idle, because > discard/trim is entirely a metadata operation that doesn't require IO > with the drive data itself. Now, some drives might CHOOSE to > implement it that way, but they don't have to. On an SSD, a discard > command does not mean that the drive needs to erase or move any data > at all. It just means that if there is a subsequent erase that would > impact that block, it isn't necessary to first read the data and > re-write it afterwards. A discard could be implemented entirely in > non-volatile metadata storage, such as with a bitmap. For a DM-SMR > using flash for this purpose would make a lot of sense - you wouldn't > need much of it. I don't know if SMRs use flash to record their STL status and data allocati= on=20 between their persistent cache and shingled storage space. I would think y= es,=20 or at least they ought to. Without metadata written to different media, fo= r=20 such a small random write to take place atomically a whole SMR band will be= =20 read, modified in memory, written to a new temporary location and finally=20 overwrite the original SMR band. > This is probably why you have endless arguing online about whether > discard/trim is helpful for SSDs. It completely depends on how the > drive implements the command. The drives I've owned can discard > blocks without any impact on IO, but I've heard some have a terrible > impact on IO. It is just like how you can complete the same sort > operation in seconds or hours depending on how dumb your sorting > algorithm is. I have an old OCZ which would increase IO latency to many seconds if not=20 minutes whenever trim was running, to the point where users started=20 complaining I had 'broken' their PC. As if I would do such a thing. LOL! = =20 Never mind trying to write anything, reading from the disk would take ages = and=20 the drive IO LED on the case stayed on for many minutes while TRIM was=20 running. I reformatted with btrfs, overprovisioned enough spare capacity a= nd=20 reduced the cron job for trim to once a month, which stopped them complaini= ng. =20 I don't know if the firmware was trying to write zeros to the drive=20 deterministically, instead of just de-allocating the trimmed blocks. > In any case, to really take advantage of SMR the OS needs to > understand exactly how to structure its writes so as to not take a > penalty, and that requires information about the implementation of the > storage that isn't visible in a DM-SMR. =20 Yes, I think all the OS can do is seek to minimise random writes and from w= hat=20 I read a SMR-friendlier fs will try to do this. > Sure, some designs will do > better on SMR even without this information, but I don't think they'll > ever be all that efficient. It is no different from putting f2fs on a > flash drive with a brain-dead discard implementation - even if the OS > does all its discards in nice consolidated contiguous operations it > doesn't mean that the drive will handle that in milliseconds instead > of just blocking all IO for an hour - sure, the drive COULD do the > operation quickly, but that doesn't mean that the firmware designers > didn't just ignore the simplest use case in favor of just optimizing > around the assumption that NTFS is the only filesystem in the world. =46or all I know consumer grade USB sticks with their cheap controller chip= s use=20 no wear levelling at all: https://support-en.sandisk.com/app/answers/detailweb/a_id/25185/~/learn-abo= ut-trim-support-for-usb-flash%2C-memory-cards%2C-and-ssd-on-windows-and Consequently, all flash friendly fs can do is perhaps compress and write in= =20 batched mode to minimise write ops. I can see where an SMR drive would be a suitable solution for storing media= =20 files, but I don't know if the shingled bands would cause leakage due to th= eir=20 proximity and eventually start losing data. I haven't seen any reliability= =20 reports on this technology. --nextPart3291666.AJdgDx1Vlc Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEXqhvaVh2ERicA8Ceseqq9sKVZxkFAmc49rYACgkQseqq9sKV ZxmtwxAA1MBt2lq0rk5pKr+//eiBp3BKpAwp+5uAQUpf9iT/z+pTGfky4CmXM1K+ BSZ3mGUfHCcCJffxbC/zEKWf2O5Mp/jEP1Tb32mNeBIzSUC8QHbXmaeZkCucuXnm PJjuequjx3XvlQk8ME4LMZyAaCWRaL8JGWxsYX7XuMIIMax6JR/1zfj1isuvZcGq bA8RNcNPm9Tp+To+me4nMCuqv4P359Dpm2A5TaLdSGzoWX2FM/4GJ2pKMXVY/0hJ Nz0ppRASXzeZaXeYtAjc2aO81ckFcOpuxDoCjinR7zFbVdWsdEthHwA+b+9dHrUo hRL/SpX4Fi32dpJKv2DBm61e1vIITCNHSwI1rgShS55GLgZV+OutVJDS/q26ZqTe HUvmDVyIq5Y4ZY8SJMwJ1YMuITRXBq2W8hzIcwDZUuAQXUP+OTJSadScQzELeHjm BX/9oDYdMJbFWWtn7Ymxluo0Q/bTXhgbAp84F4IIC/Ye+7G6vrmVnYG39hrqgZqC cW7TNWw3GMSHcbl5b0a+timEIbN78CyTIFDJRHsmwoUm3wQPxEl+g2A4oY6bPtEO 4BIFcJk5GqAhoAnUZmAm9FAMBtCukzkyYq1eweP8ff6E2L74Yg1zfi3UVTdNRDa4 2r6N0Z2HxFNh9nBBWgfVFHx1bJ5UAZY/jjNQrxMjgr+uazUdeFE= =Zb40 -----END PGP SIGNATURE----- --nextPart3291666.AJdgDx1Vlc--