From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 74E5B158083 for ; Mon, 16 Sep 2024 10:24:40 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 79F79E29B5; Mon, 16 Sep 2024 10:24:34 +0000 (UTC) Received: from mail-yb1-xb29.google.com (mail-yb1-xb29.google.com [IPv6:2607:f8b0:4864:20::b29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 39B12E29AD for ; Mon, 16 Sep 2024 10:24:34 +0000 (UTC) Received: by mail-yb1-xb29.google.com with SMTP id 3f1490d57ef6-e1a7e54b898so3335315276.2 for ; Mon, 16 Sep 2024 03:24:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726482273; x=1727087073; darn=lists.gentoo.org; h=content-transfer-encoding:in-reply-to:mime-version:user-agent:date :message-id:autocrypt:from:references:to:subject:from:to:cc:subject :date:message-id:reply-to; bh=cTiFO1L+S0dqIhZ5G2A8PR13o9PrReRdNvuqfyZ3/7A=; b=jsF354zCRVqyGad3Uqytuht9doxW2kiU1VIsY3O6zSMFMUU+9i+UlvNkP2+7gSCwqs 6oCy3PpxZmIglBpYmKgciHAdIqfJlLy4a+k1c2Lv9/zK/7G7ovLuh5erZN1MYy4vlawM o2XKOztrKxn8cibY+1/WN0WgdR64QAhwDv6e7pU7QuRhYtgtDwOTa1UgjfH1XA9dZByj Au0emljSUx9QqX/FhTsF6GsTOOo/zl9UIpXANJQZSzaXqYtw8uSjkxrKceMj3Zp+fWbN juREGZYL4Lbe5pInZZZ5N4j3/JfotTzIyE+W+mNGDSBHoGMopS6pcHZ0qidath12GJsN sBxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726482273; x=1727087073; h=content-transfer-encoding:in-reply-to:mime-version:user-agent:date :message-id:autocrypt:from:references:to:subject:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=cTiFO1L+S0dqIhZ5G2A8PR13o9PrReRdNvuqfyZ3/7A=; b=cTTSmsUjHbokiNHfpJWgjVsORfReqh/jw9nsFkR7B0raJx0s2zKbF9Lea7NJnMM1op misW+r2a2QC0YCjrzMIXeZiw1+jKSk3DYETfljzllcCqP102Ia7h5niIVFNkA/CGeCOE /WSfMNN4BP0rdjCWDNAakEkaeG2t57PmJjzXZI4SxB/y7wbrhJuC9fi5H2KZcRNfKLu+ xibSfWgx6eVssJzmxpAwS/UEtS2lkFFUejdfbSuuyKTACcbc7A+ojk0WCs5VHCW53AO/ f3Ke8K+2F4wX7YpU4ajZojB/tTizPZc7lDsnNU6u5C+VbYoutPI5bsWTXyIiiob44+r+ /FoQ== X-Gm-Message-State: AOJu0YxSDIuk9S7odVDCa05ZL8eIrjTlvkHbKCAT00EKIuysiDPkby9j 8XDiOhaaPwR5TbCV7VydIndySQqSYuZdsbuGoVdeimg/Go6uA6PiNtdaOA== X-Google-Smtp-Source: AGHT+IEBWoYntcJPh7AQPv8xlVeV5t+btv4om4k/TtcR1Tq/B5K/1EgiJvSyDDkLoSvXu0laT8dtPw== X-Received: by 2002:a05:690c:6d13:b0:6dd:b885:4271 with SMTP id 00721157ae682-6ddb8854642mr34550157b3.38.1726482273269; Mon, 16 Sep 2024 03:24:33 -0700 (PDT) Received: from [10.8.8.13] ([138.199.2.136]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6dbe2ed009fsm8526757b3.97.2024.09.16.03.24.32 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 16 Sep 2024 03:24:32 -0700 (PDT) Subject: Re: [gentoo-user] Re: Package compile failures with "internal compiler error: Segmentation fault". To: gentoo-user@lists.gentoo.org References: <8c26be16-d033-ea3f-06e1-a9ce84cbbafb@gmail.com> <7866172.lvqk35OSZv@rogueboard> <15276605.tv2OnDr8pf@rogueboard> <5ad461dd-6122-ae67-5609-0a39d5a64fe1@gmail.com> From: Dale Autocrypt: addr=rdalek1967@gmail.com; prefer-encrypt=mutual; keydata= mQINBGFSciYBEADcEGMyJBSuavKO/XKUVvgkxck7Nl8Iuu8N2lcnRji/rSKg5c1Acix1ll9i oW8JBCHwvn0+Xy60BvEsqcup3YSHw5STl/bR1ePEehtnYrg8FdjdS91+B805RfnKMm69rFVI wLSBHQrSG1yxHd8CloWoEdhmVtP24buajbh114bgXd9ahtpZrCVMrWdWYUg2mEXguGV5uNAh Rf8SWxDNc79w24JxsV34a8niMUYMjzWr0rafIbzk732X38vGjVMLo/2mMpkbp9mPp++LHoY+ 0Pet8zxxdXPJSCd475kza1AD+hhSyBZXB9yknYWgyY3cZe1rGmooJSi2KX4QxO7npwLThcO1 be6KKRkd35+Fi/a1BzVOHsZMiK/gcwxEFoMd27gir4ehaeHJfFXl+65w4hj0EsOZSxrJrm2C R50g5By2czSKP1bADEygFNpIJj51AR+wM88NImG2RPtlT2maYBzazvF05g65cdHXGp1C7W5P wwwKU2DgABB2t7N7z5A69LnryBRw4zUYDRRYLTYlBlYgg+xILm2c0OrBdxJgLJa7JE50Eo25 d3PFwt9J0gYvqy6sPFLl9So0sDg9zm0hKQtXOP5kgropUFGrNoJI+mjwF4rYLRBVzZwNAvlO OhEvHubBo3mEllv4x+FeptwXZxlk7gUsdqI8AxnFB8K9wi6FVQARAQABtBtEYWxlIDxyZGFs ZWsxOTY3QGdtYWlsLmNvbT6JAk4EEwEIADgCGyMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AW IQQSG1h01ruv/WNXc3Q3RqOgiQH1GwUCYVJy8gAKCRA3RqOgiQH1G+waEACeTZCt77jnRAmQ AV7otKuZekDWiLi3Eig8tj5ZJiCNSYA/hIxzmexRP0GMqjitcXK1iGwWcvMzzvIq30GAjIfB 4BR38cnXbtBa6fNewiT7QaZe/Hn6yBRldXNQypzbHy+/o27bUEy+oX4rE7etUgEHQAjuw7xz XFWg4tH1/KJvsOVY5upnWc5LdxYhsuQ3dQD4b22GsK0pOBDfb9PiirYM8eGKvrVuq4E/c75z lDDFhINl18lNZ9D0ZFL3IkTjHsAAqFH9uhnnEB8CWdHbBewPEfRaOhBUYWZ3Q8uTkmDgZT8q D9jlvLEdw7Nh2ApdxoepnI/4D+ql2Gr4DtH7SEPydr5gcf1Qr/2bXRb1hAYnIVcbncs/Bm3Z bkRKPVWMfE3Fusa+p5hMzixk0YysMaTHlc7mYRYAEZGnPMXnmcCbetwARU7A0yz1M1kCMOAQ Lsz8KH5kv3cRenMB6SFfjND2JfAK61H5TtnPq3L8noS2ZykRYxq9Nm3X64O1tJojIKBoZFr8 AwYNCvqC6puUyGMuzHPh7jPof8glfrrEKIYUvNPGMDoVX3IGetxh/9l6NcxgFA4JGoR+LS3C zmeNrwlllAe3OEUfKoWVQ+pagpSdM+8hHolaSda4Ys66Z3fCR4ZvcTqfhTAVskpqdXa4isAk 7vTcXu3L499ttywEp7rJTbkCDQRhUnImARAAncUdVhmtRr59zqpTUppKroQYlzR0jv8oa7DG K4gakTAT2N7evnI9wpssmzyVk8VEiLzhnFQ/Ol3FRt6hZCXDJt0clyHOyTfvz/MNFttWuZTc mLpSvmRR6VRjAH+Tz3Eam2xUw3PGuH97BcXQ3NnX3msv1UDxtxxBu6e2YrdeOhrCUSgzokcJ 98ChUNy934cgepPybAI12lSWqVFQ1aG7jExZfiUk+333fPSDbpKoZbTW5YJLXbycmW/C1IWL qYQyNjRWKaGoJtUWFhhmNiOQct7n90aKivNVPavmN+UQ9LlMaINtf9T6XCzLfogCFsulDCDJ 0yNQLDTurHaB4E71xoctgXmLLq9z1RQ0W2XiVAAOZQj6K3+d0AOUjDhCQ2QW8dUSq0ckkZXV DKVJOGS8Nhf2eIWIqRnP3AcUiiaiFGqUaVUmUAZ6h/oJmgghEu/1S+pcuUKU5i69+XCZ3hH2 Jzwzbf7K+FAIkOhCfHncF8i1N1pk00pOVykNnqHTfFo3qFusHt0ZWgXVnnn4pYdXqZNoDhvF BRE5Vm4k/k96Pw8HRx6Os6eFSRrlqGzRgqsu86FekxusXB9UGv4lJhtU/J+8MRWsh22K718s DbQnABicGKFz1qQlWvcf59oTByhLINJCBt1WXl+TzJDXepr3QSkqmK41dO9Hob97C9dMiK8A EQEAAYkCNgQYAQgAIAIbDBYhBBIbWHTWu6/9Y1dzdDdGo6CJAfUbBQJhUnLyAAoJEDdGo6CJ AfUbVHIQAKSWw620vPhR3A/njU2z77F3z/Jk+HTKdE3fIyWSWdkYN7CBFL0NguOMP30WZ+qE sJhZu7T5hf251MwQUUt27xlfnKYOmQs7CqONlXuXlGZI6WufrUjxNcVz+5gJsqvUWuuJWsgg sDmE92IBnfG/f81fPHWQyfr/SF4wYDMyoFp5xCCQpp1zB63iuFvvrhxBkEHzmbRtVDOhl0Xp BVEDR1w3QRACw9QJD/KM05Czv9JNQYlwinWO/OaQ9cMlUpKLgswUPg9IZ5vucxScfuAUA5uC B1jlAQ8ZPlVukBmbEv5RGOv+lpuEbA3YDMVtEeH4YMFbjt/+vH3Cr2vTbp5JlpByLburJEH0 WXZLUawEfUsZvVwpOuJK75vaa2HYXee+Cb3iCIzwfIfctdlqzUcbGRczlRNM59hpvj4z29Gh 3kAxVHItAYq54ikxQ9l4hQ8s9sLYPbX/WtcBxNX8crBSw0FLnmzGleVEtBHyqtt5CLzQNgrj GYWl1vKDUmRPw1CdZ1c+fMN9CY11jOM5B5ZnqZWfDeVYO2iJ5SuvTycChexCb8WYn1bdCBIo bBtga2RBXbVt4Mh9E4owsszefn51MwfjXxB20Fc5k3GU1AVpTCMs3ayYCzo0b2pvEvdjtDcA CYLEFPWgaFX9iQAM/CDfKvTtvgGWpqtCL2raq/mQoJEU Message-ID: <8a4fa2b6-8986-dc18-e3b9-18ad7c7c7ec8@gmail.com> Date: Mon, 16 Sep 2024 05:24:31 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0 SeaMonkey/2.53.19 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Archives-Salt: 2039583c-8154-4a04-bd6b-2c722eb2da95 X-Archives-Hash: c5103f8e6f47e945cf10df45899943b3 Frank Steinmetzger wrote: > Am Sat, Sep 14, 2024 at 02:46:35PM -0500 schrieb Dale: > >> I was running the command again and when I was checking on it, it >> stopped with this error.  >> >> >> >>   File "/root/dh", line 1209, in >>     main() >>   File "/root/dh", line 1184, in main >>     directory_hash(dir_path, '', dir_files, checksums) >>   File "/root/dh", line 1007, in directory_hash >>     os.path.basename(old_sums[filename][1]) >>                      ~~~~~~~~^^^^^^^^^^ >> KeyError: 'Some Video.mp4' > What was the exact command with which you ran it? > Apparently the directory has a file 'Some Video.mp4', which was not listed > in an existing checksum file. I'm fairly sure it was this command.  /root/dh -c -f -F 1Checksums.md5 -v I may have changed the -c to -u because I think it was the second run.  I'd start with the thought it was -u if it were me.  There's another command running right now and I cleared the scrollback part.  Once it finishes, I can up arrow and be more sure.  At the moment, I'm letting it test the files against the checksum it created, to be sure everything is good.  It's almost half way through and no problems so far.  I might add, I did a second run with -u, which I think produced the error above, and it seems to have missed some directories.  When looking I noticed some directories didn't have a checksum file in it.  That's when I ran it a second time.  It skipped the ones it already did but found the ones that was missed in first run.  There are almost 46,000 files in almost 800 directories.  Is there some tool your script relies on that could make one of those numbers to high?  > I also noticed a problem recently which happens if you give dh a directory > as argument which has no checksum file in it. Or something like it, I can’t > reproduce it from memory right now. I have been doing some refactoring > recently in order to get one-file-per-tree mode working. > >> I was doing a second run because I updated some files.  So, it was >> skipping some and creating new for some new ones.  This is the command I >> was running, which may not be the best way.  >> >> >> /root/dh -c -f -F 1Checksums.md5 -v > Yeah, using the -c option will clobber any old checksums and re-read all > files fresh. If you only changed a few files, using the -u option will > drastically increase speed because only the changed files will be read. > Use the -d option to clean up dangling entries from checksum files. That was my thinking.  When I update a set, I'll likely just cd to that directory and update, -u, that one directory instead of everything.  That will save time and all.  Doing everything takes days.  LOL > >> Also, what is the best way to handle this type of situation.  Let's say >> I have a set of videos.  Later on I get a better set of videos, higher >> resolution or something.  I copy those to a temporary directory then use >> your dmv script from a while back to replace the old files with the new >> files but with identical names.  Thing is, file is different, sometimes >> a lot different.  What is the best way to get it to update the checksums >> for the changed files?  Is the command above correct?  > dh has some smarts built-in. If you changed a file, then its modification > timestamp will get udpated. When dh runs in -u mode and it finds a file > whose timestamp is newer than its associated checksum file, that means the > file may have been altered since the creation of that checksum. So dh will > re-hash the file and replace the checksum in the checksum file. > Sounds good.  I wasn't sure if it would see the change or not.  >> I'm sometimes pretty good at finding software bugs.  But hey, it just >> makes your software better.  ;-)  > Me too, usually. If it’s not my software, anyways. ^^ > But I think you may be the first other of that tool other than me. > One thing I've noticed, when I run this tool, my video sputters at times.  It does fine when not running.  This tool makes that set of hard drives pretty busy.  It might be nice to add a ionice setting.  I'd just set it inside the script.  If a person wants, they can edit and change it.  Just set it to a little lower than normal stuff should be fine.  If I restart smplayer, I may set its ionice to a higher priority.  Just a thought and likely easy enough to do.  I don't want to stop your script given it is so far along. It sometimes pops up a question.  I figured out that I type in the answer with the letter that is in parentheses.  Could you explain the options a bit just to be sure I understand it correctly?  So far, this is a nice tool.  It should find corruption, like my bad memory stick, bit rot, bad drive or anything else that could corrupt a file.  Even power failure I'd think.  It takes a while to do the checksums but the script itself is fast.  Once you really happy with this and feel like it is ready, you should really make a announcement that it is ready.  Anyone who does a lot of write once and read many but are concerned with files becoming corrupt over time for any reason should be interested in this tool.  It wouldn't work well for files that change a lot but there are tons of situations where once a file is generated, video just being one of them, then never changes after that.  Given the number of files I have, how I change things, I should be able to find any bugs or needed features.  Example, making my video sputter and needing a lower drive priority.  You may not run into that but I noticed it here.  Different use case.  By the way.  Still using that other script you threw together a good while back.  I used it the other day to update some better videos I found.  Thanks much for both tools.  I wish this old dog could learn new tricks.  ROFL  I'm having trouble remembering old tricks.  :/  Dale :-)  :-)