1 |
Hi! |
2 |
|
3 |
I have tried hard to make this message to the point. But that wasn't easy, as |
4 |
what I figured out about gzip --only after hours of work-- I still do need |
5 |
confirmation about, even though all appears correct. |
6 |
|
7 |
gzip apparently inconsistent behavior occupies the most part of the report on |
8 |
inconsistencies here (esp. the script make_gzip_archives_consistent.sh). |
9 |
|
10 |
Another part is actually on Wireshark mailing list. Pls. see: |
11 |
|
12 |
Filtering on (negated) frame.time_relative filters out wrong frame.number |
13 |
https://www.wireshark.org/lists/wireshark-users/201704/msg00037.html |
14 |
as well as my study at: |
15 |
https://www.croatiafidelis.hr/foss/cap/cap-170313-git-devuan-mail/git-devuan-mail-4.php |
16 |
(and the previous ones there, but I gave the last as it is simplest/fastest to check) |
17 |
|
18 |
There is information that any advanced reader can easily provide by retracing |
19 |
some of my steps there, and which would clear some uncertainties here. |
20 |
|
21 |
The third part is about Bash apparent misbehavior. |
22 |
|
23 |
And the fourth part is about eix complaining it has a bug, it's near the bottom part |
24 |
of this email. |
25 |
|
26 |
That is my starting point, and at this stage, after having spent quite a few |
27 |
hours on this issue below (but I have other issues not-directly-related to my system |
28 |
that haven't allowed me to finish and send this yet)... |
29 |
|
30 |
This issue below I thought at first could be related to my Wireshark woes, and |
31 |
now I don't... and at this stage (the proofreading time) I'm actually asking for |
32 |
any confirmation/denial on my findings about it... |
33 |
|
34 |
miro@g0n ~ $ ls -lh wireshark-users_miroR_q_bug_q.txt |
35 |
-rw-r--r-- 1 miro miro 802 2017-04-30 17:16 wireshark-users_miroR_q_bug_q.txt |
36 |
miro@g0n ~ $ ls -l --block-size=K wireshark-users_miroR_q_bug_q.txt |
37 |
-rw-r--r-- 1 miro miro 1K 2017-04-30 17:16 wireshark-users_miroR_q_bug_q.txt |
38 |
miro@g0n ~ $ mv -vi wireshark-users_miroR_q_bug_q.txt gzip_buggy.txt |
39 |
'wireshark-users_miroR_q_bug_q.txt' -> 'gzip_buggy.txt' |
40 |
|
41 |
miro@g0n ~ $ tar czf gzip_buggy.txt.tar.gz gzip_buggy.txt |
42 |
miro@g0n ~ $ tar czf gzip_buggy.txt_1.tar.gz gzip_buggy.txt |
43 |
miro@g0n ~ $ tar czf gzip_buggy.txt_2.tar.gz gzip_buggy.txt |
44 |
miro@g0n ~ $ sha256sum gzip_buggy.txt*.tar.gz |
45 |
26bcad15b4a81a3606859fdd4560af0e2c4cd875f1271808afb4569c50bfb5ec gzip_buggy.txt_1.tar.gz |
46 |
bd8f51bcd3274bbee4f73fa7ba9e1a2acd5db5d27f5643f72de10c52e224c22a gzip_buggy.txt_2.tar.gz |
47 |
251a57e83ebaf4f07179f4a6500be80612a39ac6d2e97e482b30fb0a4de66f56 gzip_buggy.txt.tar.gz |
48 |
miro@g0n ~ $ |
49 |
|
50 |
You may check attachments: |
51 |
gzip_buggy.txt_1.tar.gz |
52 |
gzip_buggy.txt_2.tar.gz |
53 |
gzip_buggy.txt.tar.gz |
54 |
|
55 |
I think this is not like it should be. But the important question, really, is: |
56 |
|
57 |
is this the case in my machine only, or is it Gentoo-userdom wide? |
58 |
|
59 |
I fear it's the former, but like I asked in that Wireshark thread |
60 |
( |
61 |
And it was amazing to see how little people care... there are always plenty of |
62 |
readers of Wireshark ML, and while not all can understand that thread --it |
63 |
took me years since I first used Wireshark to get to be able to minimally |
64 |
contribute there... lots of readers there such--, and of course those that can |
65 |
not really understand, need not reply... but lots of readers of Wireshark ML |
66 |
can easily repeat my steps that show in which exact fashion my Wireshark |
67 |
misbehaves... |
68 |
|
69 |
I'll be thankful is some Gentoo reader fills in this huge gap of |
70 |
carelessness.. |
71 |
) |
72 |
[but like I asked in that Wireshark thread,] I ask here on Gentoo users ML (or |
73 |
readers from elsewhere). |
74 |
|
75 |
Pls. somebody confirm if this is, or is not, the case in their Gentoo |
76 |
machines. By simply decompressing the tar-gzip'd archive are do the tar-gzip |
77 |
compression as I did, and checking the sums of them. |
78 |
|
79 |
If you don't like my text file, you can use any file (like the ASCII file |
80 |
below, which I hope should be simple and fine enough), and reproduce what I |
81 |
tried with it. |
82 |
|
83 |
So let's try /usr/bin/eix-installed-after |
84 |
It's ASCII, it's Gentoo, everybody has it on this ML, but for visitors, it's in |
85 |
|
86 |
You may check attachments: |
87 |
|
88 |
eix-installed-after_1.tar.gz |
89 |
eix-installed-after_2.tar.gz |
90 |
eix-installed-after.tar.gz |
91 |
|
92 |
In my Gentoo, I copied it in my homedir and... |
93 |
|
94 |
miro@g0n ~ $ tar czf eix-installed-after.tar.gz eix-installed-after |
95 |
miro@g0n ~ $ tar czf eix-installed-after_1.tar.gz eix-installed-after |
96 |
miro@g0n ~ $ tar czf eix-installed-after_2.tar.gz eix-installed-after |
97 |
miro@g0n ~ $ sha256sum eix-installed-after*.tar.gz |
98 |
80d004b1ecc6e6c2aec7cd453d05c7b4e4cc4092b9cef6b7c3fb31d148eb2310 eix-installed-after_1.tar.gz |
99 |
f4c30c6d4ccc90d1ec2cc6df2ac10cae2545d04a85c7ff39207d93129cd0ec78 eix-installed-after_2.tar.gz |
100 |
a8161c5771b661476966250af31d2b8bd80581a3e0c52d535b401524ee541ae7 eix-installed-after.tar.gz |
101 |
miro@g0n ~ $ |
102 |
|
103 |
And let me go further. Inspecting those...: |
104 |
|
105 |
miro@g0n ~ $ ls -l eix-installed-after*.tar.gz |
106 |
-rw-r--r-- 1 miro miro 3771 2017-04-30 17:39 eix-installed-after_1.tar.gz |
107 |
-rw-r--r-- 1 miro miro 3771 2017-04-30 17:39 eix-installed-after_2.tar.gz |
108 |
-rw-r--r-- 1 miro miro 3771 2017-04-30 17:39 eix-installed-after.tar.gz |
109 |
miro@g0n ~ $ ls -l --block-size=K eix-installed-after*.tar.gz |
110 |
-rw-r--r-- 1 miro miro 4K 2017-04-30 17:39 eix-installed-after_1.tar.gz |
111 |
-rw-r--r-- 1 miro miro 4K 2017-04-30 17:39 eix-installed-after_2.tar.gz |
112 |
-rw-r--r-- 1 miro miro 4K 2017-04-30 17:39 eix-installed-after.tar.gz |
113 |
miro@g0n ~ $ |
114 |
|
115 |
...those tiny files, I find that...: |
116 |
|
117 |
miro@g0n ~ $ hexdump -C eix-installed-after.tar.gz | head -c38 |
118 |
00000000 1f 8b 08 00 25 05 06 59 00 miro@g0n ~ $ hexdump -C eix-installed-after_1.tar.gz | head -c38 |
119 |
00000000 1f 8b 08 00 27 05 06 59 00 miro@g0n ~ $ hexdump -C eix-installed-after_2.tar.gz | head -c38 |
120 |
00000000 1f 8b 08 00 29 05 06 59 00 miro@g0n ~ $ |
121 |
miro@g0n ~ $ |
122 |
|
123 |
...it's always the... ah well, for clarity, let's head just the first 25 bytes |
124 |
of the hex representation, whihc will later be 5 bytes for dd commands...: |
125 |
|
126 |
miro@g0n ~ $ hexdump -C eix-installed-after.tar.gz | head -c25 |
127 |
00000000 1f 8b 08 00 25 miro@g0n ~ $ hexdump -C eix-installed-after_1.tar.gz | head -c25 |
128 |
00000000 1f 8b 08 00 27 miro@g0n ~ $ hexdump -C eix-installed-after_2.tar.gz | head -c25 |
129 |
00000000 1f 8b 08 00 29 miro@g0n ~ $ |
130 |
|
131 |
...that it's always (or almost always, see below about PART3 sometimes being |
132 |
different) just the 5th byte that is different, in the way that doing the |
133 |
procedure below will get those files which differed at creation time...: |
134 |
|
135 |
80d004b1ecc6e6c2aec7cd453d05c7b4e4cc4092b9cef6b7c3fb31d148eb2310 eix-installed-after_1.tar.gz |
136 |
f4c30c6d4ccc90d1ec2cc6df2ac10cae2545d04a85c7ff39207d93129cd0ec78 eix-installed-after_2.tar.gz |
137 |
a8161c5771b661476966250af31d2b8bd80581a3e0c52d535b401524ee541ae7 eix-installed-after.tar.gz |
138 |
|
139 |
( |
140 |
Notice: the hash is always new. You won't get any of those hashes above |
141 |
if you tar-gzip it. This is about the non-identical results. |
142 |
) |
143 |
|
144 |
...to become identical. This procedure, that I'll put in the script: |
145 |
|
146 |
Pls see attachment: |
147 |
make_gzip_archives_consistent.sh |
148 |
|
149 |
The fact is, I have, hours ago (this is slow work for me...) made a backup |
150 |
(and spent all this time, and more to write that script), and noticed how my |
151 |
gzip behaves, or should I say the gzip program *in my machine*... And... |
152 |
|
153 |
And I can tell that I get different instances of archives consistently fixed |
154 |
with that script. |
155 |
I only can't get identical archives when PART3 is different in some (usually |
156 |
only one of the three) of the archives. |
157 |
NOTE (at proofreading time): But I'm not investigating that anymore, find how |
158 |
I confirmed that the archives upon decompression are all of verifiably same |
159 |
content, and that suffices. |
160 |
|
161 |
I really have tried only on my backup, on multiple series of three different |
162 |
tar-gzip'ed instances each of which in the series was done on the same |
163 |
directory with the script above. |
164 |
|
165 |
The script is just a proof of concept, and a demonstration of the apparent |
166 |
issue. Because this previously does not seem to me to have been the case, but |
167 |
maybe I misremember? |
168 |
|
169 |
I checked a few of the tar-gzip'd archives of my backup, and so far they have |
170 |
always, upon decompression, shown identical in all the different instances. |
171 |
|
172 |
The finding |
173 |
=========== |
174 |
|
175 |
I've typically compared series of three instances of same directories that I |
176 |
backed up, such as /home/miro, /root and /var/log, and they always were |
177 |
identical upon decompression. |
178 |
^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ |
179 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | |
180 |
That means *probably* all is fine with gzip, and there is no issue. |
181 |
|
182 |
(But this result was not trivial to get by... and since I'm unsure in my |
183 |
conclusion, I'm still posting this, asking for confirmation or denial.) |
184 |
|
185 |
So maybe I've been chasing shadows here... And for long hours... |
186 |
|
187 |
And I'm sorry if that is so, but look at that issue that I have with |
188 |
Wireshark! Look at that! That's not a shadow. That's a serious bug or a |
189 |
serious malfunction in my Gentoo, the latter being most likely... |
190 |
|
191 |
And if it is the latter, it can only be one or the other way. One: the cause |
192 |
is in some Gentoo packge. Two: it is an attack by some unknown means. |
193 |
|
194 |
( |
195 |
If Air-Gapped is some info, I did try and editcap (and the whole |
196 |
Wireshark) behave in the same wrong way in my Air-Gapped too. |
197 |
|
198 |
Same holds for the gzip inconsistency --or "inconsistency". |
199 |
) |
200 |
|
201 |
And looking for solution to that Wireshark issue, I found the above issue |
202 |
which appeared to me to be some kind of inconsistency... |
203 |
|
204 |
And I have other inconsistencies, bogus or true that they might be, such as |
205 |
with bash. |
206 |
|
207 |
One is very quick to report. Here it goes... |
208 |
|
209 |
Thanks for any report on how this issue, and the one with Wireshark, shows, |
210 |
completely in order or just like in my machine, in any other Gentoo users' |
211 |
systems. |
212 |
|
213 |
This is one of a series of commands that I used to check one of the backups, |
214 |
in three different instances of tar-gzip'd archive I checked (such as the |
215 |
/root directory tar-gzip'd today), and which showed faultless upon |
216 |
decompression in all the three instances, despite the three instances of |
217 |
tar-gzip'd archives not being identical (as their SHA256 sums show): |
218 |
|
219 |
# for i in $(ls -1d root_170430_g0n*.d/); do sum=$(echo $i|sed 's/\.d\//\.sum/'); echo $sum ; read FAKE ; j=$(echo $i | sed 's/\.d\//\.tar.gz/'); ls -l $j $i ; cd $i; pwd ; read FAKE ; for file in $(find ./ -name '*'); do if [ -f "$file" ]; then sha256sum $file >> ../$sum ; fi; done ; cd - ; done ; |
220 |
|
221 |
Now if I just place the cursor, by moving with Alt-F (skipping "words") and Ctrl-F (skipping 1 char) to just after: |
222 |
|
223 |
"for i in $(ls -1d root_170430_g0n*.d/" in that command, |
224 |
|
225 |
and if I then hit Tab for completion on the experssion there, I get (and I'm |
226 |
sorry for the mess, but that's what I get): |
227 |
|
228 |
g0n ~ # for i in $(ls -1d root_170430_g0n*.dbash: unexpected EOF while looking for matching `)'bash: syntax error: unexpected end of file//\.tar.gz/'); ls -l $j $i ; cd $i; pwd ; read FAKE ; for file in $(find ./ -name '*'); do if [ -f "$file" ]; then sha256sum $file >> ../$sum ; fi; done ; cd - ; done ; |
229 |
|
230 |
NOTE (at proofreading time): rechecked, I do get that same behavior the day |
231 |
after (wrote most of this yesterday, still to send this morning). |
232 |
|
233 |
[[ |
234 |
NOTE (before delayed sending): In fact, it is only this clone that exibits the |
235 |
above Bash malfunctioning. I just checked the same for loop command (some six |
236 |
paragraphs above) in my Air-Gapped master [1] (never any internet it sees, |
237 |
longer workaround/detailed checking before updating it with stuff from |
238 |
internet, sneakernet or optical media), and it is just fine. That line, simply |
239 |
gave what it should: |
240 |
|
241 |
# for i in $(ls -1d root_170430_g0n*.d/); do sum=$(echo $i|sed 's/\.d\//\.sum/'); echo $sum ; read FAKE ; j=$(echo $i | sed 's/\.d\//\.tar.gz/'); ls -l $j $i ; cd $i; pwd ; read FAKE ; for file in $(find ./ -name '*'); do if [ -f "$file" ]; then sha256sum $file >> ../$sum ; fi; done ; cd - ; done |
242 |
root_170430_g0n_1.d// root_170430_g0n_2.d// root_170430_g0n.d// |
243 |
# [[and the same command line was back here]] |
244 |
|
245 |
under exact same conditions/circumstances as the clone of my Air-Gapped. And |
246 |
it's similar with some other completion issues: they seem non-existent in my |
247 |
Air-Gapped. |
248 |
]] |
249 |
|
250 |
IOW, first, Bash sullied the entire line, which is not very considerate of |
251 |
Her, and second that's not some usual error. Just for clarity, it wrote this: |
252 |
|
253 |
bash: unexpected EOF while looking for matching `)'bash: syntax error: unexpected end of file |
254 |
|
255 |
(and it wrote it by overwriting, which I never used to see in Bash) |
256 |
|
257 |
What's going on there?... Ah... Importantly: |
258 |
|
259 |
do any of you other users get some erratic unusual behavior like this with Bash? |
260 |
|
261 |
Of course, I can move to the start of the line with Ctrl-A and then issue |
262 |
Ctrl-K to clear and capture to the entire line and then issue Ctrl-Y to paste |
263 |
it back, and no disorderly message remains, but Bash isn't behaving... |
264 |
|
265 |
I'll try and send this soon, but I first need to finish my backup... |
266 |
|
267 |
Backup is done. Just, I guess if the reader has this bash version installed: |
268 |
$ bash --version |
269 |
GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu) |
270 |
Copyright (C) 2016 Free Software Foundation, Inc. |
271 |
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> |
272 |
|
273 |
This is free software; you are free to change and redistribute it. |
274 |
There is NO WARRANTY, to the extent permitted by law. |
275 |
$ |
276 |
they might be able to reproduce such kind of misbehavior. |
277 |
|
278 |
And finally, and this is what eix throws on any package that I would check: |
279 |
|
280 |
g0n ~ # eix memtest86+ |
281 |
* sys-apps/memtest86 |
282 |
Available versions: 4.3.7 (~)4.3.7-r1 {serial} |
283 |
Homepage: http://www.memtest86.com/ |
284 |
Description: A stand alone memory test for x86 computers |
285 |
|
286 |
* sys-apps/memtest86+ |
287 |
Available versions: 2.01^t 4.00^t 4.20-r1 (~)4.20-r3 5.01-r2 (~)5.01-r3 {floppy iso serial} |
288 |
Homepage: http://www.memtest.org/ |
289 |
Description: Memory tester based on memtest86 |
290 |
|
291 |
Found 2 matches |
292 |
Received SIGSEGV - you probably found a bug in eix. |
293 |
Please proceed with the following few instructions and help us find the bug: |
294 |
* install gdb (sys-devel/gdb) |
295 |
* reemerge eix with FEATURES="nostrip" CXXFLAGS="-g -ggdb3" LDFLAGS="" |
296 |
* enter gdb with "gdb --args eix your_arguments_for_eix" |
297 |
* type "run" and wait for the segfault to happen |
298 |
* type "bt" to get a backtrace (this helps us a lot) |
299 |
* post a bugreport and be sure to include the output from gdb. |
300 |
|
301 |
Sorry for the inconvenience and thanks in advance! |
302 |
g0n ~ # |
303 |
|
304 |
Too many inconsistencies. Where do I start searching for the causes? |
305 |
|
306 |
(As far as the fourth "inconsistency", I was thinking about trying memtest as |
307 |
per: |
308 |
|
309 |
Message-ID: <LO1P123MB067395FD4E9010B549743C9280120@××××××××××××××××××××××××××××××××××.COM> |
310 |
|
311 |
How to get memtest onto a USB drive |
312 |
https://lists.gt.net/gentoo/user/325837#325837 |
313 |
|
314 |
, but that's just for lack of other ideas, these issues don't look |
315 |
like bad memory. I might still try it, but when I go to sleep, not sooner. |
316 |
) |
317 |
|
318 |
Regards! |
319 |
--- |
320 |
[1] My methods are still these: |
321 |
Air-Gapped Gentoo Install, Tentative |
322 |
https://forums.gentoo.org/viewtopic-t-987268.html |
323 |
|
324 |
and |
325 |
|
326 |
Postfix smtp/TLS, Bkp/Cloning Mthd, Censorship/Intrusion |
327 |
https://forums.gentoo.org/viewtopic-t-999436.html#7613044 |
328 |
|
329 |
-- |
330 |
Miroslav Rovis |
331 |
Zagreb, Croatia |
332 |
https://www.CroatiaFidelis.hr |