Gentoo Archives: gentoo-user

From: "J. Roeleveld" <joost@××××××××.org>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] multipath.conf : learning how to use
Date: Wed, 14 Aug 2019 06:37:18
Message-Id: 3271924.fr91aXinDs@andromeda
In Reply to: [gentoo-user] multipath.conf : learning how to use by "Stefan G. Weichinger"
1 Stefan,
2
3
4 On maandag 29 juli 2019 21:28:50 CEST Stefan G. Weichinger wrote:
5 > At a customer I have to check through an older gentoo server.
6 >
7 > The former admin is not available anymore and among other things I have
8 > to check how the SAN storage is attached.
9
10 If you ever encounter that admin, make sure you hide the body :)
11
12 > As I have to plan a new installation with minimal downtime I'd like to
13 > understand that multipath-stuff before any changes ;-)
14
15 I use multipath, but only using internal HBAs connected to internal
16 backplanes.
17
18 > The system runs:
19 >
20 > sys-fs/multipath-tools-0.5.0-r1
21
22 I use "sys-fs/multipath-tools-0.7.9"
23
24 > and has a multipath.conf:
25
26 Same here
27
28 > (rm-ed comments)
29 >
30 > defaults {
31 > # udev_dir /dev
32 > polling_interval 15
33 > # selector "round-robin 0"
34 > path_grouping_policy group_by_prio
35 > failback 5
36 > path_checker tur
37 > # prio_callout "/sbin/mpath_prio_tpc /dev/%n"
38 > rr_min_io 100
39 > rr_weight uniform
40 > no_path_retry queue
41 > user_friendly_names yes
42 >
43 > }
44 > blacklist {
45 > devnode cciss
46 > devnode fd
47 > devnode hd
48 > devnode md
49 > devnode sr
50 > devnode scd
51 > devnode st
52 > devnode ram
53 > devnode raw
54 > devnode loop
55 > devnode sda
56 > devnode sdb
57 > }
58 >
59 > multipaths {
60 > multipath {
61 > wwid 3600c0ff0001e91b2c1bae25601000000
62 > ## To find your wwid, please use /usr/bin/sg_vpd --page=di /dev/DEVICE.
63 > ## The address will be a 0x6. Remove the 0x and replace it with 3.
64 > alias MSA2040_SAMBA_storage
65 > }
66 > }
67
68 This looks like a default one. Mine is far simpler:
69 ***
70 defaults {
71 path_grouping_policy multibus
72 path_selector "queue-length 0"
73 rr_min_io_rq 100
74 }
75 ***
76
77 Do you have any files in "/etc/multipath"? I have 2:
78 "bindings" (which only contains comments)
79 "wwids" (which, aside from comments, shows the IDs from the harddrives.
80
81 Both of these files mention they are automatically maintained.
82
83 I don't "hide" devices from multipath and let it figure it out by itself
84
85 > "multipath -l" and "-ll" show nothing.
86
87 Then multipath is NOT working, I get the following (only showing first 2
88 devices):
89
90 ***
91 35000cca25d8ec910 dm-4 HGST,HUS726040ALS210
92 size=3.6T features='0' hwhandler='0' wp=rw
93 `-+- policy='queue-length 0' prio=1 status=active
94 |- 0:0:20:0 sdt 65:48 active ready running
95 |- 0:0:7:0 sdh 8:112 active ready running
96 |- 1:0:7:0 sdaf 65:240 active ready running
97 `- 1:0:20:0 sdar 66:176 active ready running
98 35000cca25d8b5e78 dm-7 HGST,HUS726040ALS210
99 size=3.6T features='0' hwhandler='0' wp=rw
100 `-+- policy='queue-length 0' prio=1 status=active
101 |- 0:0:21:0 sdu 65:64 active ready running
102 |- 0:0:8:0 sdi 8:128 active ready running
103 |- 1:0:8:0 sdag 66:0 active ready running
104 `- 1:0:21:0 sdas 66:192 active ready running
105 ***
106
107 As per the above, every physical disk is seen 4 times by the system.
108 I have 2 HBAs connected to backplanes and as these are SAS-drives, every disk
109 is connected twice to the backplanes.
110 In other words, I have 4 different paths to get to every single disk.
111
112 > dmesg:
113 >
114 > # dmesg | grep multi
115 > [ 1.144947] md: multipath personality registered for level -4
116 > [ 1.145679] device-mapper: multipath: version 1.9.0 loaded
117 > [ 1.145857] device-mapper: multipath round-robin: version 1.0.0 loaded
118 > [21827451.284100] device-mapper: table: 253:0: multipath: unknown path
119 > selector type
120 > [21827451.285432] device-mapper: table: 253:0: multipath: unknown path
121 > selector type
122 > [21827496.130239] device-mapper: table: 253:0: multipath: unknown path
123 > selector type
124 > [21827496.131379] device-mapper: table: 253:0: multipath: unknown path
125 > selector type
126 > [21827497.576482] device-mapper: table: 253:0: multipath: unknown path
127 > selector type
128 >
129 > -
130 >
131 > I see two devices sdc and sdd that should come from the SAN.
132
133 Interesting, are these supposed to be the same?
134 what do you get back from:
135
136 # /usr/bin/sg_vpd --page=di /dev/sdc
137 # /usr/bin/sg_vpd --page=di /dev/sdd
138 (As suggested in the multipathd.conf file you listed above)
139
140 On my system I get the following for "sdt: and "sdh" (first disk listed in
141 above multipath output):
142 ***
143 # /usr/bin/sg_vpd --page=di /dev/sdt
144 Device Identification VPD page:
145 Addressed logical unit:
146 designator type: NAA, code set: Binary
147 0x5000cca25d8ec910
148 Target port:
149 designator type: NAA, code set: Binary
150 transport: Serial Attached SCSI Protocol (SPL-4)
151 0x5000cca25d8ec911
152 designator type: Relative target port, code set: Binary
153 transport: Serial Attached SCSI Protocol (SPL-4)
154 Relative target port: 0x1
155 Target device that contains addressed lu:
156 designator type: NAA, code set: Binary
157 transport: Serial Attached SCSI Protocol (SPL-4)
158 0x5000cca25d8ec913
159 designator type: SCSI name string, code set: UTF-8
160 SCSI name string:
161 naa.5000CCA25D8EC913
162 # /usr/bin/sg_vpd --page=di /dev/sdh
163 Device Identification VPD page:
164 Addressed logical unit:
165 designator type: NAA, code set: Binary
166 0x5000cca25d8ec910
167 Target port:
168 designator type: NAA, code set: Binary
169 transport: Serial Attached SCSI Protocol (SPL-4)
170 0x5000cca25d8ec912
171 designator type: Relative target port, code set: Binary
172 transport: Serial Attached SCSI Protocol (SPL-4)
173 Relative target port: 0x2
174 Target device that contains addressed lu:
175 designator type: NAA, code set: Binary
176 transport: Serial Attached SCSI Protocol (SPL-4)
177 0x5000cca25d8ec913
178 designator type: SCSI name string, code set: UTF-8
179 SCSI name string:
180 naa.5000CCA25D8EC913
181 ***
182
183 The "wwid" entry for this would be "35000cca25d8ec910"
184 (The value for "Adressed logical unit", with "0x" replaced with "3")
185
186 If "sdc" and "sdd" are the same disk, the "Adressed logical unit" id should be
187 the same for both.
188
189 > Could someone help me to research this in more detail?
190
191 I can try.
192 > This is a production server, I can't change much ...
193
194 Understood, no changes have been recommended yet. We will get to those later.
195
196 > I would like to find out how to reliably mount these SAN-devices into a
197 > new OS (maybe a new gentoo installation is coming).
198
199 If it is old, I would suggest that as well.
200
201 --
202 Joost

Replies

Subject Author
Re: [gentoo-user] multipath.conf : learning how to use "Stefan G. Weichinger" <lists@×××××.at>