1 |
Stefan, |
2 |
|
3 |
|
4 |
On maandag 29 juli 2019 21:28:50 CEST Stefan G. Weichinger wrote: |
5 |
> At a customer I have to check through an older gentoo server. |
6 |
> |
7 |
> The former admin is not available anymore and among other things I have |
8 |
> to check how the SAN storage is attached. |
9 |
|
10 |
If you ever encounter that admin, make sure you hide the body :) |
11 |
|
12 |
> As I have to plan a new installation with minimal downtime I'd like to |
13 |
> understand that multipath-stuff before any changes ;-) |
14 |
|
15 |
I use multipath, but only using internal HBAs connected to internal |
16 |
backplanes. |
17 |
|
18 |
> The system runs: |
19 |
> |
20 |
> sys-fs/multipath-tools-0.5.0-r1 |
21 |
|
22 |
I use "sys-fs/multipath-tools-0.7.9" |
23 |
|
24 |
> and has a multipath.conf: |
25 |
|
26 |
Same here |
27 |
|
28 |
> (rm-ed comments) |
29 |
> |
30 |
> defaults { |
31 |
> # udev_dir /dev |
32 |
> polling_interval 15 |
33 |
> # selector "round-robin 0" |
34 |
> path_grouping_policy group_by_prio |
35 |
> failback 5 |
36 |
> path_checker tur |
37 |
> # prio_callout "/sbin/mpath_prio_tpc /dev/%n" |
38 |
> rr_min_io 100 |
39 |
> rr_weight uniform |
40 |
> no_path_retry queue |
41 |
> user_friendly_names yes |
42 |
> |
43 |
> } |
44 |
> blacklist { |
45 |
> devnode cciss |
46 |
> devnode fd |
47 |
> devnode hd |
48 |
> devnode md |
49 |
> devnode sr |
50 |
> devnode scd |
51 |
> devnode st |
52 |
> devnode ram |
53 |
> devnode raw |
54 |
> devnode loop |
55 |
> devnode sda |
56 |
> devnode sdb |
57 |
> } |
58 |
> |
59 |
> multipaths { |
60 |
> multipath { |
61 |
> wwid 3600c0ff0001e91b2c1bae25601000000 |
62 |
> ## To find your wwid, please use /usr/bin/sg_vpd --page=di /dev/DEVICE. |
63 |
> ## The address will be a 0x6. Remove the 0x and replace it with 3. |
64 |
> alias MSA2040_SAMBA_storage |
65 |
> } |
66 |
> } |
67 |
|
68 |
This looks like a default one. Mine is far simpler: |
69 |
*** |
70 |
defaults { |
71 |
path_grouping_policy multibus |
72 |
path_selector "queue-length 0" |
73 |
rr_min_io_rq 100 |
74 |
} |
75 |
*** |
76 |
|
77 |
Do you have any files in "/etc/multipath"? I have 2: |
78 |
"bindings" (which only contains comments) |
79 |
"wwids" (which, aside from comments, shows the IDs from the harddrives. |
80 |
|
81 |
Both of these files mention they are automatically maintained. |
82 |
|
83 |
I don't "hide" devices from multipath and let it figure it out by itself |
84 |
|
85 |
> "multipath -l" and "-ll" show nothing. |
86 |
|
87 |
Then multipath is NOT working, I get the following (only showing first 2 |
88 |
devices): |
89 |
|
90 |
*** |
91 |
35000cca25d8ec910 dm-4 HGST,HUS726040ALS210 |
92 |
size=3.6T features='0' hwhandler='0' wp=rw |
93 |
`-+- policy='queue-length 0' prio=1 status=active |
94 |
|- 0:0:20:0 sdt 65:48 active ready running |
95 |
|- 0:0:7:0 sdh 8:112 active ready running |
96 |
|- 1:0:7:0 sdaf 65:240 active ready running |
97 |
`- 1:0:20:0 sdar 66:176 active ready running |
98 |
35000cca25d8b5e78 dm-7 HGST,HUS726040ALS210 |
99 |
size=3.6T features='0' hwhandler='0' wp=rw |
100 |
`-+- policy='queue-length 0' prio=1 status=active |
101 |
|- 0:0:21:0 sdu 65:64 active ready running |
102 |
|- 0:0:8:0 sdi 8:128 active ready running |
103 |
|- 1:0:8:0 sdag 66:0 active ready running |
104 |
`- 1:0:21:0 sdas 66:192 active ready running |
105 |
*** |
106 |
|
107 |
As per the above, every physical disk is seen 4 times by the system. |
108 |
I have 2 HBAs connected to backplanes and as these are SAS-drives, every disk |
109 |
is connected twice to the backplanes. |
110 |
In other words, I have 4 different paths to get to every single disk. |
111 |
|
112 |
> dmesg: |
113 |
> |
114 |
> # dmesg | grep multi |
115 |
> [ 1.144947] md: multipath personality registered for level -4 |
116 |
> [ 1.145679] device-mapper: multipath: version 1.9.0 loaded |
117 |
> [ 1.145857] device-mapper: multipath round-robin: version 1.0.0 loaded |
118 |
> [21827451.284100] device-mapper: table: 253:0: multipath: unknown path |
119 |
> selector type |
120 |
> [21827451.285432] device-mapper: table: 253:0: multipath: unknown path |
121 |
> selector type |
122 |
> [21827496.130239] device-mapper: table: 253:0: multipath: unknown path |
123 |
> selector type |
124 |
> [21827496.131379] device-mapper: table: 253:0: multipath: unknown path |
125 |
> selector type |
126 |
> [21827497.576482] device-mapper: table: 253:0: multipath: unknown path |
127 |
> selector type |
128 |
> |
129 |
> - |
130 |
> |
131 |
> I see two devices sdc and sdd that should come from the SAN. |
132 |
|
133 |
Interesting, are these supposed to be the same? |
134 |
what do you get back from: |
135 |
|
136 |
# /usr/bin/sg_vpd --page=di /dev/sdc |
137 |
# /usr/bin/sg_vpd --page=di /dev/sdd |
138 |
(As suggested in the multipathd.conf file you listed above) |
139 |
|
140 |
On my system I get the following for "sdt: and "sdh" (first disk listed in |
141 |
above multipath output): |
142 |
*** |
143 |
# /usr/bin/sg_vpd --page=di /dev/sdt |
144 |
Device Identification VPD page: |
145 |
Addressed logical unit: |
146 |
designator type: NAA, code set: Binary |
147 |
0x5000cca25d8ec910 |
148 |
Target port: |
149 |
designator type: NAA, code set: Binary |
150 |
transport: Serial Attached SCSI Protocol (SPL-4) |
151 |
0x5000cca25d8ec911 |
152 |
designator type: Relative target port, code set: Binary |
153 |
transport: Serial Attached SCSI Protocol (SPL-4) |
154 |
Relative target port: 0x1 |
155 |
Target device that contains addressed lu: |
156 |
designator type: NAA, code set: Binary |
157 |
transport: Serial Attached SCSI Protocol (SPL-4) |
158 |
0x5000cca25d8ec913 |
159 |
designator type: SCSI name string, code set: UTF-8 |
160 |
SCSI name string: |
161 |
naa.5000CCA25D8EC913 |
162 |
# /usr/bin/sg_vpd --page=di /dev/sdh |
163 |
Device Identification VPD page: |
164 |
Addressed logical unit: |
165 |
designator type: NAA, code set: Binary |
166 |
0x5000cca25d8ec910 |
167 |
Target port: |
168 |
designator type: NAA, code set: Binary |
169 |
transport: Serial Attached SCSI Protocol (SPL-4) |
170 |
0x5000cca25d8ec912 |
171 |
designator type: Relative target port, code set: Binary |
172 |
transport: Serial Attached SCSI Protocol (SPL-4) |
173 |
Relative target port: 0x2 |
174 |
Target device that contains addressed lu: |
175 |
designator type: NAA, code set: Binary |
176 |
transport: Serial Attached SCSI Protocol (SPL-4) |
177 |
0x5000cca25d8ec913 |
178 |
designator type: SCSI name string, code set: UTF-8 |
179 |
SCSI name string: |
180 |
naa.5000CCA25D8EC913 |
181 |
*** |
182 |
|
183 |
The "wwid" entry for this would be "35000cca25d8ec910" |
184 |
(The value for "Adressed logical unit", with "0x" replaced with "3") |
185 |
|
186 |
If "sdc" and "sdd" are the same disk, the "Adressed logical unit" id should be |
187 |
the same for both. |
188 |
|
189 |
> Could someone help me to research this in more detail? |
190 |
|
191 |
I can try. |
192 |
> This is a production server, I can't change much ... |
193 |
|
194 |
Understood, no changes have been recommended yet. We will get to those later. |
195 |
|
196 |
> I would like to find out how to reliably mount these SAN-devices into a |
197 |
> new OS (maybe a new gentoo installation is coming). |
198 |
|
199 |
If it is old, I would suggest that as well. |
200 |
|
201 |
-- |
202 |
Joost |