1 |
Hi Rene, |
2 |
|
3 |
Rene Zbinden wrote: |
4 |
> hello |
5 |
> |
6 |
> has someone experience with installing gentoo on a san attached disk |
7 |
> with mulitpath support? if someone can give me some hints, I would |
8 |
> appreciate that. |
9 |
> |
10 |
> thanks in advance |
11 |
> rene |
12 |
|
13 |
I have tested multipath-tools from |
14 |
|
15 |
http://christophe.varoqui.free.fr/wiki/wakka.php?wiki=Home |
16 |
|
17 |
during the past three weeks and send some messages to the dm-devel mailing |
18 |
list. Here is a summary: |
19 |
|
20 |
|
21 |
|
22 |
|
23 |
I.) ***** Installation ***** |
24 |
|
25 |
http://christophe.varoqui.free.fr/wiki/wakka.php?wiki=QuickInstallGuide |
26 |
|
27 |
emerge multipath-tools |
28 |
|
29 |
|
30 |
|
31 |
|
32 |
II.) ***** Configuration ***** |
33 |
|
34 |
|
35 |
1.) load dm-mapper and HBA kernel modules. lower HBA timeouts (see FAQ). |
36 |
|
37 |
|
38 |
2.) create partitions on each lun (e.g.: lun0 has two paths sda and sdb - |
39 |
"fdisk sda" creates the partitions for sda _and_ sdb) - you have to do this |
40 |
step _before_ creating multipath devices (kpartx in |
41 |
/etc/dev.d/block/multipath.dev reads this partition information) |
42 |
|
43 |
|
44 |
3.) create /etc/multipath.conf |
45 |
|
46 |
- check multipath.conf.annotated and multipath.conf.synthetic in the |
47 |
multipath-tools tarball |
48 |
- get lun wwid's with "scsi_id -g -u -s /block/sd?" |
49 |
- create a "multipath { ... }" section for each lun (define "wwid" and "alias") |
50 |
- create a "device { ... }" section for your storage (see "TestedEnvironments" |
51 |
located at the multipath-tools homepage) - set "path_grouping_policy" and |
52 |
"prio_callout" (setting "prio_callout" is important. if you choose a wrong |
53 |
value, you will see nothing after starting multipath) |
54 |
|
55 |
|
56 |
4.) start multipathd (debug with "-d" or "-v4" (syslog)) |
57 |
|
58 |
Note: multipathd doesn't work with my hardware (I am using HP DL380, EVA 3000 |
59 |
HSV100). But the daemon isn't needed necessarily: |
60 |
|
61 |
My question at the dm-devel mailing list: |
62 |
>> ===> Multipathing seems to work without but not with multipathd. |
63 |
>> [...] Does I really need multipathd? I suppose so :-) |
64 |
>> |
65 |
|
66 |
And the answer from christophe varoqui (multipath-tools developer): |
67 |
|
68 |
> multipathd is needed to reinstate paths. |
69 |
> In your case the rport disappears and reappears so the mecanism is all |
70 |
> hotplug-driven and thus may work without the daemon ... if memory |
71 |
> ressources permits hotplug and multipath(8) execution, that is. |
72 |
|
73 |
Nicola (see below) has an EMC CX 300 storage system and uses multipathd |
74 |
without problems. |
75 |
|
76 |
|
77 |
|
78 |
5.) run "multipath /dev/sd?" for each sd? device |
79 |
|
80 |
multipath should create device-mapper table entries (see "dmsetup table"), |
81 |
block-device-files in /dev/mapper/ and links in /dev/ for the lun and for each |
82 |
partition |
83 |
|
84 |
e.g.: you have created two partitions and your lun alias is "yellow" (defined |
85 |
in /etc/multipath.conf): |
86 |
|
87 |
# dmsetup table |
88 |
yellow1: ... |
89 |
yellow: ... |
90 |
yellow2: ... |
91 |
|
92 |
# ls /dev/mapper/ |
93 |
yellow yellow1 yellow2 control |
94 |
|
95 |
# ls -F /dev/yellow* |
96 |
/dev/yellow@ /dev/yellow1@ /dev/yellow2@ |
97 |
|
98 |
and check "multipath [-v3] -ll" |
99 |
|
100 |
|
101 |
6.) create filesystems and mount /dev/yellow1 and /dev/yellow2 |
102 |
|
103 |
|
104 |
you can delete the current configuration with "dmsetup remove yellow2; dmsetup |
105 |
remove yellow1; dmsetup remove yellow" |
106 |
|
107 |
|
108 |
|
109 |
|
110 |
III.) ***** Test ***** |
111 |
|
112 |
|
113 |
I tested the multipath environment with a little perl script and a mysql database: |
114 |
|
115 |
My trafficmaker-host executed this script 27 times (parallel): |
116 |
|
117 |
... |
118 |
for(my $count=1;$count<=1000000;$count++) |
119 |
{ |
120 |
... |
121 |
my $sql="INSERT INTO $table VALUES($id,\"$value\")"; |
122 |
my $return=$dbh->do($sql); |
123 |
... |
124 |
} |
125 |
... |
126 |
{ |
127 |
my $sql="SELECT COUNT(*) FROM $table WHERE id=$id"; |
128 |
my $sth=$dbh->prepare($sql); |
129 |
my $return=$sth->execute(); |
130 |
... |
131 |
$selectCount=$sth->fetchrow_array(); |
132 |
...; |
133 |
} |
134 |
|
135 |
|
136 |
The database host had to insert this 30 byte strings and I have started some |
137 |
copy-jobs (cp -a /usr/* /partition_mounted_with_multipath/ etc.) to increase |
138 |
the I/O load. During this test I have disabled and enabled the different |
139 |
HBA-Switch-Ports with the following result: It took 6 to 15 seconds before |
140 |
"multipath -l" showed that a path is down (15 seconds because the host had a |
141 |
30.0 CPU load and responded very slowly), but no INSERT got lost :-))) |
142 |
|
143 |
|
144 |
|
145 |
IV.) ***** More Documentation ***** |
146 |
|
147 |
Check the homepage (I will add more documentation and helper scripts in the |
148 |
next days) and the dm-devel mailing list archive: |
149 |
|
150 |
https://www.redhat.com/mailman/listinfo/dm-devel |
151 |
|
152 |
E.g. Nicola Murino posted his experience with an EMC storage controller. He is |
153 |
using Gentoo too. |
154 |
|
155 |
|
156 |
Hope that helps |
157 |
Simon |
158 |
|
159 |
-- |
160 |
gentoo-server@g.o mailing list |