Re: [gentoo-server] boot from san with multipath support

Hi Rene,

Rene Zbinden wrote:
> hello
>
> has someone experience with installing gentoo on a san attached disk
> with mulitpath support? if someone can give me some hints, I would
> appreciate that.
>
> thanks in advance
> rene

I have tested multipath-tools from

http://christophe.varoqui.free.fr/wiki/wakka.php?wiki=Home

during the past three weeks and send some messages to the dm-devel mailing 
list. Here is a summary:




I.) ***** Installation *****

http://christophe.varoqui.free.fr/wiki/wakka.php?wiki=QuickInstallGuide

emerge multipath-tools




II.) ***** Configuration *****


1.) load dm-mapper and HBA kernel modules. lower HBA timeouts (see FAQ).


2.) create partitions on each lun (e.g.: lun0 has two paths sda and sdb - 
"fdisk sda" creates the partitions for sda _and_ sdb) - you have to do this 
step _before_ creating multipath devices (kpartx in 
/etc/dev.d/block/multipath.dev reads this partition information)


3.) create /etc/multipath.conf

- check multipath.conf.annotated and multipath.conf.synthetic in the 
multipath-tools tarball
- get lun wwid's with "scsi_id -g -u -s /block/sd?"
- create a "multipath { ... }" section for each lun (define "wwid" and "alias")
- create a "device { ... }" section for your storage (see "TestedEnvironments" 
located at the multipath-tools homepage) - set "path_grouping_policy" and 
"prio_callout" (setting "prio_callout" is important. if you choose a wrong 
value, you will see nothing after starting multipath)


4.) start multipathd (debug with "-d" or "-v4" (syslog))

Note: multipathd doesn't work with my hardware (I am using HP DL380, EVA 3000 
HSV100). But the daemon isn't needed necessarily:

My question at the dm-devel mailing list:
 >> ===> Multipathing seems to work without but not with multipathd.
 >> [...] Does I really need multipathd? I suppose so :-)
 >>

And the answer from christophe varoqui (multipath-tools developer):

 > multipathd is needed to reinstate paths.
 > In your case the rport disappears and reappears so the mecanism is all
 > hotplug-driven and thus may work without the daemon ... if memory
 > ressources permits hotplug and multipath(8) execution, that is.

Nicola (see below) has an EMC CX 300 storage system and uses multipathd 
without problems.



5.) run "multipath /dev/sd?" for each sd? device

multipath should create device-mapper table entries (see "dmsetup table"), 
block-device-files in /dev/mapper/ and links in /dev/ for the lun and for each 
partition

e.g.: you have created two partitions and your lun alias is "yellow" (defined 
in /etc/multipath.conf):

# dmsetup table
yellow1: ...
yellow: ...
yellow2: ...

# ls /dev/mapper/
yellow yellow1 yellow2 control

# ls -F /dev/yellow*
/dev/yellow@  /dev/yellow1@  /dev/yellow2@

and check "multipath [-v3] -ll"


6.) create filesystems and mount /dev/yellow1 and /dev/yellow2


you can delete the current configuration with "dmsetup remove yellow2; dmsetup 
remove yellow1; dmsetup remove yellow"




III.) ***** Test *****


I tested the multipath environment with a little perl script and a mysql database:

My trafficmaker-host executed this script 27 times (parallel):

...
for(my $count=1;$count<=1000000;$count++)
{
   ...
   my $sql="INSERT INTO $table VALUES($id,\"$value\")";
   my $return=$dbh->do($sql);
   ...
}
...
{
   my $sql="SELECT COUNT(*) FROM $table WHERE id=$id";
   my $sth=$dbh->prepare($sql);
   my $return=$sth->execute();
   ...
   $selectCount=$sth->fetchrow_array();
   ...;
}


The database host had to insert this 30 byte strings and I have started some 
copy-jobs (cp -a /usr/* /partition_mounted_with_multipath/ etc.) to increase 
the I/O load. During this test I have disabled and enabled the different 
HBA-Switch-Ports with the following result: It took 6 to 15 seconds before 
"multipath -l" showed that a path is down (15 seconds because the host had a 
30.0 CPU load and responded very slowly), but no INSERT got lost :-)))



IV.) ***** More Documentation *****

Check the homepage (I will add more documentation and helper scripts in the 
next days) and the dm-devel mailing list archive:

https://www.redhat.com/mailman/listinfo/dm-devel

E.g. Nicola Murino posted his experience with an EMC storage controller. He is 
using Gentoo too.


Hope that helps
Simon

-- 
gentoo-server@g.o mailing list