1 |
Hello everybody |
2 |
|
3 |
After 3 days messing around I'm at my wits' end. I have two Sun X4200M2 |
4 |
servers which I would like to setup as Xen dom0s with a drbd-mirrored |
5 |
file system for paravirtualized domU images. |
6 |
|
7 |
I setup an LVM2 on top of DRBD to create the domU volumes and to be able |
8 |
to backup the domUs with LVM snapshot. |
9 |
|
10 |
I was able to configure this setup and make it run, but when accessing |
11 |
the block-frontend device in domU this is terrible slow. E.g. copying a |
12 |
5G file via scp and gigabit Ethernet onto the domU leads to a lot of |
13 |
stalls during transfer. The peak transfer rate is about 5MB/s every 1-2 |
14 |
minutes but generally it's about a few hundred KB/s. When copying the |
15 |
same file onto the DRBD volume mounted directly in dom0 it works with |
16 |
the expected 30-40MB/s. Another test was to sync the portage tree but |
17 |
also this takes maybe about 20 minutes or spontaneously raises some |
18 |
timeout errors. Sometimes after a while the disk fully detaches from the |
19 |
domU, so every call of a binary in domU leads to a "Input/output error". |
20 |
I guess the domU kernel adds this troubles, but why? There are no |
21 |
statements in any log files. E.g. /var/log/xen/xend-debug.log doesn't |
22 |
notify the detached block device. |
23 |
|
24 |
I use Gentoo Linux and tried so far with xen-3.1.2 and xen-3.2.0, |
25 |
xen-sources-2.6.18-r8, xen-sources-2.6.20-r6 and xen-sources-2.6.22, |
26 |
drbd-8.0.7, drbd-8.0.8 and drbd-8.0.11. |
27 |
|
28 |
The longer the more I expect a general problem with Xen and this setup, |
29 |
a kernel problem or a misconfiguration. Attached are the most important |
30 |
configurations for this setup. |
31 |
|
32 |
Are there any hints or other experiences for such a setup? All ideas are |
33 |
welcome. |
34 |
|
35 |
Thanks for reading and giving any hints... |
36 |
Cheers Reto Gantenbein |
37 |
|
38 |
|
39 |
################ Configurations ################################# |
40 |
|
41 |
/etc/drbd.conf |
42 |
,---------------------------------------------------------------- |
43 |
| global { |
44 |
| usage-count no; |
45 |
| dialog-refresh 1; |
46 |
| } |
47 |
| |
48 |
| common { |
49 |
| startup { |
50 |
| wfc-timeout 60; |
51 |
| degr-wfc-timeout 60; |
52 |
| } |
53 |
| |
54 |
| disk { |
55 |
| on-io-error detach; |
56 |
| # also tried 'pass-on' with same behavior |
57 |
| } |
58 |
| |
59 |
| syncer { |
60 |
| rate 500M; |
61 |
| al-extents 80; |
62 |
| } |
63 |
| } |
64 |
| |
65 |
| resource "gentoo" { |
66 |
| protocol C; |
67 |
| net { |
68 |
| # required for xen live migration |
69 |
| allow-two-primaries; |
70 |
| after-sb-0pri discard-zero-changes; |
71 |
| after-sb-1pri discard-secondary; |
72 |
| } |
73 |
| |
74 |
| on vserver03 { |
75 |
| device /dev/drbd1; |
76 |
| disk /dev/xen-vg/gentoo; |
77 |
| address 10.0.0.1:7789; |
78 |
| meta-disk internal; |
79 |
| } |
80 |
| |
81 |
| on vserver04 { |
82 |
| device /dev/drbd1; |
83 |
| disk /dev/xen-vg/gentoo; |
84 |
| address 10.0.0.2:7789; |
85 |
| meta-disk internal; |
86 |
| } |
87 |
| } |
88 |
`---------------------------------------------------------------- |
89 |
|
90 |
/etc/xen/gentoo |
91 |
,---------------------------------------------------------------- |
92 |
| name = "gentoo" |
93 |
| |
94 |
| kernel = "/cluster/xen/kernel/vmlinuz-gentoo-domU" |
95 |
| extra = "" |
96 |
| root = "/dev/hda1" |
97 |
| |
98 |
| memory = 4096 |
99 |
| vcpus = 3 |
100 |
| disk = [ 'drbd:gentoo,hda1,w' ] |
101 |
| vif = [ 'mac=00:40:a6:00:45:01,bridge=xenbr-core', |
102 |
| 'mac=00:40:a6:00:45:02,bridge=xenbr-public' ] |
103 |
`---------------------------------------------------------------- |
104 |
-- |
105 |
gentoo-cluster@l.g.o mailing list |