Gentoo Archives: gentoo-user

From: Joris Dobbelsteen <joris@××××××××××××××××××.nl>
To: gentoo-user@l.g.o
Subject: [gentoo-user] Xen 3.2.1 - all HVM network packets dropped
Date: Mon, 16 Jun 2008 12:51:22
Message-Id: 485661C1.1080702@familiedobbelsteen.nl
Dear,

I've attempted to upgrade a Xen box from 3.1.2 to 3.2.1. Unfortunately
it has been a partial success only. At this point I'm really stuck
with a network problem that affects my HVM machines (Linux PV run
fine).

The really odd thing is that everything seems to work properly, except
that the "vif" network interfaces in dom0 seem to have only dropped TX
packets for the HVM domains and 0 RX packets. It seems like something
is broken in the network communication path between the HVM domain and
dom0.

A second change I made was to use Gentoo for configuration of the
bridge device, rather than the network-bridge script. I did this
already for some other bridges, but these don't have any HVM domains
connected.

Also, ipvs (linux virtual server) doesn't seem to work. Although I 
cannot yet claim this is caused by the upgrade, it seems likely. Xen was 
te only change made before I broke that functionality.
Every configuration, I expect you should know is, I hope, included below:

Anyone know/guessing what's going wrong?
Is there any known problem?
How can I solve this?
How can I diagnose this?

Unfortunally I need my services and I restored my previous backup, so I 
don't really have a box handy any more to try out these things.

- Joris

===

*Configuration*

Hardware
Intel Core 2 Duo E6750 2.66 GHz (clocked at 2.0 GHz)
Intel PRO/1000 NIC (eth0)
Realtek RTL8168b/8111b (eth1)

Hypervisor:
Gentoo 2007, Xen 3.2.1. 64-bit
(XEN) Xen version 3.2.1 (root@×××××××.local) (gcc version 3.4.6
(Gentoo Hardened 3.4.6-r2 p1.5, ssp-3.4.6-1.0, pie-8.7.10)) Sun Jun 15
19:54:30 CEST 2008

Dom0:
Gentoo 2007 Linux. 64-bit
Linux version 2.6.20-xen-r6 (root@susa) (gcc version 3.4.6 (Gentoo
Hardened 3.4.6-r2 p1.5, ssp-3.4.6-1.0, pie-8.7.10)) #15 SMP Tue Jan 22
17:12:46 CET 2008
(This version was working perfectly in combination with Xen 3.1.2)

DomU (HVM):
Windows Server 2003 SP2, 32-bit
Ubuntu 6.06 LTS Dapper Drake, 32-bit

===

*Dom0 ifconfig (portions only)*

Please notice the "TX dropped!"
eth0      Link encap:Ethernet  HWaddr 00:1B:21:05:02:77
           UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
           RX packets:58606 errors:0 dropped:0 overruns:0 frame:0
           TX packets:97615 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:5015772 (4.7 Mb)  TX bytes:43265853 (41.2 Mb)
           Base address:0xb000 Memory:f1020000-f1040000
--
eth1      Link encap:Ethernet  HWaddr 00:1A:4D:50:02:9C
           UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
           RX packets:979621 errors:0 dropped:0 overruns:0 frame:0
           TX packets:967219 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:300713690 (286.7 Mb)  TX bytes:292573959 (279.0 Mb)
           Interrupt:16 Base address:0x6000
--
vif16.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:11676 overruns:0 carrier:0
           collisions:0 txqueuelen:32
           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

--
vif18.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:11617 overruns:0 carrier:0
           collisions:0 txqueuelen:32
           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

vif19.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:11600 overruns:0 carrier:0
           collisions:0 txqueuelen:32
           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

--
vif21.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:11428 overruns:0 carrier:0
           collisions:0 txqueuelen:32
           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
--
xenbr0    Link encap:Ethernet  HWaddr 00:1B:21:05:02:77
           inet addr:192.168.10.32  Bcast:192.168.10.255  Mask:255.255.255.0
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:77866 errors:0 dropped:0 overruns:0 frame:0
           TX packets:67193 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:8868267 (8.4 Mb)  TX bytes:35487507 (33.8 Mb)

xenbr1    Link encap:Ethernet  HWaddr 00:1A:4D:50:02:9C
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:213 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:9700 (9.4 Kb)  TX bytes:0 (0.0 b)

xenbr2    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:2 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:56 (56.0 b)  TX bytes:0 (0.0 b)

===

*Configuration files:*
#  -*- mode: python; -*-
import os, re
arch = os.uname()[4]
if re.search('64', arch):
     arch_libdir = 'lib64'
else:
     arch_libdir = 'lib'
kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
memory = 512
name = "shushan"
vcpus=1
pae=1
acpi=1
apic=1
vif = [ 'type=ioemu, mac=02:60:68:e5:e4:a5, bridge=xenbr0' ]
disk = [ 'phy:/dev/vg_raid_continues/shushan_primary,ioemu:hda,w' ]
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
boot="cd"
sdl=0
vnc=1
vncdisplay=99
vncunused=0
stdvga=0
serial='pty'

(Other HVM are just variations on this script, with only the obvious
parameters different (memory, name, vcups, vif[mac], disk[location],
vncdisplay).

===

*brctl show*
bridge name     bridge id               STP enabled     interfaces
xenbr0          8000.001b21050277       no              eth0
                                                         vif13.0
                                                         vif14.0
                                                         vif15.0
                                                         vif16.0
                                                         vif17.0
                                                         vif18.0
                                                         vif19.0
                                                         vif20.0
                                                         vif21.0
                                                         vif22.0
                                                         vif23.0
                                                         vif24.0
xenbr1          8000.001a4d50029c       no              eth1
                                                         vif14.2
xenbr2          8000.feffffffffff       no              vif14.1

This looks very OK.

===

*/etc/conf.d/net (Gentoo)*
modules=( "iproute2" )
dns_domain=( "joris2k.local" )
dns_domain_lo=( "joris2k.local" )
dns_domain_eth0=( "joris2k.local" )
dns_domain_eth1=( "joris2k.local" )
dns_servers=( "192.168.10.a 192.168.10.b 192.168.10.c" )
dns_servers_eth0=( "192.168.10.a 192.168.10.b 192.168.10.c" )
dns_servers_eth1=( "192.168.10.a 192.168.10.b 192.168.10.c" )
config_eth0=("null")
bridge_xenbr0=("eth0")
brctl_xenbr0=("setfd 0" "sethello 0" "stp off")
config_xenbr0=( "192.168.10.d netmask 255.255.255.0" )
routes_xenbr0=( "default via 192.168.10.e" )
depend_xenbr0() {
         need net.eth0
}
config_eth1=( "null" )
brctl_xenbr1=("setfd 0" "sethello 0" "stp off")
bridge_xenbr1=("eth1")
config_xenbr1=("null")
depend_xenbr1() {
         need net.eth1
}
brctl_xenbr2=("setfd 0" "sethello 0" "stp off")
config_xenbr2=( "null" )


-- 
gentoo-user@l.g.o mailing list