Re: [gentoo-cluster] openib, no /dev/infiniband

[Brian Budge]

Hi Bryan -

Thanks!  I inserted the rules, and restarted udev.  The permissions were
wrong on the components, but I manually changed them and it works!

Thanks again for your help,
  Brian

On Jan 2, 2008 4:41 PM, Bryan Green <bryan.d.green@××××.gov> wrote:

> "Brian Budge" writes:
> > Hi Bryan -
> >
> > I don't seem to have a 40-ib.rules in any of my /etc/udev/rules.d on any
> > node.
>
> Aha.  That file is part of sys-cluster/openib-drivers, which you don't
> have
> installed.  You can use the infiniband support that is part of the kernel,
> but the driver versions won't match your openib userspace software
> versions
> (not necessarily a problem), and you'll be missing the startup scripts.
>
> Older versions of the files are installed by the openib-files ebuild.
>  That
> ebuild is currently incompatible with openib-userspace, though it perhaps
> shouldn't be.  I guess I could fix that so you could install openib-files.
> But if possible, I'd recommend turning off the kernel builtin drivers, and
> emerge openib-drivers.
>
> In the mean time, just installing '40-ib.rules' might help, but I'm not
> sure.
>
> ####  /etc/udev/rules.d/40-ib.rules  ####
> KERNEL=="umad*", NAME="infiniband/%k"
> KERNEL=="issm*", NAME="infiniband/%k"
> KERNEL=="ucm*", NAME="infiniband/%k", MODE="0666"
> KERNEL=="uverbs*", NAME="infiniband/%k", MODE="0666"
> KERNEL=="uat", NAME="infiniband/%k", MODE="0666"
> KERNEL=="ucma", NAME="infiniband/%k", MODE="0666"
> KERNEL=="rdma_cm", NAME="infiniband/%k", MODE="0666"
> ########
>
> > My /sys/class/infiniband directory contains mthca0, which contains:
> > > ls -la /sys/class/infiniband/mthca0/
> > total 0
> > drwxr-xr-x 3 root root    0 Jan  2 20:54 .
> > drwxr-xr-x 3 root root    0 Jan  2 20:54 ..
> > -r--r--r-- 1 root root 4096 Jan  2 21:07 board_id
> > lrwxrwxrwx 1 root root    0 Jan  3 00:01 device ->
> > ../../../devices/pci0000:20/0000:20:0a.0/0000:21:00.0
> > -r--r--r-- 1 root root 4096 Jan  2 21:07 fw_ver
> > -r--r--r-- 1 root root 4096 Jan  2 21:07 hca_type
> > -r--r--r-- 1 root root 4096 Jan  2 21:07 hw_rev
> > -rw-r--r-- 1 root root 4096 Jan  2 21:07 node_desc
> > -r--r--r-- 1 root root 4096 Jan  2 21:07 node_guid
> > -r--r--r-- 1 root root 4096 Jan  2 21:06 node_type
> > drwxr-xr-x 3 root root    0 Jan  2 21:07 ports
> > lrwxrwxrwx 1 root root    0 Jan  3 00:01 subsystem ->
> > ../../../class/infiniband
> > -r--r--r-- 1 root root 4096 Jan  2 21:07 sys_image_guid
> > --w------- 1 root root 4096 Jan  2 20:54 uevent
>
> Is this what is shown on the node that does not have '/dev/infiniband'?
>
> What about '/sys/class/infiniband_verbs/'?
>
> --
> gentoo-cluster@g.o mailing list
>
>