Re: [gentoo-cluster] openib, no /dev/infiniband

[Bryan Green]

"Brian Budge" writes:
>
> Hi all -
>
> I'm new to infiniband and still getting my feet wet.  I am admining a very
> small cluster of 5 nodes, and have recently installed infiniband HCAs.  I
> have the infiniband modules built into the kernel, and I am using the
> openib-userspace package in the gentoo-science overlay.
>
> The strange thing with my situation is that I have infiniband working with
> openmpi on 4 of my 5 nodes, but the 5th one is a mystery.
>
> All 4 working nodes have a /dev/infiniband directory that look roughly like
> this:
>
> crw-rw---- 1 root root 231,  64 Dec 31 09:13 issm0
> crw-rw-rw- 1 root root 231, 224 Dec 31 09:13 ucm0
> crw-rw---- 1 root root 231,   0 Dec 31 09:13 umad0
> crw-rw-rw- 1 root root 231, 192 Dec 31 09:13 uverbs0
>
>
> But the 5th node doesn't, which could indicate the problem (it isn't
> completely the problem, as I tried making those nodes myself to match, but
> it doesn't help).  I'm just not sure what the difference is, because I
> installed them all the same way, they all have the same hardware, and they
> are all running the same kernel.
 
The '/dev/infiniband' subdir is created by the udev rules in '/etc/udev/rules.d/40-ib.rules'

Does the '/sys/class/infiniband' directory exist?
If so, what does it contain?  What loaded modules with an 'ib_' prefix does
lsmod report?

-bryan

-- 
gentoo-cluster@g.o mailing list