InfiniBand is a switched fabric computer network communications link used in high-performance computing and enterprise data centers. This tag should be used for questions about IB related hardware and software.

learn more… | top users | synonyms

1
vote
1answer
22 views

set the Ca name on unmanaged mellanox switch

How can I set the Hostname/Description of a Mellanox/Infiniband unmanaged switch? I would like a way to abstractly distinguish quickly which switches are which when doing 'ibswitches' or ...
0
votes
0answers
43 views

Centos. infiniband multicast join failed for status -22

I have a cluster with Linux Centos 6.5 as OS. All the machines are connected by infiniband, the network is working fine but in the /var/log/messages of all the machines I can see the following message ...
0
votes
0answers
8 views

How to disconnect a SRP (SCSI RDMA Protocol) drive after unmounting

I have an InfiniBand network working with SRP sharing. On the target system, I am running targetcli (the -fb free branch version) and OpenFabric's srptools. On the initiator system, I am running ...
2
votes
2answers
23 views

Mellanox and Infiniband latency

I got two hosts with Voltaire HCA500Ex-D (MT25408 ConnectX Mellanox) 10Gbit cards connected to a Cisco SFS7000d IB switch via CX4 3m cables. I'm really concern about latency which is higher than on ...
0
votes
0answers
22 views

forwarding traffic between the ports

How to set up forwarding traffic between interfaces "HCA"? I want to implement a ring topology. PC Listing GUIDs: Port 0: 0x2c903000f6d27 Port 1: 0x2c903000f6d28 PC Listing GUIDs: Port 0: ...
0
votes
1answer
107 views

VMware infiniband configuration

I have a server which is intended to be a vmware host. This server has two Infiniband Mellanox ConnectX-2 dual-port cards. What I want to do is to make the following setup: The first card should work ...
2
votes
2answers
129 views

Infiniband Verifying that RDMA is working

I have two identical computers with Mellanox cards connected to each other through a cable. No switch. Using opensm. I have run several tests, including ping_pong tests, ibping, etc. They all seem to ...
0
votes
0answers
115 views

opensm CentOS 7 software switch

I have two computers both with infiniband cards in them that I would like to connect without a switch. I have a cable connecting the two computers through their QSFP ports. I have read the ...
3
votes
1answer
227 views

Connect two infiniband cards to each other without a switch

Is it possible to connect two Mellanox ConnectX-2 cards on two seperate computers to each other without a switch in between? I am just trying to learn to use verbs programming without having to spend ...
1
vote
1answer
64 views

Infiniband cable

What is the difference between an active and a passive cable? What is the difference between QSFP and QSFP+? Given the choice between copper and fiber, for a very short distance like 3 feet max, is ...
1
vote
0answers
386 views

CentOS 7 - Systemctl dependency problems with network mount and OpenSM

I have the follwoing problem. I want to mount a glusterfs share over Infiniband on an CentOS 7 client. The glusterfs share '/data' is severed by a host called storage1. For the client I I setup and ...
1
vote
0answers
110 views

Setting up Infiniband network - wrong route

So I'm trying to set up an Infiniband network alongside my Ethernet network. I have 10 compute nodes and one conductor node. All 11 machines are running CentOS and are plugged in to an Infiniband ...
0
votes
1answer
91 views

Debian-based Infiniband point-to-point Voltaire dual port cards

I have the following configuration: I------------------I I------------------I I I(P0)---------(P0)I I I Voltaire HCA 0 I I ...
0
votes
0answers
94 views

opensm infiniband route table

I have two centos 6.6 vmware vms. VMware is told to do a bridged network and to replicate physical network. eth2 is dhcp managed by NetworkManager. I am able to get to the point where I can assign ips ...
0
votes
0answers
180 views

Building OFED 3.12 Ubuntu

I am trying to build OFED 3.12 (I have tried many versions they all have unique problems) on Ubuntu 14.04 TLS. When I run sudo ./install.pl and choose option 1, I get this error: CC [M] ...
0
votes
0answers
37 views

Getting eth_ipoib in Ubuntu

I've just discovered an ethernet driver called eth_ipoib which I'd like to experiment with. However, there appears to be no kernel module for it in the mainline. http://lwn.net/Articles/507258/ How ...
4
votes
1answer
698 views

Passing through RDMA network devices to docker containers

I'm wanting to passthrough infiniband to a docker container so that I can run some high performance apps over ipoib and use rdma. Currently, I'm doing this with xen virtual machines. Now I'm looking ...
1
vote
0answers
117 views

Infiniband + KVM

this question is a copy from stackoverflow, as I was told, my question would fit here better. Despite quite a long search for days, I could not find any good information about the following - though ...
0
votes
0answers
106 views

Possible to connect multiple InfiniBand switches?

Is it possible to connect multiple InfiniBand switches? If so, how can this be done? I've attempted to connect the switches directly via a cable, but that is not working, so I assume that I will need ...
0
votes
2answers
365 views

Is ibdump in Mellanox OFED 2 supported in Ubuntu 14.04?

I have Ubuntu 14.04. I have Mellanox Technologies MT27500 Family [ConnectX-3] HCAs. I was able to pass all the tests like ibping and ib_rdma_lat. Now I'm running MPI tests to verify that InfiniBand is ...
1
vote
1answer
183 views

InfiniBand RDMA latency test on Xen's dom0 crashes

The short story: while setting up InfiniBand connection between two servers, I cannot complete the RDMA latency test. It crashes even with breaking the ssh connection. The long story. The first ...
1
vote
0answers
130 views

HP ConnectX cards identifying as ConnectX EN

I bought a lot of four HP branded ConnectX card on Ebay. The cards are : HP InfiniBand 4X DDR ConnectX PCI-E Gen2 Dual-Port HCA 483514-B2 My problem is that two out of four cards identify themselves ...
2
votes
3answers
74 views

Is it possible to get any kind of unique identifier for the Infiniband switch that a particular Linux box is connected to?

I am trying to identify which Infiniband NIC's are connected to which switches, in a manner that is very similar to this question. However, the solution proposed there is not working for Mellanox ...
2
votes
0answers
141 views

Can two infiniband ports be bridged together to make a switch?

I have 3 machines with infiniband cards, two with single ports and one with two ports. I do not own any switches. I have my network setup such that the computer with 2 ports, running Linux, routes ...
4
votes
2answers
778 views

what does `pipe <number>` mean in `ping` output?

While running a test to diagnose some alleged network problem, I got this output from ping: --- r01c01b11n02 ping statistics --- 30000 packets transmitted, 29995 received, 0% packet loss, time 6309ms ...
0
votes
1answer
85 views

How much throughput increase to expect from link aggregation (LACP/802.3ad)?

I identified that data transfer within my cluster will be bottlenecked by the network interconnect (I'm saturating the dual SAS connection to my storage with sequential workloads), and I'm undecided ...
2
votes
0answers
160 views

Infiniband link is at 4X SDR with DDR HCAs; has anyone seen this before?

I've recently upgraded my Infiniband HCAs from SDR to DDR, however upon upgrading, it has not improved the link speed at all. Receiving the following output from 'iblinkinfo': CA: ip-san-1-au3a ...
4
votes
1answer
455 views

iSCSI TGTD over InfiniBand Performance

I have just configured a server-to-server 4x SDR InfiniBand link to increase bandwidth for the backend storage of my virtual machines, specifically a database server. I am using iSCSI with TGTD on ...
2
votes
1answer
740 views

How to configure hosts to show host name in ibnetdiscover (infiniband)?

Windows machines with the OFED stack installed show the host names in ibnetdiscover. Under Ubuntu Linux all I see is "MT25408 ConnectX Mellanox Technologies". Is there a way of configuring the ...
3
votes
1answer
796 views

Is it possible to update Supermicro ConnectX-3 Firmware with newer firmware from Mellanox?

I have a Supermicro server with a X9DRT-HIBFF motherboard installed. It has on-board Infiniband based on the ConnectX-3 chipset from Mellanox. Supermicro supply firmware version 2.10.0700 and ...
3
votes
1answer
2k views

How do I enable SR-IOV for Mellanox ConnectX-3 Ubuntu 12.04.3?

I have a Supermicro 2027TR-HTFRF server that I'm using as a xen server. What I'm wanting to do is get SR-IOV operating for the Connectx-3 card so that I can expose Infiniband to virtual machines. ...
0
votes
1answer
831 views

How do I get IP packats forwarded/routed to/from my Infiniband network

I have two networks. One is a standard Ethernet network running IP. The second network is Infiniband which in addition to some custom protocols can talks IPoIB (IP over Infiniband). The router ...
1
vote
2answers
787 views

Alternatives to bridging infiniband ipoib within xen domains?

I've installed xen-hypervisor-4.1 package onto an Ubuntu 12.04.3 server. It has an infiniband adaptor installed and I run IPoIB with it. What I'm wanting to do is provide the ipoib to one or more ...
1
vote
1answer
1k views

Infiniband port status UP but can't open UMAD port ((null):0)

My system has 2 infiniband devices, one of which has both the ports up. $> ibstatus Infiniband device 'mlx4_0' port 1 status: default gid: fe80:0000:0000:0000:0002:c903:000f:0a9f ...
-1
votes
1answer
119 views

2 servers with infiniband cards and one IB switch i want to use the Ip Over IB

I have 2 super Micro servers with XCP installed i have 2 Infiniband cards on them, i have IB switch also, Now i want to connect Both servers with IP O IB, How can i do that, if i am using Infiniband i ...
2
votes
2answers
2k views

Mellanox dual-port HCA, can ping if ib0 pair connected but not if only ib1 pair connected

I have installed 2 mellanox FDR dual-port ConnectX-3 HCA cards (CX354A), each to a separate machines. The machines are connected directly to each other (switchless configuration). Both ports on the ...
1
vote
2answers
427 views

Infiniband, SRP and failover

I am considering an linux-based infiniband storage setup, but I am tripping over whether or not I can do active/passive or active/active failover with DRBD and SRP targets. I am led to believe that ...
0
votes
0answers
455 views

Setting up computational cluster using infiniband fails

My task: I need to assemble computational cluster, consists of two nodes using infiniband technology. When I try to start mpirun ... for two nodes, the process hangs. If i try to use flag ...
1
vote
3answers
932 views

are DDR infiniband and 10Gbase-CX4 cables the same?

They looks the same. It's SFF-8470 Is it ok to buy a "10Gbase-CX4" cable for an infiniband network ?
10
votes
3answers
2k views

How can I determine which switch the Infiniband subnet manager is running on?

I recently inherited an Infiniband network containing multiple switches, and I know that one of these switches is running the subnet manager. The rest supposedly have that feature turned off, or were ...
0
votes
1answer
401 views

Cannot use SDP in Debian Squeezy

I have severe problems running SDP on Debian Squeeze. I'm using two machines with Mellanox adapters each. My /etc/modules looks like this: mlx4_ib # Mellanox ConnectX cards #ib_mthca # some mellanox ...
5
votes
4answers
3k views

switchless Infiniband between two servers on RHEL 6.3

I have 2 servers running RHEL 6.3 which have 2 port Infiniband cards >lspci | grep -i infini 07:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02) I'm interested in connecting ...
-7
votes
3answers
797 views

infiniband network between 3 servers [closed]

Let's say I have 3 different servers, each one with an infiniband card. Each card has 2 different ports. (I don't know about the model yet) Is it possible to create 3 different networks and to allow ...
2
votes
0answers
206 views

How do I make dnsmasq serve IP addresses via IPoIB?

I have a cluster farm that I'm setting up. The nodes (computers in the farm) are connected via ethernet & IP over Infiniband. I'm needing to netboot the nodes and thought dnsmasq would fit well ...
-4
votes
2answers
138 views

What is the fastest way to connect two machines for a ping-pong test? [closed]

I need to do a latency test between two machines, basically a ping-pong packet test. I wonder what is the fastest connection I can have between those two machines. I imagine my options are: Ethernet ...
1
vote
3answers
729 views

allow infiniband for non root users

I got Infiniband running on RHEL 6.3 [root@master ~]# ibv_devinfo hca_id: mthca0 transport: InfiniBand (0) fw_ver: 4.7.927 node_guid: 0017:08ff:ffd0:6f1c ...
-2
votes
3answers
824 views

Are there HP Infinihost III ex drivers for windows available?

I picked up some infiniband cards off ebay for development/testing purposes. I was hoping that there would be some drivers available under windows 7, but it doesn't seem to be recognised by the OFED ...
0
votes
3answers
574 views

Areca 1880ix RAID hangs

Areca RAID controller ARC-1880ix-12 (firmware 1.50) hangs when on high load. My setup is: Chenbro 3U chassis Intel S5500BC mainboard Xeon 5603 CPU 16GB of RAM 12 Seagate SAS drives ST32000645SS (2 of ...
3
votes
2answers
1k views

View Infiniband routing table generated by OpenSM?

As I understand it, the subnet manager of an Infiniband network calculates the best routes between each pair of nodes on the network and provides these routes to the nodes when they want to ...
1
vote
2answers
890 views

How to correctly set up iWARP? Preferably on loopback

iWARP is a protocol for doing remote direct memory access (RDMA) on top of TCP/IP, so that it can work with Ethernet and other network types as opposed to Infiniband. It works with many of the ...