InfiniBand is a switched fabric computer network communications link used in high-performance computing and enterprise data centers.

learn more… | top users | synonyms

1
vote
0answers
35 views

HP ConnectX cards identifying as ConnectX EN

I bought a lot of four HP branded ConnectX card on Ebay. The cards are : HP InfiniBand 4X DDR ConnectX PCI-E Gen2 Dual-Port HCA 483514-B2 My problem is that two out of four cards identify themselves ...
2
votes
3answers
33 views

Is it possible to get any kind of unique identifier for the Infiniband switch that a particular Linux box is connected to?

I am trying to identify which Infiniband NIC's are connected to which switches, in a manner that is very similar to this question. However, the solution proposed there is not working for Mellanox ...
2
votes
0answers
27 views

Can two infiniband ports be bridged together to make a switch?

I have 3 machines with infiniband cards, two with single ports and one with two ports. I do not own any switches. I have my network setup such that the computer with 2 ports, running Linux, routes ...
1
vote
2answers
106 views

what does `pipe <number>` mean in `ping` output?

While running a test to diagnose some alleged network problem, I got this output from ping: --- r01c01b11n02 ping statistics --- 30000 packets transmitted, 29995 received, 0% packet loss, time 6309ms ...
0
votes
0answers
36 views

How much throughput increase to expect from link aggregation (LACP/802.3ad)?

I identified that data transfer within my cluster will be bottlenecked by the network interconnect (I'm saturating the dual SAS connection to my storage with sequential workloads), and I'm undecided ...
2
votes
0answers
67 views

Infiniband link is at 4X SDR with DDR HCAs; has anyone seen this before?

I've recently upgraded my Infiniband HCAs from SDR to DDR, however upon upgrading, it has not improved the link speed at all. Receiving the following output from 'iblinkinfo': CA: ip-san-1-au3a ...
4
votes
1answer
205 views

iSCSI TGTD over InfiniBand Performance

I have just configured a server-to-server 4x SDR InfiniBand link to increase bandwidth for the backend storage of my virtual machines, specifically a database server. I am using iSCSI with TGTD on ...
2
votes
1answer
205 views

How to configure hosts to show host name in ibnetdiscover (infiniband)?

Windows machines with the OFED stack installed show the host names in ibnetdiscover. Under Ubuntu Linux all I see is "MT25408 ConnectX Mellanox Technologies". Is there a way of configuring the ...
0
votes
0answers
79 views

Infiniband dual-port redundancy

I want to setup redundant infiniband networking. I am going to use servers (Centos 6.4) with dual-port IB adapters (Mellanox Connect-X III series) and two unmanaged IB switches. I know that IB paths ...
3
votes
1answer
328 views

Is it possible to update Supermicro ConnectX-3 Firmware with newer firmware from Mellanox?

I have a Supermicro server with a X9DRT-HIBFF motherboard installed. It has on-board Infiniband based on the ConnectX-3 chipset from Mellanox. Supermicro supply firmware version 2.10.0700 and ...
3
votes
1answer
712 views

How do I enable SR-IOV for Mellanox ConnectX-3 Ubuntu 12.04.3?

I have a Supermicro 2027TR-HTFRF server that I'm using as a xen server. What I'm wanting to do is get SR-IOV operating for the Connectx-3 card so that I can expose Infiniband to virtual machines. ...
0
votes
1answer
305 views

How do I get IP packats forwarded/routed to/from my Infiniband network

I have two networks. One is a standard Ethernet network running IP. The second network is Infiniband which in addition to some custom protocols can talks IPoIB (IP over Infiniband). The router ...
1
vote
2answers
330 views

Alternatives to bridging infiniband ipoib within xen domains?

I've installed xen-hypervisor-4.1 package onto an Ubuntu 12.04.3 server. It has an infiniband adaptor installed and I run IPoIB with it. What I'm wanting to do is provide the ipoib to one or more ...
1
vote
1answer
441 views

Infiniband port status UP but can't open UMAD port ((null):0)

My system has 2 infiniband devices, one of which has both the ports up. $> ibstatus Infiniband device 'mlx4_0' port 1 status: default gid: fe80:0000:0000:0000:0002:c903:000f:0a9f ...
-1
votes
1answer
80 views

2 servers with infiniband cards and one IB switch i want to use the Ip Over IB

I have 2 super Micro servers with XCP installed i have 2 Infiniband cards on them, i have IB switch also, Now i want to connect Both servers with IP O IB, How can i do that, if i am using Infiniband i ...
2
votes
2answers
667 views

Mellanox dual-port HCA, can ping if ib0 pair connected but not if only ib1 pair connected

I have installed 2 mellanox FDR dual-port ConnectX-3 HCA cards (CX354A), each to a separate machines. The machines are connected directly to each other (switchless configuration). Both ports on the ...
0
votes
2answers
221 views

Infiniband, SRP and failover

I am considering an linux-based infiniband storage setup, but I am tripping over whether or not I can do active/passive or active/active failover with DRBD and SRP targets. I am led to believe that ...
0
votes
0answers
193 views

Setting up computational cluster using infiniband fails

My task: I need to assemble computational cluster, consists of two nodes using infiniband technology. When I try to start mpirun ... for two nodes, the process hangs. If i try to use flag ...
0
votes
3answers
378 views

are DDR infiniband and 10Gbase-CX4 cables the same?

They looks the same. It's SFF-8470 Is it ok to buy a "10Gbase-CX4" cable for an infiniband network ?
9
votes
2answers
650 views

How can I determine which switch the Infiniband subnet manager is running on?

I recently inherited an Infiniband network containing multiple switches, and I know that one of these switches is running the subnet manager. The rest supposedly have that feature turned off, or were ...
0
votes
1answer
254 views

Cannot use SDP in Debian Squeezy

I have severe problems running SDP on Debian Squeeze. I'm using two machines with Mellanox adapters each. My /etc/modules looks like this: mlx4_ib # Mellanox ConnectX cards #ib_mthca # some mellanox ...
4
votes
4answers
2k views

switchless Infiniband between two servers on RHEL 6.3

I have 2 servers running RHEL 6.3 which have 2 port Infiniband cards >lspci | grep -i infini 07:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02) I'm interested in connecting ...
-8
votes
3answers
446 views

infiniband network between 3 servers [closed]

Let's say I have 3 different servers, each one with an infiniband card. Each card has 2 different ports. (I don't know about the model yet) Is it possible to create 3 different networks and to allow ...
2
votes
0answers
134 views

How do I make dnsmasq serve IP addresses via IPoIB?

I have a cluster farm that I'm setting up. The nodes (computers in the farm) are connected via ethernet & IP over Infiniband. I'm needing to netboot the nodes and thought dnsmasq would fit well ...
-4
votes
2answers
124 views

What is the fastest way to connect two machines for a ping-pong test? [closed]

I need to do a latency test between two machines, basically a ping-pong packet test. I wonder what is the fastest connection I can have between those two machines. I imagine my options are: Ethernet ...
1
vote
3answers
491 views

allow infiniband for non root users

I got Infiniband running on RHEL 6.3 [root@master ~]# ibv_devinfo hca_id: mthca0 transport: InfiniBand (0) fw_ver: 4.7.927 node_guid: 0017:08ff:ffd0:6f1c ...
-2
votes
3answers
551 views

Are there HP Infinihost III ex drivers for windows available?

I picked up some infiniband cards off ebay for development/testing purposes. I was hoping that there would be some drivers available under windows 7, but it doesn't seem to be recognised by the OFED ...
0
votes
3answers
352 views

Areca 1880ix RAID hangs

Areca RAID controller ARC-1880ix-12 (firmware 1.50) hangs when on high load. My setup is: Chenbro 3U chassis Intel S5500BC mainboard Xeon 5603 CPU 16GB of RAM 12 Seagate SAS drives ST32000645SS (2 of ...
3
votes
2answers
865 views

View Infiniband routing table generated by OpenSM?

As I understand it, the subnet manager of an Infiniband network calculates the best routes between each pair of nodes on the network and provides these routes to the nodes when they want to ...
1
vote
2answers
584 views

How to correctly set up iWARP? Preferably on loopback

iWARP is a protocol for doing remote direct memory access (RDMA) on top of TCP/IP, so that it can work with Ethernet and other network types as opposed to Infiniband. It works with many of the ...
-2
votes
1answer
160 views

What is Infiniband? [closed]

What is Infiniband? What I have found is its a fabric switch network medium used for high performance computing. But more specificly what is it? Is it networking medium in the same way as cat5e is to ...
3
votes
4answers
5k views

Set up simple Infiniband Block Storage (SRP or iSER)

I'm trying to figure out how to set up a simple storage system which exports block storage over Infiniband, using either SRP or iSER. I'm very early in the process, and at the moment I'm basically ...
2
votes
1answer
332 views

back to back infiniband

I've been given two hpc cluster nodes, each with an infiniband hca. Is it possible to connect them back-to-back, without using a switch ?
1
vote
2answers
870 views

Virtual infiniband on kvm/qemu or another open source platform?

Is there any way to set up a virtual infiniband network within kvm/qemu? If not, is there a viable open source alternative which would allow virtual infiniband networking? (Note that I'm not looking ...
2
votes
2answers
930 views

How to set up an Infiniband SRP Target?

I want to set up a server as a SCSI Target. How good are the open source implementations? What is the tutorial one? How is performance and reliability? How dependent is the OS? How much is ...
1
vote
2answers
278 views

How to simulate high video payload to benchmark an SSD?

I'm trying to evaluate some SSDs for a video on-demand use-case. We've done some benchmarking tests on them, but we'd like to get an idea of the number of video streams that they can support with a ...
1
vote
2answers
248 views

Infiniband UIO vs Infiniband HCA

What is the fundamental difference between: Infiniband Universal I/O Card (e.g. Supermicro AOC-UINF-M2) and Infiniband Host Channel Adapter (e.g. Qlogic QLE7240-CK) Can't both of those do do ...
1
vote
1answer
435 views

NFS performance trouble: with infiniband gets stacked with many users?

I have a RAID system on A server and similar configuration on B server-both with Scientific Linux 5.5. They are connected with infiniband and RAIDS from A are mounted with NFS to ServerB. Now when the ...
1
vote
1answer
451 views

Infiniband QDR to DDR

I have a bunch of computer with onboard Mellanox ConnectX-2 QDR Infiniband 40Gbps Controllers. They have QSFP connectors. I have a switch with 24 4x CX4 DDR connectors. If I buy QSFP to CX4 cables, ...
3
votes
8answers
2k views

InfiniBand Storage

I'm contemplating the next restructuring of my medium-size storage. It's currently about 30TB, shared via AoE. My main options are: Keep as is. It can still grow for a while. Go iSCSI. currently ...
4
votes
5answers
1k views

Infiniband on Linux?

I will build computing cluster (about 64 nodes) and I'm looking for some informations. I want to connect them with Infiniband and install Linux. I tried to google it a little bit, but I couldn't find ...