Questions tagged [mellanox]

Mellanox Technologies Ltd. is an Israeli and American multinational supplier of computer networking products using InfiniBand and Ethernet technology

Filter by
Sorted by
Tagged with
0 votes
1 answer
99 views

Only getting 25Gb/s instead of 100Gb/s from a Mellanox switch running Cumulus Linux

I have a Mellanox 100gb/s switch (running Cumulus Linux 4.1) that I use for connecting multiple servers, each with a Mellanox ConnectX 5 100gb/s card. These servers connect to the switch via a DAC ...
user avatar
  • 485
0 votes
1 answer
144 views

K8s nodeLocalDns pod times out connecting to coreDns after upgrading base os to ubuntu20.04 ConnectX-4 card

Team, I have Mellanox Nic ConnectX-4 on a k8s worker node and it hosts a nodeLocal dns pod on it. The nodeLocalDns pod is timing out when trying to connect to coreDns service on k8s cluster. Same ...
user avatar
  • 109
0 votes
1 answer
253 views

X710 and mlx5_core together on same server

I have a amd server with centos 7.9 I am trying to using the the kernel driver for 2 cards: Ethernet Controller X710 for 10GbE SFP+ 1572 --> i40e MT27800 Family [ConnectX-5] 1017' --> mlx5_core ...
user avatar
  • 103
0 votes
0 answers
136 views

Infiniband adapter down

edit: On CentOS 8.5, tried with Mellanox driver 4.9-4.1.7.0 (legacy) and 5.5-1.0.3.2: I am not able to get my Infiniband adapter working. The output of ibstat states that it is down: CA 'mlx5_0' ...
user avatar
  • 31
2 votes
0 answers
94 views

Infiniband fabric with 3 nodes - newbie

I am trying to connect 3 HP z840 workstations using: Mellanox ConnectX-3 VPI 40 / 56GbE Dual-Port QSFP Adapter MCX354A-FCBT Mellanox SX6005 12-port Non-blocking Unmanaged 56Gb/s Description of ...
user avatar
  • 121
1 vote
1 answer
58 views

Infiniband OpenSM N-to-N port routing configuration

I have 10 servers with two CPUs each and one Mellanox 100G Infiniband NIC per CPU. Each NIC is connected to a single Mellanox 36 port 100G IB switch. My RDMA application runs as one process per NUMA ...
user avatar
1 vote
2 answers
328 views

Infiniband drivers : OFED or distro included?

I'm setting up a Linux cluster with infiniband network, and I'm quite a newby in infiniband wolrd, any advice is more than welcome ! We are currently using Mellanox OFED drivers, but our infiniband ...
user avatar
3 votes
0 answers
984 views

Why isn't SMB Direct working over RDMA?

In a Windows lab environment, I'm trying to test performance of SMB Direct using Mellanox RDMA adapters with global pause. I wanted to test first with Global Pause for simplicity before switching to ...
user avatar
  • 200
4 votes
0 answers
1k views

How to make systemd monitor processes started using a shell script in the service file?

I have written a service file in Linux that calls a script which starts multiple processes with one as primary and others as secondary. SERVICE FILE: [Unit] Description=TEST service Requires=abc....
user avatar
  • 41
1 vote
1 answer
117 views

A way to configure DMZ zone on Ethernet Mellanox Switch

Can't find any information is it possible at all and how to configure DMZ zone on Mellanox MSN2100 switch. If I setup ACL disabling traffic from DMZ to LAN, I can't receive response to requests made ...
user avatar
  • 113
0 votes
1 answer
640 views

Infiniband P_Keys and the Linux kernel

I'm trying to configure Infiniband partitions between two Debian based Linux hosts running the 4.15 kernel and A Mellanox SX6036 switch. I've setup a "DMZ" partition on the swich using a PKey of ...
user avatar
  • 3,434
1 vote
0 answers
329 views

Missing Azure accelerated networking interfaces while VMs initializes

I'm running my application on top of Azure Ubuntu 16.04 VMs (Wrapped as a VHD image). In several cases I have observed missing SRIOV associated VFs interfaces, while VMs were starting. Need to mention ...
user avatar
  • 11
2 votes
1 answer
1k views

Beegfs-client does not work with mellanox OFED driver

I am trying to configure and test beegfs with RDMA as explained in: https://community.mellanox.com/s/article/howto-configure-and-test-beegfs-with-rdma?t=1570613300675 My test configuration: OS: ...
user avatar
  • 21
0 votes
1 answer
116 views

Mellanox adapter on GCP instance

I'm trying to install MOFED on an instance (preferably with a GPU) and was wondering if anyone knows if a Mellanox adapter is available at any Google Cloud Platform VM. The VM instances that I've been ...
user avatar
0 votes
1 answer
335 views

Connecting Intel X722 SFP+ to Mellanox SX1012 QSFP via 4-split breakout cable using ubuntu

I have two servers running Ubuntu 18.04, each equipped with two SFP+ interfaces, each showing up in lshw like the following. *-network:1 description: Ethernet interface product: ...
user avatar
0 votes
1 answer
525 views

Can not setup Mellanox connectx-2 in 40g speed

I have Mellanox connectX-2 network card (MT26428) and I installed MLNX_OFED_LINUX-3.4-1.0.0.0-ubuntu16.04-x86_64 driver from Mellanox repository but I'm wondering this equipment setup 20G at maximum ...
user avatar
  • 101
9 votes
3 answers
1k views

QSFP28 - is there something like a crossover cable?

Can not find anything to that ;) Building a Hyperconvergend Cluster as development system. This is for us state of the art. Got 2 machines for a start, with a switch ordered to come in about 4 weeks....
user avatar
  • 50.8k
0 votes
1 answer
123 views

Managed switch for Mellanox MCX354A-FCBT cards?

I've inherited two Mellanox Infiniband environments. One on a LAN, and the other in a DMZ. Both IB networks have their own SX6005 unmanaged switches and a couple small desktop computers running the ...
user avatar
  • 3,434
1 vote
1 answer
616 views

Infiniband configured and connected but indicator lights unlit on HBAs and no functionality in OS

I have installed two dual-port FDR Infiniband VPI HBAs, one in each of two servers running CentOS 6.9, server1>lspci 03:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3] ...
user avatar
  • 185
2 votes
2 answers
558 views

Dual-port RDMA Throughput Issue

I have two nodes connected with dual-port Mellanox Connext-X3 VPI HCAs via an IB switch. The nodes are two socket machines with Hasswell CPUs and 2 16GB DIMMs per each socket (totaling 64GB). ...
user avatar
1 vote
2 answers
603 views

In Infiniband can I use the same port for IPoIB traffic as well as RDMA traffic?

Good day, and my disclaimer that I am no IB expert and have not worked on IB directly yet. My question seems a trivial one, yet I could not find anything in google that answers it. Question: In ...
user avatar
  • 21
-1 votes
1 answer
159 views

SFP+ Connector Won't Latch

We're trying to connect a Cisco Meraki MS225-48 switch to a Mellanox SN2100 switch using this breakout cable. However, the SFP+ end of the cable does not physically latch into the Meraki SFP+ ports (i....
user avatar
0 votes
1 answer
909 views

Troubles with ibverbs in OpenSUSE Leap 42.2 for Mellanox ConnectX-3 HCA

SOLVED I got troubles setting up infiniband software from OpenSUSE repository. The same card works fine with SLES12 + Mellanox OFED stack. Installing everything containing 'infiniband' from yast I ...
user avatar
  • 1
1 vote
0 answers
1k views

Mellanox MT27500 Link to Port 1 is Down

I have Mellanox MT27500 dual port card which is installed on HP SL250s. Ubuntu 14.04.5 with 3.13.0-24-generic kernel is running on the server. Both port1 type and port 2 type are defined as eth. I ...
user avatar
  • 11
2 votes
2 answers
5k views

Installing Mellanox ConnectX-2 10gbe PCIe Card on Centos 6

I just installed a Mellanox ConnectX-2 10gbe PCIe x8 Card into my server running CentOS 6.7. On boot, dmesg shows the mlx4_core driver being loaded automatically however I see no eth1 device ...
user avatar
2 votes
4 answers
1k views

set the Ca name on unmanaged mellanox switch

How can I set the Hostname/Description of a Mellanox/Infiniband unmanaged switch? I would like a way to abstractly distinguish quickly which switches are which when doing 'ibswitches' or '...
user avatar
  • 23
0 votes
1 answer
2k views

VMware infiniband configuration

I have a server which is intended to be a vmware host. This server has two Infiniband Mellanox ConnectX-2 dual-port cards. What I want to do is to make the following setup: The first card should work ...
user avatar
0 votes
2 answers
969 views

Is ibdump in Mellanox OFED 2 supported in Ubuntu 14.04?

I have Ubuntu 14.04. I have Mellanox Technologies MT27500 Family [ConnectX-3] HCAs. I was able to pass all the tests like ibping and ib_rdma_lat. Now I'm running MPI tests to verify that InfiniBand is ...
user avatar
  • 167
1 vote
1 answer
406 views

InfiniBand RDMA latency test on Xen's dom0 crashes

The short story: while setting up InfiniBand connection between two servers, I cannot complete the RDMA latency test. It crashes even with breaking the ssh connection. The long story. The first ...
user avatar
  • 167
3 votes
1 answer
3k views

Is it possible to update Supermicro ConnectX-3 Firmware with newer firmware from Mellanox?

I have a Supermicro server with a X9DRT-HIBFF motherboard installed. It has on-board Infiniband based on the ConnectX-3 chipset from Mellanox. Supermicro supply firmware version 2.10.0700 and ...
user avatar
  • 14.1k
4 votes
1 answer
4k views

How do I enable SR-IOV for Mellanox ConnectX-3 Ubuntu 12.04.3?

I have a Supermicro 2027TR-HTFRF server that I'm using as a xen server. What I'm wanting to do is get SR-IOV operating for the Connectx-3 card so that I can expose Infiniband to virtual machines. ...
user avatar
  • 14.1k
0 votes
1 answer
604 views

Cannot use SDP in Debian Squeezy

I have severe problems running SDP on Debian Squeeze. I'm using two machines with Mellanox adapters each. My /etc/modules looks like this: mlx4_ib # Mellanox ConnectX cards #ib_mthca # some mellanox ...
user avatar
  • 387
1 vote
0 answers
1k views

openstack, bridging, netfilter and dnat

In a recent upgrade (from Openstack Diablo on Ubuntu Lucid to Openstack Essex on Ubuntu Precise), we found that DNS packets were frequently (almost always) dropped on the bridge interface (br100). ...
user avatar
  • 6,613