Questions tagged [hpc]
High Performance Computing encompasses using "supercomputers" with high numbers of CPUs, large parallel storage systems and advanced networks to perform time-consuming calculations. Parallel algorithms and parallelization of storage are essential to this field, as well as issues with complex, fast networking fabrics such as Infiniband.
116
questions
1
vote
1
answer
136
views
openLDAP works for v2.4 but v2.6 returns: Invalid syntax (21) additional info: objectClass: value #0 invalid per syntax
I have been facing this problem for a few days by now, and I would like to preface this by stating that this is the first time I have ever used ldap. So after debugging a bit around the error I have ...
0
votes
0
answers
28
views
Issues Connecting to HPC Head Node from Non-Domain-Joined Machines - Help Needed!
Hello fellow ServerFaultee's
I'm encountering a challenge with my HPC cluster setup. My main hurdle is connecting to the HPC head node from computers that are not domain-joined, specifically when ...
2
votes
2
answers
861
views
Install software on Ubuntu using apt without root?
I’ve got accounts on many HPC clusters. The machines have a minimal install, and the admins won’t add much else. I need to install lots of typical software. Normally I’d do this with apt, but of ...
0
votes
0
answers
192
views
HPC node, Infiniband is DOWN
I have an HPC with 17 nodes running CentOS 7 and a dedicated Mellanox SX6036 Infiniband switch, each node has an Infiniband FDR interface.
Recently one node started giving errors and a quick look ...
0
votes
1
answer
69
views
Why does the login node connect to external networks but allocated compute node fail in Slurm-GCP?
I've noticed that connecting to the internet from the allocated compute node via Slurm-GCP keeps failing. For example, using wget from the login node works successfully:
[me@gcp-login0 ~]$ wget https:/...
0
votes
1
answer
175
views
Linux missing lvm
hello so i have a ubuntu hpc cluster and i got a problem with storage
whenever i try to access the storage from my compute nodes i cant i keep getting this error
mount:mounting 192.168.100.211:/cm/...
-1
votes
1
answer
326
views
Speeding up SAS data rate from 12Gbps
I'm curious about SAS data transfer speed.
Maximum is 12Gbps in the whole bus (not per drive) as far as I understand, but I have a scenario where I would like to have a faster data rate (hopefully ...
0
votes
0
answers
387
views
Not able to ssh into 2 Compute Nodes on HPE Cluster
I recently added two new Compute Nodes on HPE CLuster , But surprisingly, I am Unable to ssh into the new Compute Nodes from the Head Node .
[Unable to SSH to new Compute Nodes][1]
(base) [root@hn001 ~...
1
vote
1
answer
230
views
HPC master node no infiniband network influence on compute nodes - Slurm management
I'm writing because I'm facing an issue that I cannot solve trying to configure a cluster with a master node ( or Frontend node ) as a Virtual machine managing nodes with infiniband network.
I use ...
0
votes
1
answer
39
views
Single SSH login for multilpe machines?
I have a number of physical (desktop) machines running at the office as part of a new network to handle processing & serving Open Source data; some of these machines also house VMs.
At the moment, ...
2
votes
0
answers
390
views
Infiniband fabric with 3 nodes - newbie
I am trying to connect 3 HP z840 workstations using:
Mellanox ConnectX-3 VPI 40 / 56GbE Dual-Port QSFP Adapter MCX354A-FCBT Mellanox SX6005 12-port Non-blocking Unmanaged 56Gb/s
Description of ...
2
votes
1
answer
645
views
How can I set up interactive-job-only or batch-job-only partition on a SLURM cluster?
I'm managing a PBS/torque HPC cluster, and now I'm setting up another cluster with SLURM. On the PBS cluster, I can set a queue to accept only interactive jobs by qmgr -c "set queue interactive_q ...
0
votes
0
answers
829
views
Setting up slurm on a cluster
My IT admin has setup a cluster with 3 nodes, which is administered via Windows server. VMs are hosted via Hyper-V, including an Ubuntu VM to which a substantial portion of the cluster's resources ...
1
vote
1
answer
339
views
Why can't the GPUs communicate in a multi-GPU server?
This is a Dell PowerEdge r750xa server with 4 Nvidia A40 GPUs, intended for AI applications. While the GPUs work well individually, multi-GPU training jobs or indeed any multi-GPU computational ...
1
vote
1
answer
231
views
Infiniband OpenSM N-to-N port routing configuration
I have 10 servers with two CPUs each and one Mellanox 100G Infiniband NIC per CPU. Each NIC is connected to a single Mellanox 36 port 100G IB switch.
My RDMA application runs as one process per NUMA ...
2
votes
0
answers
41
views
Lustre glitch: latency of minutes
Using a HPC lustre filesystem, we occasionally experience glitchiness where even simply opening a terminal and typing "ls" can take minutes to return. That is, any process that involves the ...
1
vote
0
answers
36
views
Why a non-root installation can work across the whole cluster?
I recently installed anaconda (which includes a new python3) locally in my account folder on a cluster with a dozen of nodes (each node with several cores). I use it to install some package P that is ...
0
votes
1
answer
83
views
Changing the subnet on which my beegfs cluster operates
I've added some fiber channels between the machines that constitutes my BeegFS cluster in an effort to increase throughput. However, I have to leave the old coppy ethernet in place with its addressing ...
1
vote
1
answer
803
views
Latency of memory accesses via interconnectors
I'm trying to compare latencies of different node interconnects for a cluster. The goal is to minimize the memory access latency.
I have obtained some benchmarks regarding one of the hardware ...
2
votes
0
answers
403
views
Current single system image solutions
I'm designing a cluster for a small research institute. Since our computations require a large amount of memory, I'm looking for a solution that will allow our applications access to the whole memory ...
1
vote
1
answer
471
views
ssh port forwarding (tunneling in HPC)
I have an application server that runs on a compute node. The server opens a port (9000) and I then run a command for tunneling between my local machine and the server:
ssh -N -f -L 9000:compute-node:...
2
votes
1
answer
519
views
HPC cluster master node as virtual machine
For a given small HPC cluster (~16 nodes) a master node is used as a front-end for users to login and interact with SLURM, and not as a computing node. The master node is currently a bare-metal server....
0
votes
1
answer
824
views
OpenLDAP implementation allows only root user to set passwords of accounts
I'm working application that requires the use of AWS ParallelCluster assets for some high performance processing. After the initial setup, we need to be able to add/remove user accounts and I am ...
2
votes
1
answer
2k
views
Considerations using consumer class (high-end) GPU in server?
Motivation:
First of all, even if I have some knowledge of computer science, software development and server Linux administration, I never looked into a server hardware and I am a total "newbie&...
2
votes
2
answers
1k
views
Infiniband drivers : OFED or distro included?
I'm setting up a Linux cluster with infiniband network, and I'm quite a newby in infiniband wolrd, any advice is more than welcome !
We are currently using Mellanox OFED drivers, but our infiniband ...
2
votes
1
answer
560
views
SLURM with "partial" head node
I am trying to install SLURM with NFS on a small ubuntu 18.04 HPC cluster, in a typical fashion, e.g. configure controller (slurmctld) and clients (slurmd) and shared directory, etc. What I am curious ...
3
votes
0
answers
73
views
Bad multicore performance on DL360 Gen 10 with 2xXeon 6154
I have some issue on the multi-core performance of some server. The server are HPE DL360 Gen10, mounting 2x Xeon Gold 6154 (18 cores). When i refer to the performances they are slower then some older ...
3
votes
2
answers
763
views
What is the overhead of ZFS RAIDz1/2 in HPC SSD Environment?
Example hardware / host:
Modern 64 Core CPU, 128GB Memory
8 x Micron Pro 15.36TB u.2 SSDs
SSDs connected by dedicated Oculink per device (no backplane or PCIe sharing)
Ubuntu 20.04
Use case:
A ...
1
vote
1
answer
2k
views
Wrong LDAP user ID is mapped into Slurm account management service
I configured a Slurm head node as follows:
sssd to contact openLDAP
slurmctld/slurmdbd/slurmd/munged to act as the Slurm controller and compute node
...where ray.williams is an LDAP user. Its UID ...
2
votes
1
answer
441
views
HTCondor high availability
I am currently trying to make the job queue and submission mechanism of a local, isolated HTCondor cluster highly available. The cluster consists of 2 master servers (previously 1) and several compute ...
-1
votes
1
answer
379
views
Using HPC managers like Slurm on multiple servers in LAN [closed]
I have access to a group of servers connected with a 1Gb LAN, and each of them has 40+ cores and Ubuntu OS. They all have a common NAS. I installed SLURM on a few of them and configured it so that ...
0
votes
0
answers
100
views
What can be a reason for different clock speeds between sockets on 2 x Xeon Scalable 6148?
I have server with dual Xeon Scalable 6148 CPUs running HPC application.
Base clock: 2.4GHz
All core Turbo: 3.1 GHz
Some processing threads are not scaling well and are sensitive to cpu clock. I ...
1
vote
1
answer
703
views
Single-node SLURM server: restrict interactive CPU usage
I have SLURM setup on a single node, which is also a 'login node'. I would like to restrict interactive CPU usage, e.g. outside the scheduling system.
I found the following article which suggests to ...
0
votes
1
answer
531
views
Ubuntu server vs Ubuntu desktop for Beowulf cluster
I want to create a beowulf cluster using Ubuntu 18. Looking at some guides they all seem to use ubuntu server for this an my question is why? Is it not possible to use ubuntu desktop for the client ...
-1
votes
1
answer
120
views
How is it that Summit at Oak Ridge National Lab has 2,414,592 cores? [closed]
Top500 says that Summit has 2,414,592 cores: https://www.top500.org/system/179397.
But they have 4608 nodes, 9216 chips (each node has 2 chips), and 22 cores per chip. This is 202,752 cores. Where ...
0
votes
1
answer
1k
views
What does "CPU Minutes" mean exactly?
I'm actually trying to report cluster utilization in Slurm but i don't understand the metric CPU Minutes.
[root@XXXX]# sreport cluster Utilization Start=2018-12-01 End=2018-12-31
---------------------...
0
votes
1
answer
114
views
Exascale Power Consumption
I have read a lot of articles about exascale and found out that it may consumes approximately 20MW power envelope. Is it a daily basis or a yearly basis or every second?
Please enlighten me.
Here are ...
0
votes
1
answer
1k
views
Configure Singularity to do headless rendering / use OpenGL / glxgears / glxinfo
I want to do headless rendering on a server where I do not have root permissions. Therefore, I created a Singularity container like this:
Bootstrap: docker
From: nvidia/cuda:9.0-runtime-ubuntu16.04
%...
3
votes
1
answer
3k
views
Intel Xeon 6134 + One DIMM per channel or two DIMMs per channel for maximum memory bandwidth?
I'm unable to find this critical piece of information in spec sheets. Appreciate any insight.
We're purchasing servers for HPC work with intel Xeon Gold 6134 (Skylake) cpus
I want maximum memory ...
3
votes
1
answer
142
views
Micosoft HPC Pack 2012 R2 does not run with Network Direct after joining new domain
I am working with a 13 computer cluster, running on Windows Server 2012 R2, using MS HPC Pack 2012 R2. The headnode is working properly. The servers are connected to the corporate network via IPv4 on ...
2
votes
1
answer
80
views
Are processors more efficient at lower temperatures
I couldn't find a stackexchange site that's more suited for this question, I apologize. This doesn't necessarily have a lot to do with servers and stability....
I do not have problems with stability ...
0
votes
0
answers
239
views
How to handle mpi head node failure?
There is app which starting with mpirun. If compute node fail then all processes crush, but if only head node fail(for example reboot) then processes will stuck on compute nodes. How to get rid of ...
3
votes
3
answers
1k
views
Randomize Slurm Node Allocation
Has anyone had luck randomizing Slurm node allocations? We have a small cluster of 12 nodes that could be used by anywhere from 1-8 people at a time with jobs of various size/length. When testing our ...
3
votes
1
answer
678
views
MAAS for diskless computational hpc cluster
I'm considering to use MAAS to deploy OS for a computational cluster. All nodes are diskless. Only head node and (probably) MAAS rack controller will have hard drives. It seems MAAS have to finish a ...
0
votes
0
answers
23
views
Ideal configuration for a head node? [duplicate]
Which hardware should I concentrate on, when assembling a head node for an HPC cluster? The main task for the head node is to relay instructions to the compute nodes which will be running artificial ...
1
vote
1
answer
723
views
ifconfig apparently showing wrong RX/TX values for InfiniBand HCA
Recently, I executed a watch -n 1 ipconfig on one of our Linux cluster computing nodes while it was running a 48-process MPI run, disributed over several nodes.
Oddly, while Ethernet packets seem to ...
1
vote
2
answers
345
views
Containers for HPC batch processing
We are facing the problem that a lot of people want to run different scientific software on our high performance computing cluster.
Every user requires a different set of libraries and library ...
-3
votes
1
answer
254
views
Solutions for monetizing excess CPU cycles [closed]
My company has a big (relatively) computer farm, say, 100 physical servers (dual CPU hexacore e5 xeons with 160 Gb RAM) leased from some hardware provider (say Leaseweb or OVM) on monthly basis, means,...
2
votes
1
answer
159
views
How many infiniband adapters should be used in multi socket servers?
Should dual socket motherboards have an infinity band adapter for each CPU?
That is, should there be two infiniband band adapters, one in each CPU's PCIe slot.
Would this eliminate the signal going ...
0
votes
1
answer
198
views
SSH vs qlogin to use all processors of a computing node
I have a SGE cluster consisting of four computing nodes, each with 20 processors. I do not mind to give one particular user the full capabilities of one specific node, i.e. I do not mind he/she uses ...