High Performance Computing encompasses using "supercomputers" with high numbers of CPUs, large storage systems and advanced networks to do time-consuming calculations. Parallel algorithms and parallelization of storage are essential to this field, as well as issues with complex, fast networks.

learn more… | top users | synonyms

8
votes
6answers
2k views

How to allow users to transfer files to other users on linux

We have an environment of a few thousand users running applications on about 40 clusters ranging in size from 20 compute nodes to 98,000 compute nodes. Users on these systems generate massive files ...
6
votes
4answers
953 views

using i7 “gamer” cpu in a HPC cluster

I'm running WRF weather model. That's a ram intensive, highly parallel application. I need to build a HPC cluster for that. I use 10GB infiniband interconnect. WRF doesn't depends of core count, but ...
6
votes
1answer
997 views

XFS Adding Quotas - Skip Quota check on first mount/boot

We run a 14TB XFS fileserver on our cluster and want to add quota support. This is running 3.9.2-1.el6.elrepo.x86_64 kernel under CentOS 6.3 (Final). The issue is when we unmount the XFS RAID, and ...
4
votes
1answer
426 views

Why are most supercomputer using linux? [closed]

Referring to this BBC Article: Supercomputing superpowers Almost all the supercomputers are using Linux as operating system. Why is Linux so popular?
4
votes
3answers
510 views

Is this a HPC or HA mySQL cluster?

Can someone tell me if this is a High Performance Compute or High Available mySQL cluster? There is a picture of the setup. This is part of the config.ini they talk about [ndbd default] ...
3
votes
4answers
7k views

Set up simple Infiniband Block Storage (SRP or iSER)

I'm trying to figure out how to set up a simple storage system which exports block storage over Infiniband, using either SRP or iSER. I'm very early in the process, and at the moment I'm basically ...
3
votes
1answer
6k views

Display or install Windows product key on command line

We are currently in the process of setting up several Windows HPC clusters using Windows 2008 HPC R2. We would like to be able to perform Windows licensing commands across the clusters via the ...
3
votes
2answers
1k views

View Infiniband routing table generated by OpenSM?

As I understand it, the subnet manager of an Infiniband network calculates the best routes between each pair of nodes on the network and provides these routes to the nodes when they want to ...
3
votes
1answer
481 views

What applications can be used in a Red Hat/CentOS cluster?

When I look at the Red Hat cluster manuals 1 2, they only explain how to install it but not what applications can use it. I am new to clusters, so I don't know these things =) Let's say I want to 3 ...
3
votes
1answer
302 views

Which is the fastest way to move 1Petabyte from one storage to a new one?

First of all, thanks for reading, and sorry for asking something related to my job. I understand that this is something that I should solve by myself but as you will see its something a bit difficult. ...
2
votes
2answers
2k views

Alternative to ScaleMP?

Anyone know of an alternative to ScaleMP? They let several x86 boxes boot as one large box. Theoretically AMD's hypertransport should allow the same thing. Any other companies or OSS projects doing ...
2
votes
1answer
258 views

What makes Lustre faster and more scalable than NFS?

I have read in various places (e.g. here and here) that NFS' I/O performance does not scale, while Lustre's does, and that Lustre can deliver better I/O rates in general. There seem to be various ...
2
votes
5answers
534 views

setting up a cluster

I have 5 PC(s) connected over a LAN through a switch. I want to connect them to form a HPC cluster. The OS may be any Linux version (currently I have installed Ubuntu 8.10, 9.10 and Fedora 10) Purpose ...
2
votes
1answer
99 views

Microsoft HPC cluster - AD or AD LDS?

The instructions for deploying an HPC Cluster (e.g. step 1.5 on this page in TechNet) are very clear that HPC cluster nodes "must be members of an Active Directory domain". Does the Active Directory ...
2
votes
1answer
370 views

Does Xeon Phi works with i7 CPUs

Does Xeon Phi coprossesors works with i7 CPUs ? It's advertised for use with Xeons, but for my app (WRF), i7-3930k performs better and is 3 times cheaper than high grade xeons. So I wonder if I could ...
2
votes
2answers
103 views

Simulate HPC application data to test WAN filesystem performance over a large link

So here is the setup: we've got temporary access to a very large TCP WAN connection and we want to use this pipe to do WAN filesystem testing. We would like to generate massive amounts of data on ...
2
votes
1answer
375 views

Management of available file descriptors within a Hadoop cluster

I'm currently in charge of a rapidly-growing Hadoop cluster for my employer, currently built upon release 0.21.0 with CentOS as the OS for each worker and master node. I've worked through most of the ...
2
votes
1answer
73 views

Is Ondemand Governor enabled in current HPC clusters?

Will enabling Ondemand GOvernor on HPC cluster help save power ? Are sleep states (C-states) enabled in HPC platforms ? If not, what is the reason behind this ?
2
votes
1answer
106 views

Building Windows server cluster hpc

I'm trying to set up a Windows Server cluster with a head node and 2x compute nodes. So far, i've managed to install : (thanks to this tutorial http://msdn.microsoft.com/en-us/library/jj884142.aspx) ...
1
vote
3answers
169 views

Appropriate network file system for huge (5+ Gb) files

I've got a number of servers used for HPC / cluster computing and I noticed that, given the fact that part of the computations they run use huge files over NFS, this causes significant bottlenecks. ...
1
vote
2answers
43 views

hardware support for parallel file system

I am looking forward to use parallel file system using MPI in linux cluster. I am wondering if parallel file systems like lustre/Parallel Virtual File System require special hardware support(special ...
1
vote
2answers
355 views

Task spooler for computing server on Debian

Recently our university has bought an computing server with one multi-core Xeon and 4 powerfull GeForce videocard for lessons on discipline "High perfomance computing with CUDA". There is Debian ...
1
vote
4answers
204 views

high level server design/programming question [closed]

I am interested in designing a simple lan based server which accepts and services a limited number of connections ( < 25) from within the lan at any time. The server generates images dynamically ...
1
vote
1answer
149 views

Which compute node did a Sun Grid Engine job execute on?

What is the easiest way to determine which node a compute job was executed on, using Sun Grid Engine? qstat seems to only list running/queued jobs
1
vote
1answer
36 views

Parallel filesystem which schedules simultaneous file requests to mutually exclusive sets of OSSs

My environment is RHEL based, interconnect is infiniband. I have some experience with Lustre. What i want to know is: Is there a parallel file-system, where if simultaneous write request arrive, ...
1
vote
1answer
104 views

Windows VMs on ScaleMP cluster?

I was wondering whether the following stack would work, and if so then how well, and what sort of problems I might expect to encounter when setting it up? Hardware layer - lots of cheap servers SMP ...
1
vote
1answer
51 views

HPC OSS Node issue with unreadable local hdd error

We have a HPC setup with four OSS server(OSS1 to OSS4) and two MDS Nodes(MDS1 to MDS2) It has been running till yesterday without any problem. Today morning i found that OSS4 is in shutdown ...
1
vote
1answer
652 views

Torque and maui node status

I am new for torque and maui. I was checking for node state to looking for which nodes are free and which nodes are in use. For torque one command is pbsnodes. Which gives status and other info ...
1
vote
1answer
366 views

Optimizing Linux Compute Cluster

I am setting up a supercomputing Linux cluster at work. We ran the most recent HPCC benchmarks using OpenMPI and GoToBlas2 but got really bad results. When I ran the benchmarks using one process for ...
1
vote
4answers
2k views

A dozen mac mini's vs dell rack server for parallel image processing [closed]

I need to do some large scale image processing in parallel. I was thinking of running a dozen Mac Mini's in parallel to do the data processing. I need to run Microsoft Windows on the machines so I ...
1
vote
1answer
231 views

What cluster management software does there exist for Windows, except for MS HPC Server?

Do there exist any open-source solutions? Perhaps, cheaper commercial ones? I'm surprised that googling does not yield anything meaningful for me.
1
vote
1answer
2k views

How to get the best LINPACK result and conquer the Top500?

Given a large Linux HPC cluster with hundreds/thousands of nodes. What are your best practices to get the best possible LINPACK benchmark (HPL) result to submit for the Top500 supercomputer list? To ...
1
vote
2answers
88 views

Windows HPC Server 08 suitability for Matlabs

I want to setup another Hyper-V VM for installing Matlabs/doing some compute-intensive programming using C. I keep thinking that Windows Server HPC 2008 is designed for this sort of work. Would I be ...
1
vote
2answers
334 views

WEB based HPC cluster node management

i am working on my school diploma thesis. The main goal is to create web based application where logged users could see free and busy nodes, turn them on and off, see what process they are running ...
1
vote
2answers
172 views

Security Concerns for High Performance Clusters

This is a VERY open question since this is my first time creating a cluster. I'm just wondering what type of security concerns will there be and how to prevent them. Background information Using SGE ...
1
vote
0answers
150 views

HPC / EC2 - optimizing NFS for relialibility

In AWS-EC2, I've set-up a cluster of linux virtual machines made of an NFS fileserver and many clients. If the number of clients is above ~20, under heavy I/O, I am experiencing loss of file ...
1
vote
1answer
168 views

Open MPI can't launch remote nodes via SSH

I am trying to set up Open MPI between a few machines on out network. Open MPI works fine locally, but I just can't get it to work on a remote node. I can ssh into the remote machine (without ...
1
vote
0answers
58 views

Programs running in an NX session seem to pause when session disconnects

We are currently running an interactive HPC application which presents a graphical interface to the user, attaches to an HPC cluster and allows them to run and observe some computation. The user logs ...
1
vote
2answers
52 views

HPC 2008 Time Share Single Core (Fractional Resource Scheduling)

Windows HPC 2008 appears to be restricted to one task per core. Is there anyway to time share multiple tasks (or jobs) over a single core?
1
vote
1answer
307 views

Can't force HPZ600 workstation into PXE

I've got a HPC 2008 cluster of HP Z600 workstatsions, and though my head node can add them to the cluster (using node.xml files), and can reboot them, when the Z600 powers on the PXE attempt just ...
1
vote
1answer
135 views

Rocksclusters reinstalling nodes partitioning error

I have a HPC based on rocksclusters So when I've added new roll (torque) I send a kickstart command to all nodes to reinstall them. But after loading X installer on nodes all of them showed me an ...
1
vote
2answers
335 views

Windows HPC Server 2008: private network across VMs?

Windows HPC Server 2008 provides the option to automatically deploy OS images to new cluster node, using Windows Deployment Services. However, this requires the HPC cluster to be set up with a ...
0
votes
2answers
224 views

Running ScaleMP on top of OpenStack [closed]

Looking for a feedback if anyone has already played with running ScaleMP linux appliances in OpenStack (KVM)? A short description of the setup (w/ or w/o InfiniBand, total amount of RAM, etc) and its ...
0
votes
2answers
193 views

What is the OSPM Power management feature in current HPC clusters?

What does the operating system do in ordder to manage power in current HPC clusters ? What are the functionalities embedded in current HPC clusters in order to save power
0
votes
1answer
202 views

HPC Cluster (SLURM): recommended ways to set up a secure and stable system

I'm working with a SLURM driven HPC Cluster, containing of 1 control node and 34 computation nodes and since the current system is not exactly very stable I'm looking for guidelines or best practices ...
0
votes
1answer
34 views

Running jobs in a HPC cluster [closed]

I'm quite new to HPC environment. Is there any difference in running a job on a node utilizing 8 cores and running the same job on 8 nodes utilizing I core in terms of performance or walltime used. ...
0
votes
1answer
45 views

How do we configure Lustre to block client requests when under load, rather than failing?

We are using Lustre in a cluster with approximately 200TB of storage, 12 Object Storage Targets (that connect to a DDN storage system using QDR Infiniband), and roughly 160 quad and 8-core compute ...
0
votes
2answers
66 views

low performance on HPC cluster (sge) when running multiple jobs

O know this is a long-shot but I'm clueless here. I'm running several computer simulations on High Performance Computation cluster (HPC) of oracale grid engine (sge). A single job runs at a certain ...
0
votes
3answers
105 views

Building Windows Clusters [closed]

I'm a research student and I want to build a windows cluster at home with my laptops to test my parallel codes. The problem is I'm using Windows 7 Home Premium, not a server edition. I'm using ...
0
votes
1answer
179 views

Python Version for HPC with Numpy/Scipy

I am looking to setup an HPC cluster so that it has a modern installation of Python with Numpy/Scipy on the compute nodes. The version of Linux we are using has Python 2.4 installed by default. I ...