HPC Cluster Hardware Resources

From HPC Docs
Revision as of 12:16, 1 June 2022 by Admin (talk | contribs)
Jump to navigation Jump to search

Networking/Interconnects

All nodes in the chart listed below on this page contain one or two network connections. All node with a processor family of Broadwell, Skylake Silver or Skylake Gold contain 10Gbps Ethernet network interfaces. Other nodes (e.g. Hapertown and Nehalem) have 1Gbps Ethernet network interfaces. In addition, some nodes, as indicated in the chart, include an Infiniband interconnect. This is an alternate high-speed, low-latency network connection. Currently ELSA utilized Infiniband FDR which provides 56Gbps speeds. You can control whether your job uses the Infiniband interface by modifying your SLURM job submission script.

Network Storage

The ELSA cluster utilized network based storage that is shared among all nodes for storing personal files, project/research files, course files and the HPC applications. There are two identical storage servers. One is located in the STEM cluster room and another in the Green Hall datacenter. Data from the STEM storage server is regularly (i.e. weekly) replicated to the one in Green Hall. The is a total of approximately 2PB of raw storage (about 1.2PB of usage storage of redundancy and spares are accounted for). The storage server is Linux-based utilizing the ZFS on Linux filesystem and NFS for sharing with the cluster nodes. It is based on the Disruptive Storage Workshop from the Johns Hopkins School of Public Health and BioTeam.

Data Transfer Node

A data transfer node (DTN) is used for transferring large files in and out of a cluster. It is designed to handle high-speed, high-volume transfers. The ELSA DTN contains 12TB of disk storage for temporarily holding file transfers. It also has a 40Gbps Ethernet interface and Infiniband FDR interconnect to maximize throughput.

PerfSONAR

PerfSONAR is a network performance testing and monitoring system. It regularly runs tests bandwidth and latency tests and if issues arise, it helps pinpoint the location in the network path causing the issue.

Node Configurations

The following describes the contents of columns in the tables below.

  • Node Name = name of the node server. Login nodes (login001, login002, login003, login004) are accessible via the elsa.hpc.tcnj.edu load-balancer (e.g. using SSH) from the campus network (wired or wireless) or via the TCNJ VPN. The development node dev001 is accessible via dev1.hpc.tcnj.edu. Other nodes are not meant to be directly accessed.
  • Processor Family = the generation of processor in the node Skylake Gold > Skylake Silver > Broadwell > Nehalem > Hapertown
  • Available Cores = these are the processing cores that compute jobs can use
  • Reserved Cores = these cores are reserved for system use and not available to user jobs
  • RAM Memory = how much memory the node contains
  • Infiniband = Infiniband is a high speed interconnect (network) connection. FDR = 56Gbps
  • GPU Count = number of GPU accelerators in the node
  • NVIDIA GPU Type = the model of the GPU accelerators in the node
  • Queue Membership = which queues (SLURM partitions) this node is a member of. Nodes can be a member of multiple queues. Note some queues are used for internal purposes (e.g. remoteviz, interactive) and should not be used for submitting your jobs except under certain conditions. Please see the SLURM Partitions page for more information on the specification of these queues/partition.
Node
Name
Processor
Family
Available
Cores
Reserved
Cores
RAM
Memory
Infiniband GPU
Count
NVIDIA
GPU Type
Queue
Membership(s)
Notes
login001 virtual 8 0 8G n/a 0 n/a n/a Public hostname
elsa.hpc.tcnj.edu
login002 virtual 8 0 8G n/a 0 n/a n/a Public hostname
elsa.hpc.tcnj.edu
osg-login virtual 4 0 4G n/a 0 n/a n/a Dedicated for Open Science Grid job submissions
  Intel Broadwell 20 0 256G n/a 1 GTX 1080 n/a Currently unused
node001 Intel Nehalem 7 1 48G n/a 0 n/a interactive  
node002 Intel Nehalem 7 1 48G n/a 0 n/a interactive  
node003 Intel Nehalem 7 1 48G n/a 0 n/a interactive  
node004 Intel Nehalem 7 1 48G n/a 0 n/a interactive  
node005 Intel Nehalem 7 1 48G n/a 0 n/a interactive  
node006 Intel Nehalem 7 1 48G n/a 0 n/a short, interactive  
node007 Intel Nehalem 7 1 48G n/a 0 n/a short, interactive  
node008 Intel Nehalem 7 1 48G n/a 0 n/a short, interactive  
node009 Intel Nehalem 7 1 48G n/a 0 n/a short, interactive  
node010 Intel Nehalem 7 1 48G n/a 0 n/a short, interactive  
node011 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, interactive  
node012 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, interactive  
node013 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, interactive  
node014 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, interactive  
node015 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, interactive  
node016 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, interactive  
node017 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, interactive  
node018 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, interactive  
node019 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, interactive  
node020 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, interactive  
node021 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive  
node022 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive  
node023 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive  
node024 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive  
node025 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive  
node026 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive  
node027 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive  
node028 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive  
node029 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive  
node030 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive  
node031 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node032 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node033 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node034 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node035 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node036 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node037 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node038 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node039 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node040 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node041 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node042 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node043 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node044 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node045 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node046 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node047 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node048 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node049 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node050 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node051 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node052 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node053 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node054 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node055 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node056 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node057 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node058 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node059 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node060 Intel Nehalem 7 1 48G n/a 0 n/a short, normal, long, interactive, nolimit  
node061 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node062 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node063 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node064 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node065 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node066 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node067 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node068 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node069 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node070 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node071 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node072 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node073 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node074 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node075 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node076 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node077 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node078 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node079 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node080 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node081 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node082 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive  
node083 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive, nolimit  
node084 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive, nolimit  
node085 Intel Skylake Gold 30 2 192G FDR 0 n/a short, normal, long, interactive, nolimit  
node086 AMD EPYC2 62 2 512G FDR 0 n/a amd  
node087 AMD EPYC2 62 2 512G FDR 0 n/a amd  
node131 Intel Broadwell 30 2 128G FDR 0 n/a short, normal, long, interactive, nolimit  
node132 Intel Broadwell 30 2 128G FDR 0 n/a short, normal, long, interactive, nolimit  
node133 Intel Broadwell 30 2 128G FDR 0 n/a short, normal, long, interactive, nolimit  
node134 Intel Broadwell 30 2 128G FDR 0 n/a short, normal, long, interactive, nolimit  
node135 Intel Broadwell 30 2 128G FDR 0 n/a short, normal, long, interactive, nolimit  
node136 Intel Broadwell 30 2 128G FDR 0 n/a short, normal, long, interactive, nolimit  
node137 Intel Broadwell 30 2 128G FDR 0 n/a short, normal, long, interactive, nolimit  
node138 Intel Broadwell 30 2 128G FDR 0 n/a short, normal, long, interactive, nolimit  
gpu-node001 Intel Broadwell 20 0 256G FDR 4 GTX 1080 gpu  
gpu-node002 Intel Broadwell 20 0 256G FDR 4 GTX 1080 gpu  
gpu-node003 Intel Broadwell 20 0 256G FDR 4 GTX 1080 gpu  
gpu-node004 Intel Broadwell 20 0 256G FDR 4 GTX 1080 gpu  
gpu-node005 Intel Broadwell 20 0 256G FDR 4 GTX 1080 shortgpu  
gpu-node006 Intel Skylake Silver 20 0 192G FDR 8 GTX 1080Ti gpu  
gpu-node007 Intel Skylake Silver 20 0 192G FDR 8 GTX 1080Ti gpu  
gpu-node008 Intel Skylake Silver 20 0 192G FDR 8 GTX 1080Ti gpu  
gpu-node009 Intel Skylake Gold 32 0 384G FDR 4 RTX 2080 gpu  
gpu-node010 Intel Skylake Gold 32 0 384G FDR 4 RTX 2080 gpu  
gpu-node011 Intel Skylake Gold 32 0 384G FDR 4 RTX 2080 gpu  
gpu-node012 Intel Skylake Gold 32 0 384G FDR 4 RTX 2080 gpu  
gpu-node013 Intel Skylake Gold 32 0 384G FDR 4 RTX 2080 gpu  
gpu-node014 Intel Skylake Gold 32 0 384G FDR 4 RTX 2080 gpu  
gpu-node015 Intel Skylake Gold 32 0 384G FDR 4 RTX 2080 gpu  
gpu-node016 Intel Skylake Gold 32 0 384G FDR 4 RTX 2080 gpu  
gpu-node017 Intel Skylake Gold 32 0 384G FDR 4 RTX 2080 gpu  
gpu-node018 Intel Skylake Gold 32 0 384G FDR 4 RTX 2080 gpu  
viz-node001 Intel Skylake Gold 40 0 768G FDR 1 Tesla V100 remoteviz  
viz-node002 Intel Skylake Gold 40 0 768G FDR 1 Tesla V100 remoteviz