HPC Cluster Hardware Resources: Difference between revisions

From HPC Docs
Jump to navigation Jump to search
No edit summary
 
(32 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Networking/Interconnects ==
== Networking/Interconnects ==
All nodes in the chart listed below on this page contain one or two network connections.  All node with a processor family of Broadwell, Skylake Silver or Skylake Gold contain 10Gbps Ethernet network interfaces. Other nodes (e.g. Hapertown and Nehalem) have 1Gbps Ethernet network interfaces.  In addition, some nodes, as indicated in the chart, include an [[WikiMedia:InfiniBand|Infiniband]] interconnect. This is an alternate high-speed, low-latency network connection. Currently ELSA utilized Infiniband FDR which provides 56Gbps speeds. You can control whether your job uses the Infiniband interface by modifying your SLURM job submission script.
All nodes in the chart listed below on this page contain multiple network connections.  All nodes except gpu-node006 through gpu-node008 contain 25Gbps Ethernet network interfaces. The gpu-node006 through gpu-node008 nodes contain 10Gbps Ethernet network interfaces.


== Network Storage ==
== Network Storage ==
The ELSA cluster utilized network based storage that is shared among all nodes for storing personal files, project/research files, course files and the HPC applications. There are two identical storage servers. One is located in the STEM cluster room and another in the Green Hall datacenter.  Data from the STEM storage server is regularly (i.e. weekly) replicated to the one in Green Hall. The is a total of approximately 2PB of raw storage (about 1.2PB of usage storage of redundancy and spares are accounted for). The storage server is Linux-based utilizing the [https://zfsonlinux.org/ ZFS on Linux filesystem] and [[WikiMedia:Network_File_System|NFS]] for sharing with the cluster nodes. It is based on the [http://disruptivestorage.org/ Disruptive Storage Workshop] from the [https://www.jhsph.edu/ Johns Hopkins School of Public Health and BioTeam].
The ELSA cluster utilized network based storage that is shared among all nodes for storing personal files, project/research files, course files and the HPC applications. There are two pairs of identical storage servers. One pari is located in the STEM cluster room and another in the Green Hall datacenter.  Data from the STEM storage servers are regularly (i.e. nightly) replicated to the ones in Green Hall. The is a total of approximately 6.3PB of raw storage available. The storage servers are Linux-based utilizing the [https://zfsonlinux.org/ ZFS on Linux filesystem] and [[wikipedia:Network_File_System|NFS]] for sharing with the cluster nodes.


== Data Transfer Node ==
== Data Transfer Node ==
A [https://fasterdata.es.net/science-dmz/DTN/ data transfer node (DTN)] used for transferring large files in and out of a cluster. It is designed to handle high-speed, high-volume transfers. The ELSA DTN contains 12TB of disk storage for temporarily holding file transfers. It also has a 40Gbps Ethernet interface and Infiniband FDR interconnect to maximize throughput.
A [https://fasterdata.es.net/science-dmz/DTN/ data transfer node (DTN)] is used for transferring large files in and out of a cluster. It is designed to handle high-speed, high-volume transfers. The ELSA DTN contains 122.9TB (raw) of SSD storage for temporarily holding large file transfers. It also has a 100Gbps Ethernet interface to maximize throughput.


== PerfSONAR ==
== PerfSONAR ==
Line 13: Line 13:
== Node Configurations ==
== Node Configurations ==
The following describes the contents of columns in the tables below.
The following describes the contents of columns in the tables below.
* '''Node Name''' = name of the node server. Login nodes (login001, login002, login003, login004) are accessible via the '''elsa.hpc.tcnj.edu''' load-balancer (e.g. using SSH) from the campus network (wired or wireless) or via the [https://tcnj.teamdynamix.com/TDClient/KB/ArticleDet?ID=17531 TCNJ VPN]. The development node dev001 is accessible via '''dev1.hpc.tcnj.edu'''. Other nodes are not meant to be directly accessed.
* '''Node Name''' = name of the node server. Login nodes (login001 & login002) are accessible via the '''elsa.hpc.tcnj.edu''' load-balancer (e.g. using SSH) from the campus network (wired or wireless) or via the [https://tcnj.teamdynamix.com/TDClient/KB/ArticleDet?ID=17531 TCNJ VPN]. Other nodes are not meant to be directly accessed.
* '''Processor Family''' = the generation of processor in the node Skylake Gold > Skylake Silver > Broadwell > Nehalem > Hapertown
* '''Processor Family''' = the generation of processor in the node. Intel: Skylake Gold > Skylake Silver > Broadwell; AMD: Epyc Genoa > Epyc Rome
* '''Available Cores''' = these are the processing cores that compute jobs can use
* '''Available Cores''' = these are the processing cores that compute jobs can use
* '''Reserved Cores''' = these cores are reserved for system use and not available to user jobs
* '''Reserved Cores''' = these cores are reserved for system use and not available to user jobs
* '''RAM Memory''' = how much memory the node contains
* '''RAM Memory''' = how much memory the node contains
* '''Infiniband''' = [[WikiMedia:InfiniBand|Infiniband]] is a high speed interconnect (network) connection. FDR = 56Gbps
* '''GPU Count''' = number of GPU accelerators in the node
* '''GPU Count''' = number of GPU accelerators in the node
* '''NVIDIA GPU Type''' = the model of the GPU accelerators in the node
* '''NVIDIA GPU Type''' = the model of the GPU accelerators in the node. NVIDIA: L40S > RTX2080 >= GTX1080Ti > GTX1080
* '''Queue Membership''' = which queues (SLURM partitions) this node is a member of. Nodes can be a member of multiple queues
* '''Queue Membership''' = which queues (SLURM partitions) this node is a member of. Nodes can be a member of multiple queues. Note some queues are used for internal purposes (e.g. remoteviz, interactive) and should not be used for submitting your jobs except under certain conditions. Please see the [[HPC_Cluster_Job_Scheduler#ELSA_Job_Partitions.2FQueues|SLURM Partitions]] page for more information on the specification of these queues/partition.


{| class="wikitable"
{| class="wikitable"
|-
|-
! Node<br>Name !! Processor<br>Family !! Available<br>Cores !! Reserved<br>Cores !! RAM<br>Memory !! Infiniband !! GPU<br>Count !! NVIDIA<br>GPU Type !! Queue<br>Membership !! Notes
! Node<br>Name !! Processor<br>Family !! Available<br>Cores !! Reserved<br>Cores !! RAM<br>Memory !! GPU<br>Count !! NVIDIA<br>GPU Type !! Queue<br>Membership(s) !! Notes
|-
|-
| login001 || Harpertown || 8 || 0 || 32G || n/a || 0 || n/a || n/a || Public hostname<br>'''elsa.hpc.tcnj.edu'''
| login001 || virtual || 8 || 0 || 8G || 0 || n/a || n/a || Public hostname<br>'''elsa.hpc.tcnj.edu'''
|-
|-
| login002 || virtual || 2 || 0 || 4G || n/a || 0 || n/a || n/a || Public hostname<br>'''elsa.hpc.tcnj.edu'''
| login002 || virtual || 8 || 0 || 8G || 0 || n/a || n/a || Public hostname<br>'''elsa.hpc.tcnj.edu'''
|-
|-
| login003 || virtual || 2 || 0 || 4G || n/a || 0 || n/a || n/a || Public hostname<br>'''elsa.hpc.tcnj.edu'''
| osg-login || virtual || 4 || 0 || 4G || 0 || n/a || n/a || Dedicated for '''Open Science Grid''' job submissions
|-
|-
| login004 || virtual || 2 || 0 || 4G || n/a || 0 || n/a || n/a || Public hostname<br>'''elsa.hpc.tcnj.edu'''
| node001 || AMD Epyc Genoa || 190 || 2 || 1,536G || 0 || n/a || amdtest <!-- short, normal, long, interactive, nolimit --> || &nbsp;
|-
|-
| dev001 || Broadwell || 20 || 0 || 256G || n/a || 1 || GTX 1080 || n/a || Public hostname<br>'''dev1.hpc.tcnj.edu'''
| node002 || AMD Epyc Genoa || 190 || 2 || 1,536G || 0 || n/a || amdtest <!-- short, normal, long, interactive, nolimit --> || &nbsp;
|-
|-
| node001 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || interactive || &nbsp;
| node003 || AMD Epyc Genoa || 190 || 2 || 1,536G || 0 || n/a || amdtest <!-- short, normal, long, interactive, nolimit --> || &nbsp;
|-
|-
| node002 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || interactive || &nbsp;
| node004 || AMD Epyc Genoa || 190 || 2 || 1,536G || 0 || n/a || amdtest <!-- short, normal, long, interactive, nolimit --> || &nbsp;
|-
|-
| node003 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || interactive || &nbsp;
| node005 || AMD Epyc Genoa || 190 || 2 || 1,536G || 0 || n/a || amdtest <!-- short, normal, long, interactive, nolimit --> || &nbsp;
|-
|-
| node004 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || interactive || &nbsp;
| node006 || AMD Epyc Genoa || 190 || 2 || 1,536G || 0 || n/a || amdtest <!-- short, normal, long, interactive, nolimit --> || &nbsp;
<!--
|-
|-
| node005 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || interactive || &nbsp;
| node007 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, interactive || &nbsp;
|-
|-
| node006 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, interactive || &nbsp;
| node008 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, interactive || &nbsp;
|-
|-
| node007 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, interactive || &nbsp;
| node009 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, interactive || &nbsp;
|-
|-
| node008 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, interactive || &nbsp;
| node010 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, interactive || &nbsp;
|-
|-
| node009 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, interactive || &nbsp;
| node011 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, interactive || &nbsp;
|-
|-
| node010 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, interactive || &nbsp;
| node012 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, interactive || &nbsp;
|-
|-
| node011 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, interactive || &nbsp;
| node013 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, interactive || &nbsp;
|-
|-
| node012 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, interactive || &nbsp;
| node014 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, interactive || &nbsp;
|-
|-
| node013 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, interactive || &nbsp;
| node015 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, interactive || &nbsp;
|-
|-
| node014 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, interactive || &nbsp;
| node016 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, interactive || &nbsp;
|-
|-
| node015 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, interactive || &nbsp;
| node017 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, interactive || &nbsp;
|-
|-
| node016 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, interactive || &nbsp;
| node018 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, interactive, nolimit || &nbsp;
|-
|-
| node017 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, interactive || &nbsp;
| node019 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, interactive, nolimit || &nbsp;
|-
|-
| node018 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, interactive || &nbsp;
| node020 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, interactive, nolimit || &nbsp;
|-
|-
| node019 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, interactive || &nbsp;
| node021 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node020 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, interactive || &nbsp;
| node022 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node021 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive || &nbsp;
| node023 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node022 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive || &nbsp;
| node024 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node023 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive || &nbsp;
| node025 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node024 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive || &nbsp;
| node026 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node025 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive || &nbsp;
| node027 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node026 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive || &nbsp;
| node028 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node027 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive || &nbsp;
| node029 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node028 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive || &nbsp;
| node030 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node029 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive || &nbsp;
| node031 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node030 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive || &nbsp;
| node032 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node031 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node033 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node032 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node034 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node033 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node035 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node034 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node036 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node035 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node037 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node036 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node038 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node037 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node039 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node038 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node040 || Intel Nehalem || 7 || 1 || 48G || 0 || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node039 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node041 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node040 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node042 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node041 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node043 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node042 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node044 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node043 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node045 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node044 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node046 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node045 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node047 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node046 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node048 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node047 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node049 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node048 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node050 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node049 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node051 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node050 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node052 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node051 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node053 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node052 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node054 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node053 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node055 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node054 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node056 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node055 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node057 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node056 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node058 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node057 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node059 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node058 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node060 || Intel Nehalem || 7 || 1 || 48G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
-->
|-
|-
| node059 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node061 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node060 || Nehalem || 7 || 1 || 48G || n/a || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
| node062 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node061 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node063 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node062 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node064 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node063 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node065 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node064 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node066 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node065 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node067 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node066 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node068 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node067 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node069 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node068 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node070 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node069 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node071 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node070 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node072 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node071 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node073 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node072 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node074 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node073 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node075 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node074 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node076 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node075 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node077 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node076 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node078 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node077 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node079 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node078 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node080 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node079 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node081 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node080 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node082 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node081 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node083 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node082 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node084 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || short, normal, long, interactive, nolimit || &nbsp;
|-
|-
| node083 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node085 || Intel Skylake Gold || 30 || 2 || 192G || 0 || n/a || interactive || &nbsp;
|-
|-
| node084 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node086 || AMD Rome || 62 || 2 || 512G || 0 || n/a || amd || &nbsp;
|-
|-
| node085 || Skylake Gold || 32 || 0 || 192G || FDR || 0 || n/a || Example || &nbsp;
| node087 || AMD Rome || 62 || 2 || 512G || 0 || n/a || grahamlab || Restricted use
|-
|-
| node131 || Broadwell || 32 || 0 || 128G || FDR || 0 || n/a || Example || &nbsp;
| gpu-node001 || Intel Broadwell || 19 || 1 || 256G || 4 || GTX 1080 || gpu || GPU has 8G VRAM
|-
|-
| node132 || Broadwell || 32 || 0 || 128G || FDR || 0 || n/a || Example || &nbsp;
| gpu-node002 || Intel Broadwell || 19 || 1 || 256G || 4 || GTX 1080 || gpu || GPU has 8G VRAM
|-
|-
| node133 || Broadwell || 32 || 0 || 128G || FDR || 0 || n/a || Example || &nbsp;
| gpu-node003 || Intel Broadwell || 19 || 1 || 256G || 4 || GTX 1080 || gpu || GPU has 8G VRAM
|-
|-
| node134 || Broadwell || 32 || 0 || 128G || FDR || 0 || n/a || Example || &nbsp;
| gpu-node004 || Intel Broadwell || 19 || 1 || 256G || 4 || GTX 1080 || gpu || GPU has 8G VRAM
|-
|-
| node135 || Broadwell || 32 || 0 || 128G || FDR || 0 || n/a || Example || &nbsp;
| gpu-node005 || Intel Broadwell || 19 || 1 || 256G || 4 || GTX 1080 || shortgpu || GPU has 8G VRAM
|-
|-
| node136 || Broadwell || 32 || 0 || 128G || FDR || 0 || n/a || Example || &nbsp;
| gpu-node006 || Intel Skylake Silver || 19 || 1 || 192G || 8 || GTX 1080Ti || gpu || GPU has 11G VRAM
|-
|-
| node137 || Broadwell || 32 || 0 || 128G || FDR || 0 || n/a || Example || &nbsp;
| gpu-node007 || Intel Skylake Silver || 19 || 1 || 192G || 8 || GTX 1080Ti || gpu || GPU has 11G VRAM
|-
|-
| node138 || Broadwell || 32 || 0 || 128G || FDR || 0 || n/a || Example || &nbsp;
| gpu-node008 || Intel Skylake Silver || 19 || 1 || 192G || 8 || GTX 1080Ti || gpu || GPU has 11G VRAM
|-
|-
| gpu-node001 || Broadwell || 20 || 0 || 256G || FDR || 4 || GTX 1080 || Example || &nbsp;
| gpu-node009 || Intel Skylake Gold|| 31 || 1 || 384G || 4 || RTX 2080 || gpu || GPU has 8G VRAM
|-
|-
| gpu-node002 || Broadwell || 20 || 0 || 256G || FDR || 4 || GTX 1080 || Example || &nbsp;
| gpu-node010 || Intel Skylake Gold|| 31 || 1 || 384G || 4 || RTX 2080 || gpu || GPU has 8G VRAM
|-
|-
| gpu-node003 || Broadwell || 20 || 0 || 256G || FDR || 4 || GTX 1080 || Example || &nbsp;
| gpu-node011 || Intel Skylake Gold|| 31 || 1 || 384G || 4 || RTX 2080 || gpu || GPU has 8G VRAM
|-
|-
| gpu-node004 || Broadwell || 20 || 0 || 256G || FDR || 4 || GTX 1080 || Example || &nbsp;
| gpu-node012 || Intel Skylake Gold|| 31 || 1 || 384G || 4 || RTX 2080 || gpu || GPU has 8G VRAM
|-
|-
| gpu-node005 || Broadwell || 20 || 0 || 256G || FDR || 4 || GTX 1080 || shortgpu || &nbsp;
| gpu-node013 || Intel Skylake Gold|| 31 || 1 || 384G || 4 || RTX 2080 || gpu || GPU has 8G VRAM
|-
|-
| gpu-node006 || Skylake Silver || 20 || 0 || 256G || FDR || 8 || GTX 1080Ti || Example || &nbsp;
| gpu-node014 || Intel Skylake Gold|| 31 || 1 || 384G || 4 || RTX 2080 || gpu || GPU has 8G VRAM
|-
|-
| gpu-node007 || Skylake Silver || 20 || 0 || 256G || FDR || 8 || GTX 1080Ti || Example || &nbsp;
| gpu-node015 || Intel Skylake Gold|| 31 || 1 || 384G || 4 || RTX 2080 || gpu || GPU has 8G VRAM
|-
|-
| gpu-node008 || Skylake Silver || 20 || 0 || 256G || FDR || 8 || GTX 1080Ti || Example || &nbsp;
| gpu-node016 || Intel Skylake Gold|| 31 || 1 || 384G || 4 || RTX 2080 || gpu || GPU has 8G VRAM
|-
|-
| gpu-node006 || Skylake Silver || 20 || 0 || 192G || FDR || 8 || GTX 1080Ti || Example || &nbsp;
| gpu-node017 || Intel Skylake Gold|| 31 || 1 || 384G || 4 || RTX 2080 || gpu || GPU has 8G VRAM
|-
|-
| gpu-node007 || Skylake Silver || 20 || 0 || 192G || FDR || 8 || GTX 1080Ti || Example || &nbsp;
| gpu-node018 || Intel Skylake Gold|| 31 || 1 || 384G || 4 || RTX 2080 || gpu || GPU has 8G VRAM
|-
|-
| gpu-node008 || Skylake Silver || 20 || 0 || 192G || FDR || 8 || GTX 1080Ti || Example || &nbsp;
| gpu-node019 || AMD Epyc Genoa || 30 || 2 || 512G || 4 || L40S || gputest || GPU has 48G VRAM
|-
|-
| gpu-node009 || Skylake Gold|| 32 || 0 || 384G || FDR || 4 || RTX 2080 || Example || &nbsp;
| gpu-node020 || AMD Epyc Genoa || 30 || 2 || 512G || 4 || L40S || gputest || GPU has 48G VRAM
|-
|-
| gpu-node010 || Skylake Gold|| 32 || 0 || 384G || FDR || 4 || RTX 2080 || Example || &nbsp;
| gpu-node021 || AMD Epyc Genoa || 30 || 2 || 512G || 4 || L40S || gputest || GPU has 48G VRAM
|-
|-
| gpu-node011 || Skylake Gold|| 32 || 0 || 384G || FDR || 4 || RTX 2080 || Example || &nbsp;
| viz-node001 || Intel Skylake Gold|| 38 || 2 || 768G || 1 || Tesla V100 || remoteviz || '''Tesla V100 (high-end)''' remote visualization option
|-
|-
| gpu-node012 || Skylake Gold|| 32 || 0 || 384G || FDR || 4 || RTX 2080 || Example || &nbsp;
| viz-node002 || Intel Skylake Gold|| 38 || 2 || 768G || 1 || Tesla V100 || remoteviz || '''Tesla V100 (high-end)''' remote visualization option
|-
|-
| gpu-node013 || Skylake Gold|| 32 || 0 || 384G || FDR || 4 || RTX 2080 || Example || &nbsp;
| dev-node001 || Intel Broadwell || 20 || 0 || 256G || 1 || GTX 1080 || dev || '''GTX1080 (low-end)''' remote visualization option
|-
|-
| gpu-node014 || Skylake Gold|| 32 || 0 || 384G || FDR || 4 || RTX 2080 || Example || &nbsp;
|-
| gpu-node015 || Skylake Gold|| 32 || 0 || 384G || FDR || 4 || RTX 2080 || Example || &nbsp;
|-
| gpu-node016 || Skylake Gold|| 32 || 0 || 384G || FDR || 4 || RTX 2080 || Example || &nbsp;
|-
| gpu-node017 || Skylake Gold|| 32 || 0 || 384G || FDR || 4 || RTX 2080 || Example || &nbsp;
|-
| gpu-node018 || Skylake Gold|| 32 || 0 || 384G || FDR || 4 || RTX 2080 || Example || &nbsp;
|-
| viz-node001 || Skylake Gold|| 40 || 0 || 768G || FDR || 4 || Tesla V100 || remoteviz || &nbsp;
|-
| viz-node002 || Skylake Gold|| 40 || 0 || 768G || FDR || 4 || Tesla V100 || remoteviz || &nbsp;
|}
|}

Latest revision as of 16:25, 6 September 2024

Networking/Interconnects

All nodes in the chart listed below on this page contain multiple network connections. All nodes except gpu-node006 through gpu-node008 contain 25Gbps Ethernet network interfaces. The gpu-node006 through gpu-node008 nodes contain 10Gbps Ethernet network interfaces.

Network Storage

The ELSA cluster utilized network based storage that is shared among all nodes for storing personal files, project/research files, course files and the HPC applications. There are two pairs of identical storage servers. One pari is located in the STEM cluster room and another in the Green Hall datacenter. Data from the STEM storage servers are regularly (i.e. nightly) replicated to the ones in Green Hall. The is a total of approximately 6.3PB of raw storage available. The storage servers are Linux-based utilizing the ZFS on Linux filesystem and NFS for sharing with the cluster nodes.

Data Transfer Node

A data transfer node (DTN) is used for transferring large files in and out of a cluster. It is designed to handle high-speed, high-volume transfers. The ELSA DTN contains 122.9TB (raw) of SSD storage for temporarily holding large file transfers. It also has a 100Gbps Ethernet interface to maximize throughput.

PerfSONAR

PerfSONAR is a network performance testing and monitoring system. It regularly runs tests bandwidth and latency tests and if issues arise, it helps pinpoint the location in the network path causing the issue.

Node Configurations

The following describes the contents of columns in the tables below.

  • Node Name = name of the node server. Login nodes (login001 & login002) are accessible via the elsa.hpc.tcnj.edu load-balancer (e.g. using SSH) from the campus network (wired or wireless) or via the TCNJ VPN. Other nodes are not meant to be directly accessed.
  • Processor Family = the generation of processor in the node. Intel: Skylake Gold > Skylake Silver > Broadwell; AMD: Epyc Genoa > Epyc Rome
  • Available Cores = these are the processing cores that compute jobs can use
  • Reserved Cores = these cores are reserved for system use and not available to user jobs
  • RAM Memory = how much memory the node contains
  • GPU Count = number of GPU accelerators in the node
  • NVIDIA GPU Type = the model of the GPU accelerators in the node. NVIDIA: L40S > RTX2080 >= GTX1080Ti > GTX1080
  • Queue Membership = which queues (SLURM partitions) this node is a member of. Nodes can be a member of multiple queues. Note some queues are used for internal purposes (e.g. remoteviz, interactive) and should not be used for submitting your jobs except under certain conditions. Please see the SLURM Partitions page for more information on the specification of these queues/partition.
Node
Name		 Processor
Family		 Available
Cores		 Reserved
Cores		 RAM
Memory		 GPU
Count		 NVIDIA
GPU Type		 Queue
Membership(s)		 Notes
login001 virtual 8 0 8G 0 n/a n/a Public hostname
elsa.hpc.tcnj.edu
login002 virtual 8 0 8G 0 n/a n/a Public hostname
elsa.hpc.tcnj.edu
osg-login virtual 4 0 4G 0 n/a n/a Dedicated for Open Science Grid job submissions
node001 AMD Epyc Genoa 190 2 1,536G 0 n/a amdtest  
node002 AMD Epyc Genoa 190 2 1,536G 0 n/a amdtest  
node003 AMD Epyc Genoa 190 2 1,536G 0 n/a amdtest  
node004 AMD Epyc Genoa 190 2 1,536G 0 n/a amdtest  
node005 AMD Epyc Genoa 190 2 1,536G 0 n/a amdtest  
node006 AMD Epyc Genoa 190 2 1,536G 0 n/a amdtest  
node061 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node062 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node063 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node064 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node065 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node066 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node067 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node068 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node069 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node070 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node071 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node072 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node073 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node074 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node075 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node076 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node077 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node078 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node079 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node080 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node081 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node082 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node083 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node084 Intel Skylake Gold 30 2 192G 0 n/a short, normal, long, interactive, nolimit  
node085 Intel Skylake Gold 30 2 192G 0 n/a interactive  
node086 AMD Rome 62 2 512G 0 n/a amd  
node087 AMD Rome 62 2 512G 0 n/a grahamlab Restricted use
gpu-node001 Intel Broadwell 19 1 256G 4 GTX 1080 gpu GPU has 8G VRAM
gpu-node002 Intel Broadwell 19 1 256G 4 GTX 1080 gpu GPU has 8G VRAM
gpu-node003 Intel Broadwell 19 1 256G 4 GTX 1080 gpu GPU has 8G VRAM
gpu-node004 Intel Broadwell 19 1 256G 4 GTX 1080 gpu GPU has 8G VRAM
gpu-node005 Intel Broadwell 19 1 256G 4 GTX 1080 shortgpu GPU has 8G VRAM
gpu-node006 Intel Skylake Silver 19 1 192G 8 GTX 1080Ti gpu GPU has 11G VRAM
gpu-node007 Intel Skylake Silver 19 1 192G 8 GTX 1080Ti gpu GPU has 11G VRAM
gpu-node008 Intel Skylake Silver 19 1 192G 8 GTX 1080Ti gpu GPU has 11G VRAM
gpu-node009 Intel Skylake Gold 31 1 384G 4 RTX 2080 gpu GPU has 8G VRAM
gpu-node010 Intel Skylake Gold 31 1 384G 4 RTX 2080 gpu GPU has 8G VRAM
gpu-node011 Intel Skylake Gold 31 1 384G 4 RTX 2080 gpu GPU has 8G VRAM
gpu-node012 Intel Skylake Gold 31 1 384G 4 RTX 2080 gpu GPU has 8G VRAM
gpu-node013 Intel Skylake Gold 31 1 384G 4 RTX 2080 gpu GPU has 8G VRAM
gpu-node014 Intel Skylake Gold 31 1 384G 4 RTX 2080 gpu GPU has 8G VRAM
gpu-node015 Intel Skylake Gold 31 1 384G 4 RTX 2080 gpu GPU has 8G VRAM
gpu-node016 Intel Skylake Gold 31 1 384G 4 RTX 2080 gpu GPU has 8G VRAM
gpu-node017 Intel Skylake Gold 31 1 384G 4 RTX 2080 gpu GPU has 8G VRAM
gpu-node018 Intel Skylake Gold 31 1 384G 4 RTX 2080 gpu GPU has 8G VRAM
gpu-node019 AMD Epyc Genoa 30 2 512G 4 L40S gputest GPU has 48G VRAM
gpu-node020 AMD Epyc Genoa 30 2 512G 4 L40S gputest GPU has 48G VRAM
gpu-node021 AMD Epyc Genoa 30 2 512G 4 L40S gputest GPU has 48G VRAM
viz-node001 Intel Skylake Gold 38 2 768G 1 Tesla V100 remoteviz Tesla V100 (high-end) remote visualization option
viz-node002 Intel Skylake Gold 38 2 768G 1 Tesla V100 remoteviz Tesla V100 (high-end) remote visualization option
dev-node001 Intel Broadwell 20 0 256G 1 GTX 1080 dev GTX1080 (low-end) remote visualization option
Retrieved from "https://docs.hpc.tcnj.edu/index.php?title=HPC_Cluster_Hardware_Resources&oldid=587"