NIIF Systems
NIIF systems are maintained by NIIF.
There are useful guides about it in their wikipedia, but mostly in hungarian.
Legacy information: Computing Service Legacy
Debrecen
This system maintained by NIIF.
Specification
Type | SGI ICE8400EX |
# of CPUs / node | 2 |
# of cores / CPU | 6 |
Memory / node | 47 GB |
Memory / core | 3.9 GB |
CPU | Intel Xeon X5680 @ 3.33 GHz SMT on |
Architecture | x86_64 / intel64 / em64t little-endian |
Scheduler | Slurm |
MPI | SGI MPT (mpt), Intel MPI (impi), Open MPI (ompi) |
Precompiled Environment
From Skynet sync up the precompiled environment:
cd /share/niif/debrecen sshput -m debrecen -s .
Log into debrecen:
sshto -m debrecen
Then add the following into your .profile:
PATH=$PATH:$HOME/bin
VASP
We use intel MPI for VASP. The SGI MPT version might be good, but it might break the nodes with unkillable orphan processes, beware! Generally please stick to the impi version supplied here if possible.
First please transfer the compiled VASP binary and projectors from skynet.
cd /share/niif/ sshput -t 2 -m debrecen -s vasp/5.4.1.03082016.impi sshput -t 2 -m debrecen -s vasp/proj echo -e "\nexport VASP_PROJ_HOME=$HOME/vasp/proj" >> $HOME/.profile
You can find a sample job of an ozone molecule in $HOME/jobsamples
cd $HOME/jobsamples/ozone
Please fill your email in the debrecen jobfile,
mcedit debrecen_impi
then submit it:
sbatch debrecen_impi
This job should finish in mere seconds. Please replace the partition to "prod" from "test" for actual large scale runs.
#SBATCH --partition=prod
You can easily increase the number of nodes:
#SBATCH --nodes=4
Job Monitoring
Pécs
The system is maintained by NIIF.
Specification
Type | SGI UV 1000 |
# of CPUs / node | 2 |
# of cores / CPU | 6 |
Memory | 6 TB |
CPU | Intel Xeon X7542 @ 2.66 GHz SMT off |
Architecture | x86_64 / intel64 / em64t little-endian |
Scheduler | Slurm |
MPI | SGI MPT (mpt) |
ESZR | local |
Shell Environment
Precompiled Environment
From Skynet sync up the precompiled environment:
cd /share/niif/pecs sshput
The precompiled environment does not contain the program binaries and data libraries. To transfer a PROGRAM:
cd /share/niif/pecs/pkg sshtx put pecs PROGRAM
Shell Framework
Login to pecs:
sshin pecs
On pecs:
cd $HOME git clone git://github.com/hornos/shf3.git
Source and setup the Shell Framework in $HOME/.profile
:
source $HOME/shf3/bin/shfrc # set the prompt shf3/ps1 PECS[\\h] # set framework features shf3/alias yes shf3/screen yes shf3/mc/color yes # screen workaround if shf3/is/screen ; then source "/etc/profile.d/modules.sh" fi # tab complete source $HOME/shf3/bin/complete
Parallel Compressor
Enable the parallel compressor for the framework:
cd $HOME echo "sys_zip_xg_gz=pigz" > shf3/lib/sys/zip/xg/config.$USER
Module Environment
ESZR is our unified computing environment. Enable ESZR system modules in $HOME/.profile
:
# reset module purge module use ${HOME}/site/eszr/mod/common module load eszr/local module load eszr/env/local module load eszr/sys/niif/pecs module load eszr/sys/niif/pecs.mpt module use ${HOME}/site/eszr/mod/local module load sgi/2011 source ${ESZR_ROOT}/env/alias
Logout and login again.
Scheduler
The job scheduler is SGE.
General information about the queues:
qstat -g c
Queue | Allowed Groups | Purpose |
---|---|---|
test.q | ALL | Test queue |
pecs.q | ALL | Production |
General information on jobs:
qstat -u "*"
Job Setup
Setup the Queue file and edit the parameters:
cd $HOME/shf3/mid/que cp templates/niif/pecs . mcedit debrecen
Job template is in $HOME/shf3/mid/que/templates/niif/pecs.job
Job Monitoring
Currently, monitoring is possible only by chart. You also have to enable the numainfo
script in the queue file. In the job's submit directory:
pcpview -c -j StdOut
Maximum utilization in the chart is 6 (# of cores per node).
Special Options
The UV is a ccNUMA SMP machine and you allocate CPU sockets and cores on one node. It is mandatory to set in the job file:
NODES=1
The total number of SGE slots will be SCKTS*CORES
.
Szeged
The system is maintained by NIIF.
Specification
Type | HP CP4000BL |
# of CPUs / node | 4 |
# of cores / CPU | 12 |
Memory / node | 132 GB |
Memory / core | 2.75 GB |
CPU | AMD Opteron 6174 @ 2.2GHz |
Architecture | x86_64 / intel64 / em64t little-endian |
Scheduler | Slurm |
MPI | Intel (impi) |
ESZR | local |
Shell Environment
Precompiled Environment
From Skynet sync up the precompiled environment:
cd /share/niif/szeged sshput
The precompiled environment does not contain the program binaries and data libraries. To transfer a PROGRAM:
cd /share/niif/szeged/pkg sshtx put szeged PROGRAM
Shell Framework
Login to szeged:
sshin szeged
On szeged:
cd $HOME git clone git://github.com/hornos/shf3.git
Source and setup the Shell Framework in $HOME/.bash_profile
:
source $HOME/shf3/bin/shfrc # set the prompt shf3/ps1 SZEGED[\\h] # set framework features shf3/alias yes shf3/screen yes shf3/mc/color yes # screen workaround if shf3/is/screen ; then source "/etc/profile.d/modules.sh" fi # tab complete source $HOME/shf3/bin/complete
Parallel Compressor
Enable the parallel compressor for the framework:
cd $HOME echo "sys_zip_xg_gz=pigz" > shf3/lib/sys/zip/xg/config.$USER
Module Environment
ESZR is our unified computing environment. Enable ESZR system modules in $HOME/.bash_profile
:
# reset module purge module use ${HOME}/site/eszr/mod/common module load eszr/local module load eszr/env/local module load eszr/sys/niif/szeged module use ${HOME}/site/eszr/mod/local source ${ESZR_ROOT}/env/alias
Scheduler
The job scheduler is SGE.
General information about the queues:
qstat -g c
Queue | Allowed Groups | Purpose |
---|---|---|
test.q | ALL | Test queue (2 nodes) |
szeged.q | ALL | Production |
General information on jobs:
qstat -u "*"
Job Setup
Setup the Queue file and edit the parameters:
cd $HOME/shf3/mid/que cp templates/niif/szeged . mcedit szeged
Job template is in $HOME/shf3/mid/que/templates/niif/szeged.job
Special Options
If you need to allocate a full node but want to start arbitrary number of MPI processes set in the job file:
SLTPN=48
which will specify the total number of SGE slots per node and the total number of slots will be NODES*SLTPN
. You can use the following combinations in MPI-OMP mode if you run out of memory. Set hybrid mode in the job file:
MODE=mpiomp/impi
and the socket/core number accoring to your needs.
SCKTS (# of MPI proces / node) | CORES (# of OMP threads / MPI proc) | Memory / MPI proc |
---|---|---|
2 | 24 | 66 GB |
4 | 12 | 33 GB |
8 | 6 | 16.5 GB |
12 | 4 | 8.3 GB |
24 | 2 | 4.3 GB |
Budapest
The system is maintained by NIIF.
Specification
Type | HP CP4000BL |
# of CPUs / node | 2 |
# of cores / CPU | 12 |
Memory / node | 66 GB |
Memory / core | 2.75 GB |
CPU | AMD Opteron 6174 @ 2.2GHz |
Architecture | x86_64 / intel64 / em64t little-endian |
Scheduler | Slurm |
MPI | Intel (impi) |
ESZR | local |
Shell Environment
Precompiled Environment
From Skynet sync up the precompiled environment:
cd /share/niif/budapest sshput
The precompiled environment does not contain the program binaries and data libraries. To transfer a PROGRAM:
cd /share/niif/budapest/pkg sshtx put budapest PROGRAM
Shell Framework
Login to budapest:
sshin budapest
On budapest:
cd $HOME git clone git://github.com/hornos/shf3.git
Source and setup the Shell Framework in $HOME/.bash_profile
:
source $HOME/shf3/bin/shfrc # set the prompt shf3/ps1 BUDAPEST[\\h] # set framework features shf3/alias yes shf3/screen yes shf3/mc/color yes # screen workaround if shf3/is/screen ; then source "/etc/profile.d/modules.sh" fi # tab complete source $HOME/shf3/bin/complete
Parallel Compressor
Enable the parallel compressor for the framework:
cd $HOME echo "sys_zip_xg_gz=pigz" > shf3/lib/sys/zip/xg/config.$USER
Module Environment
ESZR is our unified computing environment. Enable ESZR system modules in $HOME/.bash_profile
:
# reset module purge module use ${HOME}/site/eszr/mod/common module load eszr/local module load eszr/env/local module load eszr/sys/niif/budapest module use ${HOME}/site/eszr/mod/local source ${ESZR_ROOT}/env/alias
Scheduler
The job scheduler is SGE.
General information about the queues:
qstat -g c
Queue | Allowed Groups | Purpose |
---|---|---|
test.q | ALL | Test queue (2 nodes) |
budapest.q | ALL | Production |
General information on jobs:
qstat -u "*"
Job Setup
Setup the Queue file and edit the parameters:
cd $HOME/shf3/mid/que cp templates/niif/budapest . mcedit budapest
Job template is in $HOME/shf3/mid/que/templates/niif/budapest.job
Special Options
If you need to allocate a full node but want to start arbitrary number of MPI processes set in the job file:
SLTPN=24
which will specify the total number of SGE slots per node and the total number of slots will be NODES*SLTPN
. You can use the following combinations in MPI-OMP mode if you run out of memory. Set hybrid mode in the job file:
MODE=mpiomp/impi
and the socket/core number accoring to your needs.
SCKTS (# of MPI proc / node) | CORES (# of OMP threads / MPI proc) | Memory / MPI proc |
---|---|---|
2 | 12 | 33 GB |
4 | 6 | 16.5 GB |
8 | 3 | 8.3 GB |