Platform"HybriLIT"

Installed Software


System software (Linux, SLURM, Modules)

  • Scientific Linux

    7.5 (Nitrogen)

  • SLURM
  • Modules
  • gcc

    gcc/v4.9.3-1, gcc/v5.3.0-1, gcc/v6.2.0-2, gcc/v7.2.0-1, gcc/v8.2.0-1, gcc/v8.3.0, gcc/v9.1.0-1, gcc/v10.2.0, gcc/v11.2.0


Parallel computing software

* To view the current software version, type the following command in terminal: module avail and the name of the module, for example:

module avail CUDA

Application Packages

* To view the current software version, type the following command in terminal: module avail and the name of the module, for example:

module avail CMake
CMake

CMake is a cross-platform free and open-source software application for managing the build process of software using a compiler-independent method.
Versions: v3.16.5, v3.19.1, v3.22.0
To set environmental variable, please use the following command (according to each version):

module add CMake/v3.16.5

Website: https://cmake.org/

FairSoft

The FairSoft framework is an object oriented simulation, reconstruction and data analysis framework.
Versions: may16p1_gcc485, oct17p1, may18p1, june19p1_gcc485, june19p2_gcc485, apr21patches_gcc1120
To set environmental variable, please use the following command (according to each version):

module add FairSoft/apr21patches_gcc1120

Website: https://fairroot.gsi.de/

FairRoot

The FairRoot framework is an object oriented simulation, reconstruction and data analysis framework.
Versions: v16.06_gcc485, oct17p1, v18.0.4, v18.2.0_gcc485, v18.2.1_gcc485, v18.4.2_gcc1120
To set environmental variable, please use the following command (according to each version):

module add FairRoot/oct17p1

Website: https://fairroot.gsi.de/

FLAIR

FLAIR – is an advanced user friendly interface for FLUKA to facilitate the editing of FLUKA input files, execution of the code and visualization of the output files.
Version: v2.3.0
To set environmental variable, please use the following command:

module add FLAIR v2.3.0

Website: http://www.fluka.org/flair/

FLUKA

FLUKA – is a fully integrated Monte Carlo simulation package for the interaction and transport of particles and nuclei in matter. FLUKA has many applications in particle physics, high energy experimental physics and engineering, shielding, detector and telescope design, cosmic ray studies, dosimetry, medical physics, radiobiology.
Versions: v2011.2x-8, v2020.0.3, v2021.2.2
To set environmental variable, please use the following command (according to each version):

module add FLUKA/v2011.2x-8

Website: http://www.fluka.org/fluka.php

GEANT4

GEANT – is a platform for “the simulation of the passage of particles through matter” using Monte Carlo methods.
Version: v4.10.07.p01_gcc910
To set environmental variable, please use the following command:

module add GEANT4/v4.10.07.p01_gcc910

Website: http://www.geant4.org/geant4/

GROMACS

GROMACS is a molecular dynamics package mainly designed for simulations of proteins, lipids, and nucleic acids.
Versions: v2019.3, v5.1.3_gcc485_cuda80
To set environmental variable, please use the following command (according to each version):

module add GROMACS/v2019.3

Website: http://www.gromacs.org/

java

java is a compiler for the programming language Java.

  • Version: v8u181
    To set environmental variable, please use the following command:

    module add java v8u181
  • Version: v8u91-1
    To set environmental variable, please use the following command:

    module add java v8u91-1
LAMMPS

LAMMPS – is a molecular dynamics program from Sandia National Laboratories. LAMMPS makes use of Message Passing Interface (MPI) for parallel communication and is free and open-source software, distributed under the terms of the GNU General Public License.
Version: v12.12.18
To set environmental variable, please use the following command:

module add LAMMPS v12.12.18

Website: https://lammps.sandia.gov/

PandaRoot

PandaRoot – a framework for offline simulation framework for The PANDA experiment at FAIR in Darmstadt based upon the ROOT and Virtual Monte-Carlo.Versions: dec17p2b, may19
To set environmental variable, please use the following command (according to each version):

module add PandaRoot/dec17p2b

Website: https://panda.gsi.de/

Python

Python – an interpreted, high-level, general-purpose programming language. Python has a design philosophy that emphasizes code readability, notably using significant whitespace. It provides constructs that enable clear programming on both small and large scales.
Versions: 2.7.10-3, 3.6.5
To set environmental variable, please use the following command (according to each version):

module add Python/v2.7.10-3

Website: https://www.python.org/

REDUCE

REDUCE is a system for doing scalar, vector and matrix algebra by computer, which also supports arbitrary precision numerical approximation and interfaces to gnuplot to provide graphics.Version: svn-4830
Version: svn-4830
To set environmental variable, please use the following command:

module add reduce-algebra/svn-4830

Website: https://reduce-algebra.sourceforge.io

ROOT

ROOT is a modular scientific software toolkit. It provides all the functionalities needed to deal with big data processing, statistical analysis, visualisation and storage.
Version: 6-18-00
To set environmental variable, please use the following command:

module add ROOT/v6-18-00

Website: https://root.cern.ch/


Licensed software packages

* To view the current software version, type the following command in terminal: module avail and the name of the module, for example:

module avail COMSOL
  • Comsol Multiphysics

    Comsol Multiphysics is a cross-platform finite element analysis, solver and multiphysics simulation software. It allows conventional physics-based user interfaces and coupled systems of partial differential equations (PDEs). COMSOL provides an IDE and unified workflow for electrical, mechanical, fluid, and chemical applications.
    Version: 5.6, 6.0
    Available licenses: 4 graphic and 1 for each module
    To set environmental variable, please use the following command:

    module add COMSOL/v6.0

    Website: https://www.comsol.ru/comsol-multiphysics

  • Maple

    Maple is a symbolic and numeric computing environment, and is also a multi-paradigm programming language. Developed by Maplesoft, Maple also covers other aspects of technical computing, including visualization, data analysis, matrix computation, and connectivity.
    Version: 2020.2
    Available licenses: 5
    To set environmental variable, please use the following command:

    module add Maple/v2020.2

    Website: https://www.maplesoft.com/products/maple/

  • Wolfram Mathematica

    Wolfram Mathematica is a modern technical computing system spanning most areas of technical computing — including neural networks, machine learning, image processing, geometry, data science, visualizations, and others. The system is used in many technical, scientific, engineering, mathematical, and computing fields.
    Version: 11.2-1
    Available licenses: 5
    To set environmental variable, please use the following command:

    module add Mathematica/v11.2-1

    Website: https://reference.wolfram.com/language/

  • Matlab

    Matlab (matrix laboratory) is a multi-paradigm numerical computing environment and proprietary programming language developed by MathWorks. MATLAB allows matrix manipulations, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other languages, including C, C++, C#, Java, Fortran and Python.
    Version: R2020b
    Available licenses: 6
    To set environmental variable, please use the following command:

    module add MATLAB/R2020b

    Website: https://www.mathworks.com/help/matlab/


Библиотеки

* To view the current software version, type the following command in terminal: module avail and the name of the module, for example:

module avail CLN
CLN

CLN is a library for efficient computations with all kinds of numbers in arbitrary precision.
Version: 1.3.4-1
To set environmental variable, please use the following command:

module add CLN/v1.3.4-1

Website: https://www.ginac.de/CLN/

FFTW

The Fastest Fourier Transform in the West (FFTW) is a software library for computing discrete Fourier transforms (DFTs).
Versions: v3.3.7-2, v3.3.7-5
To set environmental variable, please use the following command (according to each version):

module add fftw/v3.3.7-2

Website: http://www.fftw.org/

Ginac

Ginac – is C++ libarary, is a free computer algebra system released under the GNU General Public License. The name is a recursive acronym for “GiNaC is Not a CAS” (Computer Algebra System).
Version: v1.7.3-1
To set environmental variable, please use the following command:

module add Ginac/v1.7.3-1

Website: https://www.ginac.de/

GSL v1.16

The GNU Scientific Library (or GSL) is a software library for numerical computations in applied mathematics and science. The GSL is written in C; wrappers are available for other programming languages.

Versions: v1.16-1, v2.6
To set environmental variable, please use the following command:

module add gsl/v2.6

Website: https://www.gnu.org/software/gsl/

PostgreSQL

PostgreSQL is an object-relational database management system (ORDBMS) with an emphasis on extensibility and standards compliance.
Version: 12.1
To set environmental variable, please use the following command:

module add PostgreSQL/v12.1_gcc910

Website: https://postgresql.org/

Protocol Buffers

Protocol Buffers is a method of serializing structured data developed by Google.
Version: 3.11.3
To set environmental variable, please use the following command:

module add protobuf/v3.11.3

Website: https://developers.google.com/protocol-buffers/

Quantum ESPRESSO

Quantum opEn-Source Package for Research in Electronic Structure, Simulation, and Optimization is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials.
Version: 6.4.1, 6.6_oneapi
To set environmental variable, please use the following command:

module add quantum-espresso/v6.4.1

Website: http://www.quantum-espresso.org/

TRNG

Tina’s Random Number Generator Library is a state of the art C++ pseudo-random number generator library for sequential and parallel Monte Carlo simulations.
Version: 4.24
To set environmental variable, please use the following command:

module add trng/v4.24_gcc102

Website: https://www.numbercrunch.de/trng/

zlib

Free cross-platform data compression library.
Version: 1.2.11
To set environmental variable, please use the following command:

module add zlib/v1.2.11-1

Website: http://www.zlib.net/


Intel Software Development Tools

Aplication Performance Snapshot

Aplication Performance Snapshot is a tool for application profiling. This application is included to the Intel Parallel Studio installed on the HybriLIT cluster.
Aplication Performance Snapshot allows analyze such parameters as memory usage, work time of each process, operating delay, etc.

In order tp profile an application, it’s necessary to:

  1. Load a module that provides support of envarinmental variables for work with Intel Parallel Studio:
$ module add intel/v2019.0.018
  1. Compile the application by means of loading a library:

For MPI programs

$ mpiicc name_app.cpp

for OpenMP programs

$ icc –qopenmp name_app.cpp

or

$ icc name_app.cpp

for a sequential program where name_app.cpp – is the name of the compiled file.

  1. Run the executable file (by default a.out) with the key – aps, or using a script-file:

For MPI programs

#!/bin/sh
#SBATCH -p cpu
#SBATCH -n 4
#SBATCH -t 600
mpirun aps ./a.out

for OpenMP programs

#!/bin/sh
#SBATCH -p cpu
#SBATCH -n 4
#SBATCH -t 600
export OMP_NUM_THREADS=5
export OMP_PLACES=cores
aps ./a.out

for a sequential program

#!/bin/sh
#SBATCH -p cpu
#SBATCH -n 1
#SBATCH -t 600
aps ./a.out
  1. All received data will be placed in the folder aps_result_[launch date]. In order to display these data, generate a report in HTML (pic 1, pic 2) using the following command:
$ aps -R aps_result_ [дата запуска]

Pic.1 . Screenshot of an output-file of Aplication Perfomance Snapshot for MPI application (click the picture to see it at the report page)

Pic. 2. Screenshot of an output-file of Aplication Perfomance Snapshot for OpenMP application (click the picture to see it at the report page)

Intel Trace Analyzer and Collector

Intel Trace Analyzer and Collector is aimed at tracing MPI processes..

Main tasks:

  • Behavior vizualization of a parallel application;
  • Estimation of statistic profile and load balancing;
  • Performance analysis of programs and code blocks;
  • Detailing of model of exchange and performance data;
  • Detection of a hotspot;
  • Decrease of execution time and increase of application efficiency.

To launch an application use parameter –trace. Please see an example below:

#!/bin/sh
#SBATCH -p cpu
#SBATCH -n 4
#SBATCH -t 600
mpirun -trace ./a.out

To view results, please use ITAC GUI by means of the command:

$ traceanalyzer

As a result, an application winsow will be opened and you will need to upload a file (a.out.stf by default). An examples is shown on Pic. 3.

Distribution of computational load on MPI processes and interprocess communication (Event Timeline scale) are shown at the top of Pic.3.; numeric values are shown at the bottom of Pic.3.

itac1

Pic. 3. Examples of communication in an MPI application.

By means of Message Profile, it is now possible to estimate which processes comunicate and which communications are the most time consuming (Pic. 4.).

itac2

Pic. 4. Interprocess communication in an MPI application.

In Collective operations page, it is possible to prepare a table that reflects total span time for collective operations in MPI application (Pic.5.).

Collective_operations-2

Рис. 5. Таблица временных затрат на коллективные операции в MPI-приложении.

Conclusion
Application Performance Snapshot —  is aimed at fast estimation of efficiency;  it doesn’t require additional charges; provides profiling up to 32000 MPI processes; allows get fasr estimation of MPI and OpenMP disbalance; and provides total estimation of performance (GFLOPS).
Intel Trace Analyzer and Collector — allows carrying out detailed analysis of MPI applications, detect communication patterns, and locate specific bottle neck of programs.

Open Visual Inference & Neural Network Optimization

OpenVINO (Open Visual Inference и Neural Network Optimization) is a free set of tools that simplifies the optimization of the deep learning model from the infrastructure and deployment using the output mechanism on Intel hardware.

The Movidius Neural Compute Stick (NCS Movidius)is a miniature accelerator for solutions related to artificial intelligence, neural networks and deep learning. The accelerator allows implementing elements of artificial intelligence on various platforms and devices. The given device is not designed for training neural network models.

The major advantage of the product is its performance, minimum size and almost zero number of dependencies. The product is ideal for implementing applications that use Deep Learning and Computer Vision to solve problems. Its disadvantage is that support for training neural networks is not included in the product.

This manual describes the steps to install and configure the tool OpenVINO Tools for Windows and Linux, as well as the work with the model optimizer.

For output to Inference Engine, read the documentation specifically for your case: Inference Engine Developer Guide


Quantum computing tools

Intel-QS

Intel Quantum Simulator (Intel-QS), also known as qHiPSTER (The Quantum High Performance Software Testing Environment), is a simulator of quantum circuits optimized to take maximum advantage of multi-core and multi-nodes architectures. It is based on a complete representation of the qubit state, but avoids the explicit representation of gates and other quantum operations in terms of matrices. Intel-QS uses the MPI (message-passing-interface) protocol to handle communication between the distributed resources used to store and manipulate quantum states.

To set environmental variable, please use the following command:

module add intel-qs/v20-07-14
module add intel-qs/v21-01-14

Website: https://github.com/iqusoft/intel-qs

QuEST

The Quantum Exact Simulation Toolkit is a high-performance simulator of quantum circuits, state-vectors and density matrices. QuEST uses multithreading, GPU acceleration and distributed computing systems.

Website: https://quest.qtechtheory.org/