Cluster Course - 250213
Cluster Course - 250213
This course will consolidate material presented in the beginner cluster course and expand on the concepts to be aware of when trying to optimize use of the cluster.
If you have never used the CRG cluster, this is your mandatory course to obtain the user license. If you are already using the CRG Cluster and need an update, you will need to take the this course.
The main message of the course is to embrace the parallelism available within the cluster and that pipelines should be made from lots of small independent pieces that are spread throughout the cluster rather than large monolithic long jobs that run on a single node. The course will show why this should be done and how to achieve it.
Topics that are going to be addressed:
- Video tour of the data centre
- What is a cluster
- Logging in
- Queuing / the scheduler
- What resource are available at the CRG cluster
- Simple batch scripts - directives
- Troubleshooting - what happened to my jobs?
- Interactive sessions
- Supercomputers, beowulf clusters, horizontal v vertical scaling
- Hardware considerations
- Multithreaded jobs, parallelism, Amdahl's Law
- Job arrays
- Job dependencies
- Building a pipeline
- Storage issues, treemap
- Job stats, resource estimation
- Scaling analysis
What NOT to expect:
Specific bioinformatics methods, pipeline builders (nextflow, snakemake etc.)
Instructors and teachers: Emyr James, Head of SIT, and SIT co-trainers: Emilio Palumbo, Rodny Hernandez, Gabriel Gonzalez, Luis Exposito, Clemente Borges
Dates: 13th, 17th , 20th and 25th of February 2025 from 10:30-13:30h
Level: You have to have a minimum level of Linux experience OR have taken the Linux Terminal for beginners course. The level of this course is intermediate-advanced
Location: Bioinformatics room, CRG Training Centre
Maximum number of participants: 16
Registration deadline: 6th February 2025
Registration HERE
For any information, please send an email to CRG Training and Academic office (TAO): training@crg.eu
