Computational Biology Lab Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
##Login and sending jobs to CBLab

The Computational Biology Lab cluster is a set of computers and programs that let make simultaneous calculations. We can divide this set in two big groups, the nodes (that make all the calculations) and the support system.
All the applications of the cluster run in a Bio·Linux8 (Ubuntu 14.04LTS).

User data are stored in the ''/home/usuaris/<user\_name>'' directory and we can find most of the programs that run in the cluster in ''/home/soft/<program>'' directory. There is a ''/home/db'' directory where we want to store all the databases necessary for user's work and others that can be useful for all users.

By default, a maximum 4GB RAM and one processor (from 284 installed) are assigned to each process sended, but we will see how to modify this.

You can download a presentation here: 
{{ :cblab_user_guide.pdf | CBLab tutorial (Español)}}


###How to log in

We use a ssh connection to the cluster. For Linux and Mac we simply write in the terminal:

```
ssh nom_usuari@cluster-ceab.ceab.csic.es
```

Windows' users can login too using [[http://www.putty.org/|putty]], [[http://sils.unc.edu/it-services/servers/using-ssh|ssh-client]], etc.

If the user wants to upload/download files we recommend [[https://filezilla-project.org/|filezilla]] or from the terminal using scp.

###How to send jobs to the cluster

The first thing we have to do is to create a plaintext file (e.g. a .txt file, and NO html, .docx etc) where we will write the commands or instructions (a script) with all the options and also the path of the file or files that the cluster will need to work.

```bash

#!/bin/bash

### Codi R

cd  ~/research/gTiger/tiger_risk_estimation/scripts

/home/soft/R-3.2.1/bin/R CMD BATCH --no-save --no-restore t002.0_sampling_effort_overlay_rsample.r t002.0_sampling_effort_overlay_rsample.out
```


We could complete our script adding other options that give us useful information:

''**<**'' <input\_file\_name>

Standard Input

''**>**'' <output\_file\_name>

Standard Output, this file will contain all the output generated by the program.

''**2>**'' <error\_file\_name>

Standard Error, if the job fails it will give us information about errors.

Written in our script it would look like this:

```bash

#!/bin/bash

### Codi R

(...)

/home/soft/program_name < ''input_file_name'' > ''output_file_name'' 2> ''nom_fitxer_error''

```

Once the script is written we save it as a ''.sh'' file. Make sure all the files that the cluster will need are in the path you have indicated previously.

From the console, we will send our job using the command qsub 
<code bash>
qsub script_name.sh
</code>

By default (with no options), a maximum 4GB RAM and one processor (from 284 installed) are assigned to each process sended. Now we will see how to modify this.
Also, each job sended, when finished, it will generate two files ''<script.name>.o<job\_id>'' (output file) and ''<script.name>.e<job\_id>'' (error file, empty if there is no errors).

###qsub options

These are some useful options:

**qsub -l h_vmem=<x>G**

This one will let us assign a different amount of RAM. '''Attention''' RAM is assigned per core (processor) not for job.

```
qsub -l h_vmem=10G ... <script_name>.sh
```

**qsub -pe make <n_processors>**

With this option we will activate the parallelization environment 'make'. It is an intra-node parallelization, so the maximum number of processors that we can use is limited by hardware (64 cores max). 
Exemple:
```
qsub -pe make 10 <script_name>.sh
```

e.g. We want to send a job with 20 cores and we want to assign 100G RAM, then:

```
qsub -pe make 20 -l h_vmem=5G <script_name>.sh
```

Please, consider that assigning more RAM or processors than necessary will "block" resources that other users could use.

**qsub -m bea -M <user_mail>**

Using this option we will receive a mail at the beginning and at the end or when we abort the sended job.

```
qsub -m bea -M x.roijals@ceab.csic.es
```
**qsub -q ceab@nodexxx**

Send a job to a chosen node where xxx is the number of the node (100 to 112)

**qsub -e error -o output**

By default these files are created, with this option we can choose the name of them. Just write this:

```
qsub -e <error_file_name.txt> -o <output_file_name.txt> <script.name>.sh
```

You can add all these option in one:

```
qsub qsub -pe make 10 -l h_vmem=10G -M <an_email> -m bea -q ceab@nodexxx <script_name>.sh
```
###Login in a node

Sometimes it can be useful to enter in a node where a job is running or for other reasons. We use the command qlogin for this. It is choose at random with no options and at least one processor of the node has to be free. You can use the same commands of the qsub too.
In this case you don't need to indicate a script.sh

```
qlogin <options>
```

###Check job status

If we want to know which of our jobs are runnning in the cluster we use the command qstat. It show us the job ID and the node where it is running.

```
qstat
```

To know other user's job:

```
qstat -u <login_user_name>
```

Or all users jobs at the same time:
```
qstat -u "*"
```

If we want more information about one particular job

```
qstat -j <job_ID>
```

If we want to know the number of cores are running jobs in the nodes:

```
xavier.roijals@cluster-ceab:/home/soft$ qstat -f
queuename                      qtype resv/used/tot. load_avg arch          states
---------------------------------------------------------------------------------
ceab@node100                   BIP   0/0/16         0.08     linux-x64     
---------------------------------------------------------------------------------
ceab@node101                   BIP   0/16/16        15.87    linux-x64     
---------------------------------------------------------------------------------
ceab@node102                   BIP   0/0/16         0.08     linux-x64     
---------------------------------------------------------------------------------
ceab@node103                   BIP   0/0/16         1.01     linux-x64     d
---------------------------------------------------------------------------------
ceab@node104                   BIP   0/0/16         0.05     linux-x64     
---------------------------------------------------------------------------------
ceab@node105                   BIP   0/0/20         0.00     linux-x64     
---------------------------------------------------------------------------------
ceab@node106                   BIP   0/20/20        19.84    linux-x64     
---------------------------------------------------------------------------------
ceab@node107                   BIP   0/0/20         0.00     linux-x64     
---------------------------------------------------------------------------------
ceab@node108                   BIP   0/0/20         0.02     linux-x64     
---------------------------------------------------------------------------------
ceab@node109                   BIP   0/10/20        10.03    linux-x64     
---------------------------------------------------------------------------------
ceab@node110                   BIP   0/62/64        3.97     linux-x64     
---------------------------------------------------------------------------------
ceab@node111                   BIP   0/6/20         5.97     linux-x64     
---------------------------------------------------------------------------------
ceab@node112                   BIP   0/0/20         0.00     linux-x64     
```

Maybe we want a more detailed report:

```
xavier.roijals@cluster-ceab:/home/soft$ qstat -f -u "*"
queuename                      qtype resv/used/tot. load_avg arch          states
---------------------------------------------------------------------------------
ceab@node100                   BIP   0/0/16         3.21     linux-x64     
---------------------------------------------------------------------------------
ceab@node101                   BIP   0/16/16        9.28     linux-x64     
  89406 0.58975 Rsnow      jgarriga     r     01/19/2021 16:47:07    16        
---------------------------------------------------------------------------------
ceab@node102                   BIP   0/0/16         3.20     linux-x64     
---------------------------------------------------------------------------------
ceab@node103                   BIP   0/0/16         1.00     linux-x64     d
---------------------------------------------------------------------------------
ceab@node104                   BIP   0/1/16         2.96     linux-x64     
  89391 0.50500 QLOGIN     fbartu       r     01/19/2021 13:29:51     1        
---------------------------------------------------------------------------------
ceab@node105                   BIP   0/0/20         0.04     linux-x64     
---------------------------------------------------------------------------------
ceab@node106                   BIP   0/20/20        11.64    linux-x64     
  89406 0.58975 Rsnow      jgarriga     r     01/19/2021 16:47:07    20        
---------------------------------------------------------------------------------
ceab@node107                   BIP   0/1/20         0.07     linux-x64     
  89405 0.50500 QLOGIN     m.pardo      r     01/19/2021 16:30:07     1        
---------------------------------------------------------------------------------
ceab@node108                   BIP   0/0/20         0.79     linux-x64     
---------------------------------------------------------------------------------
ceab@node109                   BIP   0/10/20        6.19     linux-x64     
  88997 0.50500 FEElnc_spu c.pegueroles r     12/19/2020 11:38:07     1        
  89406 0.58975 Rsnow      jgarriga     r     01/19/2021 16:47:07     9        
---------------------------------------------------------------------------------
ceab@node110                   BIP   0/62/64        3.94     linux-x64     
  88954 0.50500 QLOGIN     j.palmer     r     12/17/2020 13:31:41     1        
  89348 0.60500 scktjob    rlloret      r     01/17/2021 08:48:01    60        
  89387 0.50500 R          pol.fernande r     01/19/2021 13:16:53     1        
---------------------------------------------------------------------------------
ceab@node111                   BIP   0/6/20         3.74     linux-x64     
  89406 0.58975 Rsnow      jgarriga     r     01/19/2021 16:47:07     6        
---------------------------------------------------------------------------------
ceab@node112                   BIP   0/0/20         0.00     linux-x64     
```

### How to know if a job is parallelizing (or see the number of running cores in a node):

```
$ export TERM=xterm;htop
```

or if we want:

```
$ export TERM=xterm 
```

and then:

```
$ htop
```



###"Kill" jobs

If we want to end a job before it is finished:

```
qdel <job_ID>
```