... | ... | @@ -44,3 +44,62 @@ The SEGI (SErvice Général d'Informatique), who manages the ULiège network and |
|
|
The following repositories are already configured to enable experiments with the help of a NIC5 runner. This means, in practice, that you should only check that you have a CÉCI account and tell me you want to be granted access to the runner of the repository.
|
|
|
|
|
|
* [ESPECES/MAST/Nemo4.2.0-Bamhbi](https://gitlab.uliege.be/especes/mast/nemo4.2.0-bamhbi) (max. artifacts size: 0.5 Go)
|
|
|
|
|
|
## Configuring your CI/CD jobs for NIC5
|
|
|
|
|
|
**The main advantage of [Jacamar CI](https://ecp-ci.gitlab.io/docs/admin/jacamar/introduction.html) is being able to write your CI/CD jobs as if you were already logged in on the cluster.** This means that you do not have to worry about any file transfer between GitLab and the cluster. In particular, your cluster job will start by cloning the target repository locally, which means that your repository files are reachable at soon as the script of your CI/CD job starts.
|
|
|
|
|
|
Before going any further, it's strongly recommended to review [GitLab tutorials for CI/CD](https://docs.gitlab.com/ci/quick_start/) to get at least familiar with the syntax of a CI/CD configuration.
|
|
|
|
|
|
### Basis of a NIC5 job
|
|
|
|
|
|
1) As always, you will start by editing the ``.gitlab-ci.yml`` file at the root of your repository, or create it if doesn't exist yet.
|
|
|
|
|
|
2) Let's write the new job. It must be written so that it will always use the NIC5 runner configured for your repository. For starters, you need to provide the path to the repository as a token, as follows.
|
|
|
|
|
|
```
|
|
|
my_nic5_job:
|
|
|
stage: some-stage
|
|
|
id_tokens:
|
|
|
SITE_ID_TOKEN:
|
|
|
aud: https://gitlab.uliege.be/especes/mast/nemo4.2.0-bamhbi
|
|
|
...
|
|
|
```
|
|
|
|
|
|
3) Next, you must provide tags mapped to the runner. These tags will enforce GitLab to always use the NIC5 runner when running your new job. The tags mapped to the runner are enumerated below itself under _Assigned project runners_ in **Settings > CI/CD > Runners** (while in the project). In the case of the [Nemo4.2.0-Bamhbi](https://gitlab.uliege.be/especes/mast/nemo4.2.0-bamhbi) project, the tags are _nic5_, _compute_ and _slurm_. They can be added as follows.
|
|
|
|
|
|
```
|
|
|
my_nic5_job:
|
|
|
stage: some-stage
|
|
|
id_tokens:
|
|
|
SITE_ID_TOKEN:
|
|
|
aud: https://gitlab.uliege.be/especes/mast/nemo4.2.0-bamhbi
|
|
|
tags:
|
|
|
- nic5
|
|
|
- compute
|
|
|
- slurm
|
|
|
...
|
|
|
```
|
|
|
|
|
|
4) Now, you must provide the cluster resources you want to use, e.g., how many CPUs, tasks and memory per CPU. This can be done by adding a ``variables`` keyword followed by ``SCHEDULER_PARAMETERS``, itself followed by your job requirements.
|
|
|
|
|
|
```
|
|
|
my_nic5_job:
|
|
|
stage: some-stage
|
|
|
id_tokens:
|
|
|
SITE_ID_TOKEN:
|
|
|
aud: https://gitlab.uliege.be/especes/mast/nemo4.2.0-bamhbi
|
|
|
tags:
|
|
|
- nic5
|
|
|
- compute
|
|
|
- slurm
|
|
|
variables:
|
|
|
SCHEDULER_PARAMETERS: "--ntasks=114 --cpus-per-task=1 --mem-per-cpu=1024 --time=2:00:00
|
|
|
...
|
|
|
```
|
|
|
|
|
|
### Recommended additional keywords
|
|
|
|
|
|
By step 4 in the previous section, the job is normally already good to run on NIC5. However, it's **strongly recommended** to add a few additional keywords to have a better control of your job.
|
|
|
|
|
|
* Make sure to **include ``when: manual`` in the very first stage of your pipeline**. This will ensure your pipeline will start only when you ask, and not at every commit. This matters especially at MAST, as there may be eventually multiple pre-configured standard experiments, and we don't want, obviously, to re-run all of them at each commit. |
|
|
\ No newline at end of file |