|
|
---
|
|
|
title: Configuring a standard experiment for NIC5
|
|
|
---
|
|
|
Thanks to GitLab CI/CD features and thanks to [Jacamar CI](https://ecp-ci.gitlab.io/docs/admin/jacamar/introduction.html), you can configure a standard experiment for BAMHBI to run on the NIC5 cluster which you will be able to kickstart from ULiège GitLab with a simple click.
|
|
|
Thanks to GitLab CI/CD features and thanks to [Jacamar CI](https://ecp-ci.gitlab.io/docs/admin/jacamar/introduction.html), you can configure a standard experiment for BAMHBI to run on a cluster which you will be able to kickstart from ULiège GitLab with a simple click. For the time being, all CI/CD experiments of the MAST will be run on the NIC5 cluster, notably to benefit from the many MAST resources already stored on NIC5.
|
|
|
|
|
|
This tutorial details the requirements you should meet beforehand, for both security and storage concerns, then how you can write a suitable CI/CD configuration to create your experiment.
|
|
|
|
|
|
## Requirements
|
|
|
|
|
|
### Requirements to satisfy CÉCI (cluster administrators)
|
|
|
### Security concerns of the CÉCI (cluster administrators)
|
|
|
|
|
|
The CÉCI (Consortium des Équipements de Calcul Intensif) expects us to make reasonable use of their computing resources. I.e., the experiments we run through CI/CD should not compromise their security, in addition to having (as much as possible) a small footprint in terms of disk space. This is why pre-configured experiments written as CI/CD jobs on GitLab will only be possible if the following conditions are met.
|
|
|
The CÉCI (Consortium des Équipements de Calcul Intensif) expects us to make reasonable use of their cluster(s) through GitLab. I.e., the experiments we run through CI/CD should not compromise their security, in addition to avoid abusing their computer resources (i.e., just as with your regular jobs). This is why pre-configured experiments written as CI/CD jobs on GitLab will only be possible if the following conditions are met.
|
|
|
|
|
|
In the next lines, the repository where the BAMHBI code you want to use is located is called the **target repository**.
|
|
|
|
... | ... | @@ -17,17 +17,30 @@ In the next lines, the repository where the BAMHBI code you want to use is locat |
|
|
|
|
|
2) The contributors to the target repository who want to perform CI/CD through NIC5 **must have a CÉCI account**.
|
|
|
|
|
|
3) The **target repository must have a _custom runner_ for NIC5**. You can check if a such a runner exists by browsing **Settings > CI/CD > Runners** while in the target repository. It should be listed under _Assigned project runners_.
|
|
|
3) The **target repository must have a custom _runner_ for NIC5**. You can check if a such a runner exists by browsing **Settings > CI/CD > Runners** while in the target repository. It should be listed under _Assigned project runners_.
|
|
|
|
|
|
* If such a runner does not exist yet, you will have to contact me (@Jean-Francois.Grailet) first so that I can reach the CÉCI to create the runner and interface it with NIC5 through [Jacamar CI](https://ecp-ci.gitlab.io/docs/admin/jacamar/introduction.html).
|
|
|
|
|
|
4) Finally, **you must advertise me whether you want to use the NIC5 runner** of the target repository.
|
|
|
4) Finally, **you must tell me whether you want to use the NIC5 runner** of the target repository.
|
|
|
|
|
|
* This is because the CÉCI needs to be able to map a GitLab user to a CÉCI account (hence requirement 2) for each runner.
|
|
|
* Contact me (@Jean-Francois.Grailet) by e-mail for this purpose.
|
|
|
* This additional requirement is again motivated by security concerns at CÉCI's side, as they have to be able to map a GitLab user to a CÉCI account (hence requirement 2).
|
|
|
|
|
|
### Requirements to satisfy SEGI (ULiège IT management)
|
|
|
### Storage concerns
|
|
|
|
|
|
The SEGI (SErvice Général d'Informatique), who manages the ULiège network and its services, including GitLab, also expects us to make good use of their resources. Hopefully, they worry mostly about the disk space used by GitLab projects, so their requirements are not as strict as CÉCI's, but should be taken into consideration too.
|
|
|
There are two solutions for retrieving products of your CI/CD experiment running on a cluster such as NIC5. On the one hand, you can store them directly on the cluster at a selected location, if you have enough disk space for that. On the other hand, you can download them as _job artifacts_, i.e., files that amount essentially as the outputs of a CI/CD job.
|
|
|
|
|
|
**Coming soon** |
|
|
\ No newline at end of file |
|
|
The SEGI (SErvice Général d'Informatique), who manages the ULiège network and its services, including GitLab, also expects us to make good use of the disk space provided for GitLab projects. In other words, **you should be careful with the size of your _job artifacts_**, if you use any.
|
|
|
|
|
|
1) You should **be able to quantify the volume of data you want to produce and use**. By default, jobs artifacts are restricted to 100 Mo, but this limit can be risen by SEGI for specific repositories if needed. Knowing in advance how much data you want to get from your CI/CD pipeline will help in configuring such a limit.
|
|
|
|
|
|
* 100 Mo is enough to produce multiple figures and small diagnostic files, but is insufficient if you need, for instance, daily outputs for specific variables from the model.
|
|
|
* The SEGI can rise the maximum size of artifacts for a specific GitLab repository. You can also contact me (@Jean-Francois.Grailet) to give me a desired limit I will negotiate with SEGI.
|
|
|
|
|
|
2) You should **delete the (large) artifacts of the repository when they are no longer useful**. By default, the SEGI deletes all artifacts older than one month, but GitLab users not being able to configure the length of the period during which artifacts are kept by default, you are kindly asked to clean after yourself in **Build > Artifacts** while in the target repository, especially if you repeatedly run your standard experiment(s) for a short period.
|
|
|
|
|
|
### Repositories already configured for NIC5
|
|
|
|
|
|
The following repositories are already configured to enable experiments with the help of a NIC5 runner. This means, in practice, that you should only check that you have a CÉCI account and tell me you want to be granted access to the runner of the repository.
|
|
|
|
|
|
* [ESPECES/MAST/Nemo4.2.0-Bamhbi](https://gitlab.uliege.be/especes/mast/nemo4.2.0-bamhbi) (max. artifacts size: 0.5 Go) |
|
|
\ No newline at end of file |