Skip to content

Changes

Page history
add FAQ VPN connection + diverse modification authored by Bouquieaux Marie-Catherine's avatar Bouquieaux Marie-Catherine
Hide whitespace changes
Inline Side-by-side
mass-storage/mass-storage-structure-organisation.md
View page @ 9428ded9
......@@ -6,13 +6,13 @@ The mass storage structure is designed to enable efficient organisation of data
# Tree structure
The storage space is divided into three main entities: Administration (ADM), Platforms (PTF) and Research Units (URT). These entities are subdivided in folders and subfolders following a hierarchical structure that reflects the organisation of the administrative teams, the research groups and the platforms in the GIGA (see [table below](mass-storage/mass-storage-structure-organisation#tree-structure)).
The storage space is divided into three main entities: Administration (ADM), Platforms (PTF) and Research Units (URT). These entities are subdivided in folders and subfolders following a hierarchical structure that reflects the organisation of the administrative teams, the research teams and the platforms in the GIGA (see [table below](mass-storage/mass-storage-structure-organisation#tree-structure)).
In the Research entity (URT) for example, the main folder is subdivided into one subfolder per GIGA thematic Units, themselves subdivided into one folder per laboratory in the unit. These Laboratory folders can be subdivided further into Group folders that can themselves be subdivided into Project folders. Project folders are accessible only to users working on a specific project. If needed, a Platform-shared folder can be created in a given Group folder. This folder will be used by the platforms to provide data generated through their services.
In the Research entity (URT) for example, the main folder is subdivided into one subfolder per GIGA thematic Units, themselves subdivided into one folder per laboratory in the unit. These Laboratory folders can be subdivided further into Team folders that can themselves be subdivided into Project folders. Project folders are accessible only to users working on a specific project. If needed, a Platform-shared folder can be created in a given Team folder. This folder will be used by the platforms to provide data generated through their services.
Beside these 3 main entities (ADM, PTF and URT), there is a Resources folder containing resources available to all users of the mass storage, such as Reference Genomes and Singularity Containers. In addition, each user also has a personal folder space called Home.
Here is a schematic representation of the tree. For clarity, not all Units, Laboratories, Groups and Projects folders are represented. A complete tree can be found [here](mass-storage/mass-storage-full-structure-home).
Here is a schematic representation of the tree. For clarity, not all Units, Laboratories, Teams and Projects folders are represented. A complete tree can be found [here](mass-storage/mass-storage-full-structure-home).
```mermaid
......@@ -31,7 +31,7 @@ graph TB
CC --> sup1(...)
end
subgraph sub3[ ]
F --- GG(Groups)
F --- GG(Teams)
GG --> Q(BIOINFO)
GG --> R(NGS)
GG --> fsub(GENO)
......@@ -51,7 +51,7 @@ graph TB
HH --> sup3(...)
end
subgraph sub6[ ]
L --- LL(Groups)
L --- LL(Teams)
LL --> M(BLV)
LL --> N(FACT)
LL --> O(IBD)
......@@ -115,7 +115,7 @@ graph TB
# User home folder
Each GIGA member is given a personal “home” space on the mass storage and will be granted access to group and project folder(s) by their PI through the GIGA XENIA platform.
Each GIGA member is given a personal “home” space on the mass storage and will be granted access to team and project folder(s) by their PI through the GIGA XENIA platform.
The home folder serves as access point to all other folders this member has access to on the mass storage infrastructure.
The home folder is organized with a \_SHARE_ folder containing links to all shared folders the user has access to (Resources, Projects and Platform-shared folders).
Here is a schematic view of a hypothetical user123456’s home folder:
......@@ -148,29 +148,29 @@ graph TB
```
This personal space is currently limited to 100 GB per user (not including the volume of the data stored in the shared folders the user can reached through the links). However, to prevent duplication and/or loss of precious material, users should always favour storing data and results relative to a research project only in the corresponding project folder (see below).
This personal space is currently limited to 100 GB per user. All files in the user's personnal folder and subfolders are included in this quota, but not the files the user can reach through the "Research", "Platforms", "Administration" or "Resources" links in the ""\_SHARE\_" folder. Of note, to prevent duplication and/or loss of precious material, users should always favour storing data and results relative to a research project only in the corresponding project folder (see below).
Upon departure from GIGA, a member must take care of their home folder by cleaning its content and/or by transmitting valuable files to their PI along with pertinent documentation. After departure of the user, the user's home will be deleted and all files not transferred to the group common area will be lost. It is the PI responsibility to ensure that such procedure is followed before the deletion of the home directory of the former user.
Upon departure from GIGA, a member must take care of their home folder by cleaning its content and/or by transmitting valuable files to their PI along with pertinent documentation. After departure of the user, the user's home will be deleted and all files not transferred to the team's common area will be lost. It is the PI responsibility to ensure that such procedure is followed before the deletion of the home directory of the former user.
# Research group and project folders
# Research teams and project folders
Each GIGA member is by definition part of a research group and is working on one or multiple projects. To enable the sharing of data and results between members working on the
same project, the mass storage infrastructure also contains group and project folders. Group and project folders are created upon request by a PI. Every group folder and the
data/results contained within it are under the responsibility of this PI. Upon assignment to a project/group, GIGA members need to be granted access to the corresponding group
Each GIGA member is by definition part of a research team and is working on one or multiple projects. To enable the sharing of data and results between members working on the
same project, the mass storage infrastructure also contains team and project folders. Team and project folders are created upon request by a PI. Every team folder and the
data/results contained within it are under the responsibility of this PI. Upon assignment to a project/team, GIGA members need to be granted access to the corresponding team
folder on the mass storage. Folder privileges are set up and modified by UDI GIGA-MED IT specialists, based on PI instructions. In a near future, management of folder privileges
by the PI will be facilitated by the GIGA XENIA application.
Upon request by a PI, UDI GIGA-MED IT specialists can also create a SHARE folder that will allow people from the same group but working on different project to share data. In the
Upon request by a PI, UDI GIGA-MED IT specialists can also create a SHARE folder that will allow people from the same team but working on different project to share data. In the
same way, UDI GIGA-MED IT specialists can also create a PLATFORMS folder accessible to the platforms for providing data or images generated by their activities in the context of
the corresponding projects (see Platform-shared folders section below).
Space available in groups and projects folders belonging to a given laboratory is limited by a quota. Data from the associated platform-shared folders (see below) are included in this quota. For more information, see the ["Quota section"](mass-storage/mass-storage-quota).
Space available in teams and projects folders belonging to a given laboratory is limited by a quota. Data from the associated platform-shared folders (see below) are included in this quota. For more information, see the ["Quota section"](mass-storage/mass-storage-quota).
# Platform-shared folders
Data generated by the different GIGA platforms can be shared with the members of a group via a PLATFORMS folder situated inside the group folder. This PLATFORMS folder is created upon request by the PI. It will contain one subfolder per platform and can be subdivided further according to needs. GIGA platforms will only have reading/writing privileges to this shared folder, with no access to the associated project folders.
Data generated by the different GIGA platforms can be shared with the members of a team via a PLATFORMS folder situated inside the team folder. This PLATFORMS folder is created upon request by the PI. It will contain one subfolder per platform and can be subdivided further according to needs. GIGA platforms will only have reading/writing privileges to this shared folder, with no access to the associated project folders.
Users can either leave all data transferred by the platforms in the PLATFORMS folder or move them to the corresponding Project folder. Of note, most journals will ask for these raw data to be uploaded on a server before to accept any publication relative to them, so it’s extremely important to organise and annotate them properly to be able to retrieve them when needed. To avoid having multiple copies of large files, the platforms won’t keep a copy of these raw data. It’s the PI responsibility to make sure precious files are
stored and annotated correctly.
......@@ -178,7 +178,7 @@ stored and annotated correctly.
Because raw data are so precious, by default users won't be able to move or modify data shared by the Genomics platform. Users can create symlinks to these files in any other location if they want to have access to them from there. They will need to ask the platform if they want to delete the data or to move them somewhere else.
The whole platform-shared folder is considered to be part of the group folder. Hence, it is included in the calculation of the size quota and its content is under the responsibility of the group folder PI.
The whole platform-shared folder is considered to be part of the team folder. Hence, it is included in the calculation of the size quota and its content is under the responsibility of the team folder PI.
# Summary
......@@ -189,8 +189,8 @@ The whole platform-shared folder is considered to be part of the group folder. H
| Home | Access point to server + storage of documents not linked to a specific project | By default | user |
| Resources | Reference Genomes, Singularity containers and other shared resources | By default | general |
| Projects | Data and analysis related to a specific project, accessible via the Research link in the user's home, following the structure detailed [here](mass-storage/mass-storage-URT) | Upon request | laboratory |
| SHARE | Data shared by members of a group working on different projects, accessible via the Research link, in the group folder, next to the Project folders | Upon request | laboratory |
| PLATFORMS | Data shared by the platforms, accessible via the Research link, in the group folder, next to the Project folders | Upon request | laboratory |
| SHARE | Data shared by members of a team working on different projects, accessible via the Research link, in the team folder, next to the Project folders | Upon request | laboratory |
| PLATFORMS | Data shared by the platforms, accessible via the Research link, in the team folder, next to the Project folders | Upon request | laboratory |
#### Folders to which users who are part of a platform have access to:
......@@ -198,13 +198,13 @@ The whole platform-shared folder is considered to be part of the group folder. H
| --------- | ------------------------------------------------------------ | ------------ | ---------- |
| Home | Access point to server + storage of documents not linked to a specific project | By default | user |
| Resources | Reference Genomes, Singularity containers and other shared resources | By default | general |
| Group | Each platform has groups folders where to store data they are processing. They are accessible via the Platforms link in the user's home, following the structure detailed [here](mass-storage/mass-storage-PTF) | Upon request | platform |
| PLATFORMS | Data shared with members of research groups, accessible via the Research link, in the group folder, next to the Projects folder | Upon request | laboratory |
| Team | Each platform has teams folders where to store data they are processing. They are accessible via the Platforms link in the user's home, following the structure detailed [here](mass-storage/mass-storage-PTF) | Upon request | platform |
| PLATFORMS | Data shared with members of research teams, accessible via the Research link, in the team folder, next to the Projects folder | Upon request | laboratory |
#### Folders permissions:
| Folder | Research User | User on the same project | User of same group but different project | User from another group | Platform members |
| Folder | Research User | User on the same project | User of same team but different project | User from another team | Platform members |
|----------------|----------------------|--------------------------|------------------------------------------|-----------------------|----------------------|
| Home | read, write, execute | no access | no access | no access | no access |
| Resources | read, execute | read, execute | read, execute | read, execute | read, write, execute |
......