Skip to content

Changes

Page history
creation new wiki authored by Bouquieaux Marie-Catherine's avatar Bouquieaux Marie-Catherine
Hide whitespace changes
Inline Side-by-side
mass-storage/mass-storage-structure-organisation.md 0 → 100644
View page @ b6a1a7e8
**Table of contents**
[[_TOC_]]
The mass storage structure is designed to enable efficient organisation of data and data sharing between collaborators within defined groups.
# Tree structure
The storage space is divided into three main entities: Administration (ADM), Platforms (PTF) and Research Units (URT). These entities are subdivided in folders and subfolders following a hierarchical structure that reflects the organisation of the administrative teams, the research groups and the platforms in the GIGA (see [table below](mass-storage/mass-storage-structure-organisation#tree-structure)).
In the Research entity (URT) for example, the main folder is subdivided into one subfolder per GIGA thematic Units, themselves subdivided into one folder per laboratory in the unit. These Laboratory folders can be subdivided further into Group folders that can themselves be subdivided into Project folders. Project folders are accessible only to users working on a specific project. If needed, a Platform-shared folder can be created in a given Group folder. This folder will be used by the platforms to provide data generated through their services.
Beside these 3 main entities (ADM, PTF and URT), there is a Resources folder containing resources available to all users of the mass storage, such as Reference Genomes and Singularity Containers. In addition, each user also has a personal folder space called Home.
Here is a schematic representation of the tree. For clarity, not all Units, Laboratories, Groups and Projects folders are represented. A complete tree can be found [here](mass-storage/mass-storage-full-structure-home).
```mermaid
graph TB
subgraph sub1[Main entities]
C(Platforms)
B(Administration)
D(Research)
Z(Resources)
end
subgraph sub2[ ]
C --- CC(Platforms)
CC --> E(CRC)
CC --> F(GEN)
CC --> G(IMG)
CC --> sup1(...)
end
subgraph sub3[ ]
F --- GG(Groups)
GG --> Q(BIOINFO)
GG --> R(NGS)
GG --> fsub(GENO)
GG --> fsub2(SANGER)
end
subgraph sub4[ ]
D --- DD(Thematic Units)
DD --> H(GEN)
DD --> I(III)
DD --> sup2(...)
end
subgraph sub5[ ]
H --- HH(Laboratories)
HH --> J(BBB)
HH --> K(HG)
HH --> L(UAG)
HH --> sup3(...)
end
subgraph sub6[ ]
L --- LL(Groups)
LL --> M(BLV)
LL --> N(FACT)
LL --> O(IBD)
LL --> P(QCG)
LL --> X(TGV)
end
subgraph sub7[ ]
O --- MM(Projects)
MM --> S(...)
MM --> T(BLUEPRINT)
MM --> U(CROHN)
MM --> V(SHARE)
MM --> W(PLATFORMS)
end
style sub1 fill:#f3a581,stroke:#d65f4d,stroke-width:2px
style sub2 fill:#f3a581,stroke:#d65f4d,stroke-width:2px
style sub3 fill:#f3a581,stroke:#d65f4d,stroke-width:2px
style sub4 fill:#f3a581,stroke:#d65f4d,stroke-width:2px
style sub5 fill:#f3a581,stroke:#d65f4d,stroke-width:2px
style sub6 fill:#f3a581,stroke:#d65f4d,stroke-width:2px
style sub7 fill:#f3a581,stroke:#d65f4d,stroke-width:2px
style C fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style B fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style E fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style F fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style G fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style H fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style I fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style J fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style D fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style M fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style N fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style O fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style P fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style Q fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style R fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style S fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style T fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style U fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style V fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style W fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style X fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style K fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style L fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style Z fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style fsub fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style fsub2 fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style sup1 fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style sup2 fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style sup3 fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style CC fill:#91c5df,stroke:#4393c3,stroke-width:2px, stroke-dasharray: 5, 5
style DD fill:#91c5df,stroke:#4393c3,stroke-width:2px, stroke-dasharray: 5, 5
style GG fill:#91c5df,stroke:#4393c3,stroke-width:2px, stroke-dasharray: 5, 5
style HH fill:#91c5df,stroke:#4393c3,stroke-width:2px, stroke-dasharray: 5, 5
style LL fill:#91c5df,stroke:#4393c3,stroke-width:2px, stroke-dasharray: 5, 5
style MM fill:#91c5df,stroke:#4393c3,stroke-width:2px, stroke-dasharray: 5, 5
```
# User home folder
Each GIGA member is given a personal “home” space on the mass storage and will be granted access to group and project folder(s) by their PI through the GIGA XENIA platform.
The home folder serves as access point to all other folders this member has access to on the mass storage infrastructure.
The home folder is organized with a \_SHARE_ folder containing links to all shared folders the user has access to (Resources, Projects and Platform-shared folders).
Here is a schematic view of a hypothetical user123456’s home folder:
```mermaid
graph TB
subgraph sub1[User u123456]
C(_SHARE_)
B(user_personnal_folder_01)
D(user_personnal_folder_02)
end
subgraph sub2[ ]
C --- CC(Shared folders to which the user u123456 has access)
CC --> E(Resources)
CC --> F(Platforms)
CC --> G(Research)
CC --> H(Administration)
end
style sub1 fill:#f3a581,stroke:#d65f4d,stroke-width:2px
style sub2 fill:#f3a581,stroke:#d65f4d,stroke-width:2px
style C fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style B fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style D fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style F fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style G fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style H fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style E fill:#d2e6f0, stroke:#91c5df, stroke-width:2px
style CC fill:#91c5df,stroke:#4393c3,stroke-width:2px, stroke-dasharray: 5, 5
```
This personal space is currently limited to 100 GB per user (not including the volume of the data stored in the shared folders). However, to prevent duplication and/or loss of precious material, users should always favour storing data and results
relative to a research project only in the corresponding project folder (see below).
Upon departure from GIGA, a member must take care of their home folder by cleaning its content and/or by transmitting valuable files to their PI along with pertinent documentation. After departure of the user, the user's home will be deleted and all files not transferred to the group common area will be lost. It is the PI responsibility to ensure that such procedure is followed before the deletion of the home directory of the former user.
# Research group and project folders
Each GIGA member is by definition part of a research group and is working on one or multiple projects. To enable the sharing of data and results between members working on the
same project, the mass storage infrastructure also contains group and project folders. Group and project folders are created upon request by a PI. Every group folder and the
data/results contained within it are under the responsibility of this PI. Upon assignment to a project/group, GIGA members need to be granted access to the corresponding group
folder on the mass storage. Folder privileges are set up and modified by UDI GIGA-MED IT specialists, based on PI instructions. In a near future, management of folder privileges
by the PI will be facilitated by the GIGA XENIA application.
Upon request by a PI, UDI GIGA-MED IT specialists can also create a SHARE folder that will allow people from the same group but working on different project to share data. In the
same way, UDI GIGA-MED IT specialists can also create a PLATFORMS folder accessible to the platforms for providing data or images generated by their activities in the context of
the corresponding projects (see Platform-shared folders section below).
Group folders have a predefined size limit. This quota includes all sub-folders, including the associated platform-shared folders.
# Platform-shared folders
Data generated by the different GIGA platforms can be shared with the members of a group via a PLATFORMS folder situated inside the group folder. This PLATFORMS folder is created upon request by the PI. It will contain one subfolder per platform and can be subdivided further with one subfolder per project so that each user knows exactly where to find the raw data associated with a given project. GIGA platforms will only have reading/writing privileges to this shared folder, with no access to the associated project folders.
Users can either leave all data transferred by the platforms in the PLATFORMS folder or move them to the corresponding Project folder. Of note, most journals will ask for these raw data to be uploaded on a server before to accept any publication relative to them, so it’s extremely important to organise and annotate them properly to be able to retrieve them when needed. To avoid having multiple copies of large files, the platforms won’t keep a copy of these raw data. It’s the PI responsibility to make sure precious files are
stored and annotated correctly.
The whole platform-shared folder is considered to be part of the group folder. Hence, it is included in the calculation of the size quota and its content is under the responsibility of the group folder PI.
# Summary
#### Folders to which users who are part of a laboratory have access to:
| Folder | Description | Created | Quotas |
|------------|-------------------------------------------------------------------------------------------------------------------------------|--------------|------------|
| Home | Access point to server + storage of documents not linked to a specific project | By default | user |
| Resources | Reference Genomes, Singularity containers and other shared resources | By default | general |
| Projects | Data and analysis related to a specific project, accessible via the Research link in the user's home, following the structure detailed [here](mass-storage/mass-storage-structure-organisation#tree-structure) | Upon request | laboratory |
| SHARE | Data shared by members of a group working on different projects, accessible via the Research link, in the group folder, next to the Project folders | Upon request | laboratory |
| PLATFORMS | Data shared by the platforms, accessible via the Research link, in the group folder, next to the Project folders | Upon request | laboratory |
#### Folders to which users who are part of a platform have access to:
| Folder | Description | Created | Quotas |
| --------- | ------------------------------------------------------------ | ------------ | ---------- |
| Home | Access point to server + storage of documents not linked to a specific project | By default | user |
| Resources | Reference Genomes, Singularity containers and other shared resources | By default | general |
| Group | Each platform has a group folder where to store data they are processing. They are accessible via the Platforms link in the user's home, following the structure detailed [here](mass-storage/mass-storage-structure-organisation#tree-structure) | Upon request | platform |
| PLATFORMS | Data shared with members of research groups, accessible via the Research link, in the group folder, next to the Projects folder | Upon request | laboratory |
#### Folders permissions:
| Folder | Research User | User on the same project | User of same group but different project | User from another group | Platform members |
|----------------|----------------------|--------------------------|------------------------------------------|-----------------------|----------------------|
| Home | read, write, execute | no access | no access | no access | no access |
| Resources | read, execute | read, execute | read, execute | read, execute | read, write, execute |
| Project | read, write, execute | read, write, execute | no access | no access | no access |
| PLATFORMS | read, write, execute | read, write, execute | read, write, execute | no access | read, write, execute |
| SHARE | read, write, execute | read, write, execute | read, write, execute | no access | no access |
\ No newline at end of file