Storage¶
Your storage volumes¶
Every workspace on FLAME comes with persistent storage volumes that are automatically created and attached to your workloads:
Home PVC¶
A personal space for each workspace member.
| Default size | 100 GB |
| Path in notebooks | /home/ubuntu |
| Path in jobs | /personal |
Use this for your own scripts, notebooks, experiments, and scratch work.
Workspace storage¶
Shared storage for a workspace.
| Default size | 1 TB |
| Path | /workspace (same in notebooks and jobs) |
Use this for datasets, shared models, results, and anything your team needs to collaborate on.
Both volumes are automatically attached to JupyterHub notebook sessions, Kubernetes Jobs, and Kubeflow TrainJobs. You don't need to do anything special, the storage is there when your workload starts.
Different home path in jobs
Your home directory mounts at /home/ubuntu in notebooks but at /personal in Jobs and TrainJobs. The files are the same -- only the path differs.
Kubernetes storage: PVCs¶
Under the hood, each volume is a Persistent Volume Claim (PVC), a Kubernetes abstraction for durable storage. Think of a PVC as a portable hard drive within the cluster: it exists independently of any particular pod, and can be attached to whatever is currently running your code. When your notebook session stops or a job finishes, the PVC and all its contents remain untouched, ready to be attached to the next workload. PVCs can also be attached to multiple workloads at once, which can read and write from them concurrently (e.g. for distributed training).
FLAME manages the creation and attachment of the "home" and "workspace" PVCs for you automatically. Data on them persists across session restarts, job reruns, and cluster maintenance.
In addition to these PVCs, you can also create your own additional PVCs and attach them to workloads using standard Kuberentes tooling (kubectl, YAML manifests) if you'd like to organize your data differently.
Storage backend: Weka¶
The physical storage backing all PVCs on FLAME is a Weka parallel filesystem. Weka is a high-performance storage system designed for data-intensive workloads like large-scale model training.
Weka provides storage to workloads running in FLAME using the Container Storage Interface, a uniform standard for storage vendors like Weka to provide access to an container orchestration system like Kubernetes.