How to backup nfs storage

[wangshuai0709]

My cluster used nfs as the storage class. I want to backup the data of the pv in tektoncd-default namespace.
My backup operation:
1.annotate pod

kubectl annotate pod smart-pg-sonar-77985cf7cd-mhgb4  backup.velero.io/backup-volumes=smart-pg-sonar-varlibpostgresql-new -n tektoncd-default
kubectl annotate pod sonar-7c6fbfc875-v5ftr  backup.velero.io/backup-volumes=sonar-plugin-new -n tektoncd-default

2.create backup task

velero backup create pvbackupnew   --include-namespaces=tektoncd-default    --default-volumes-to-fs-backup

view information about the backups:

The above image has only three volume,but the namespace has 25 volume.

What is the reason for this?Please help me.Thank you.

[kaovilai]

Response to NFS Volume Backup Issue

Hi @wangshuai0709,

The reason you're only seeing 3 volumes backed up instead of 25 is likely related to how Velero handles filesystem backups with the --default-volumes-to-fs-backup flag.

Key Requirement

When using --default-volumes-to-fs-backup (which is typically needed for NFS volumes), PVCs must be bound to running pods for Velero to back them up. Velero performs filesystem backups by mounting the volumes through the pods that use them.

Troubleshooting Steps

1. Check which PVCs are actually bound to pods:

   kubectl get pvc -n tektoncd-default
   kubectl get pods -n tektoncd-default -o wide

2. Verify pod annotations:
   Make sure pods using the PVCs you want to backup have the correct annotations:

   backup.velero.io/backup-volumes: volume1,volume2,volume3

   Or use backup.velero.io/backup-volumes-excludes to exclude specific volumes.

3. Check for unbound PVCs:
   PVCs that aren't currently mounted by running pods won't be backed up with filesystem backup method.

Solutions

Option 1: Ensure all PVCs are bound to running pods

• Scale up deployments/statefulsets that use the PVCs
• Make sure pods are in Running state

Option 2: Use volume snapshots (if supported)

If your NFS storage supports snapshots, you can use volume snapshots instead:

velero backup create my-backup --include-namespaces tektoncd-default --snapshot-volumes

Option 3: Create temporary pods for unbound PVCs

For PVCs that aren't currently bound to running pods, you can create temporary pods to mount them during backup:

apiVersion: v1
kind: Pod
metadata:
  name: backup-helper-pvc-name
  namespace: tektoncd-default
  annotations:
    backup.velero.io/backup-volumes: volume-name
spec:
  containers:
  - name: backup-helper
    image: busybox
    command: ['sleep', '3600']
    volumeMounts:
    - name: volume-name
      mountPath: /data
  volumes:
  - name: volume-name
    persistentVolumeClaim:
      claimName: your-pvc-name

Verification

After making changes, verify your backup includes all expected volumes:

velero backup describe your-backup-name --details

The issue you're experiencing is common with NFS storage because filesystem backups require active pod mounts. Let me know if you need help with any of these troubleshooting steps!

Best regards

[wangshuai0709]

Are all the PVC mounted to a pod in the namespace?

I only want to backup tow volume that is sonar-plugin-new and smart-pg-sonar-varlibpostgresql-new.
I think I annotate command is wrong.I used pvc name not volume name.Could my mistake have been caused by this?

I have another question. If one only wants to back up the storage data, would it be sufficient to just specify the pv and pvc?

[wangshuai0709]

@kaovilai
I re-added the "annotate" to the pod and made another backup. After restoring, only one pod volume was successfully recovered.

velero restore logs pvbackupnew2-20250703103246|grep -C 10 error

time="2025-07-03T03:24:05Z" level=info msg="Waiting for all pod volume restores to complete" logSource="pkg/restore/restore.go:668" restore=velero/pvbackupnew2-20250703103246
time="2025-07-03T07:23:54Z" level=error msg="unable to successfully complete pod volume restores of pod's volumes" error="timed out waiting for all PodVolumeRestores to complete" logSource="pkg/restore/restore.go:1909" restore=velero/pvbackupnew2-20250703103246
time="2025-07-03T07:23:54Z" level=error msg="unable to successfully complete pod volume restores of pod's volumes" error="timed out waiting for all PodVolumeRestores to complete" logSource="pkg/restore/restore.go:1909" restore=velero/pvbackupnew2-20250703103246
time="2025-07-03T07:23:54Z" level=info msg="Done waiting for all pod volume restores to complete" logSource="pkg/restore/restore.go:684" restore=velero/pvbackupnew2-20250703103246
time="2025-07-03T07:23:54Z" level=error msg="Velero restore error: timed out waiting for all PodVolumeRestores to complete" logSource="pkg/controller/restore_controller.go:596" restore=velero/pvbackupnew2-20250703103246
time="2025-07-03T07:23:54Z" level=error msg="Velero restore error: timed out waiting for all PodVolumeRestores to complete" logSource="pkg/controller/restore_controller.go:596" restore=velero/pvbackupnew2-20250703103246

What is the reason for this?

[wangshuai0709]

@kaovilai
Why does pod volume restore time out?

[wangshuai0709]

@carlisia @ncdc @jwhitcraft
Who can help me? Thanks very mush

[kaovilai]

Why does pod volume restore time out?

Many possibilities.

• Node agents got OOMKilled
• slow network transfer
• large volume copy
• default timeout too low