The most common reason why individual server restores fail while all others are successful is due to a corrupt backup of those servers in your Veeam Repository.
Cloud IBR automatically exfiltrates the Veeam logs from the Cloud IBR VBR for every test you run. You can download the logs from the Recovery Information section within the Cloud IBR recovery session. These Veeam logs from will typically present with:
- error: S3 error: We encountered an internal error. Please retry the operation again later.
- DownloadFileVersionAsync async task has failed, path [/Veeam/Backup/Offsite/Clients/{85ffd1b6-cfa3-4eb8-8691-8a427b47e097}/4b609856-7af8-4cd0-b2f9-68f4e24497db/CloudStg/Data/{c04fc646-a303-4683-8391-2afe4f94cbd5}/{f271c628-7036-490e-a9c3-f44dba85b1fe}/91784_9993a2f1638f63c044d3aa3fe1a60ae2_cff2153e218c11d02dd393919b5b723e], version [001721785456581922630-hOQHZ3vWb4], offset [0], length [0]
Troubleshooting: Enable Health Checks on your Veeam Backup or Copy Job within the advanced settings of the job and schedule when you would like it to run. Once the Health Check completes, check the logs under History to see if there were any issues. Corrupted backups typically show the below errors for each of the individual servers that failed to restore.
- Failed Verifying disks (100% done)
- S3 error: We encountered an internal error. Please retry the operation again later. Code: InternalError Agent failed to process method {Signature.FullRecheckBackup}.
- Failed Processing finished with errors at 12/10/2024 1:27:56 PM
Resolution: Run an Active Full for your backup job if you are backing up directly to object storage or run an Active Full for your copy job that backs up to object storage. This will fix the corrupted backup. If running an Active Full does not fix the issue, you should open a ticket with Veeam support.