Manually cleaning NetWorker backups on an Avamar system

Published 22 Mar 2014 Read time 2 min(s) 49 sec(s) (3981 views). Storage

Having recently got involved on a site, it came to light very quickly that some housekeeping was required on the Avamar system, as such I spent 2 days reading a large number of EMC documentation to come up with a plan.

The scope of this document is to provide a how to like procedure aimed at cleaning up rouge and outdated NetWorker backups held on an Avamar system.

Steps involved:

Pre-Requisite Tasks
Manual Cleanup Procedure
Avamar Housekeeping Tasks
Post-Cleanup Tasks

PRE-REQUISITE TASKS

Prior to running the manual cleanup we need perform the following checks:

Open a putty session and login the Avamar server as 'admin' user.
Load the Admin keys:
```
# ssh-agent bash
# ssh-add .ssh/dpnid
```
NOTE: We need to load the admin keys to allow us to authenticate correctly with the Avamar system.
Check to make sure no backup sessions are currently running:
```
# avmaint sessions | grep path
```
Check to see if we any running/hung sessions:
```
# avmaint sessions | grep sessionid
```
The above will list all running and hung sessions (including replication sessions)
If you need to manually kill a hung session, perform:
```
# avmaint kill <sessionid>
```
replace <sessionid> with the value found in the previous step

MANUAL CLEANUP

Once the pre-requisite checks have been carried out, we can start with the cleanup operation.

Stop the scheduler so no new backups from NetWorker to Avamar start:
```
# dpnctl stop sched
```
Stop the maintenance scheduler so no checkpoint, healthcheck or garbage collection tasks start:
```
# dpnctl stop maint
```
Run the following command to identify NetWorker savesets older than a given date:
```
# delete-snapups --before=mm/dd/yyyy --domain=/NetWorker > cleanup_candidates.sh
```
This will generate a file named "cleanup_candidates.sh" that is auto formatted like a script to delete entries. So, when you execute the file, it starts deleting.
To determine the number of entries in the script that the cleanup will perform, we can simply grep and count the lines:
```
# grep "^m" cleanup_candidates.sh | wc -l
```
Execute the 'cleanup_candidates.sh' script, to purge the entries:
```
# chmod +x ./cleanup_candidates.sh
# ./cleanup_candidates.sh
```

AVAMAR HOUSEKEEPING

Once the manual cleanup has completed, we need to run a series of Avamar housekeeping tasks (checkpoint, healthcheck and garbage collection)

Run a manual checkpoint task and monitor until it completes:
```
# avmaint checkpoint --ava
# watch avmaint cpstatus
```
the status field will change to 'completed' once the checkpoint has finished (press 'ctrl-c' to exit the watch process)
Run a manual healthcheck task and monitor until it completes:
```
# avmaint hfscheck --rolling --ava
# watch avmaint hfscheckstatus
```
the 'percent-completed' field will give an indication of how far the healthcheck has got, likewise the status field will change to 'completed' once the healthcheck task has finished (press 'ctrl-c' to exit the watch process)
Run a manual garbage collection task:
```
# avmaint garbagecollect --ava --maxtime=3600
```
(setting maxtime=3600 allows the garbage collection to run for only 1 hour)
Monitor the garbage collection task until it completes:
```
# watch avmaint gcstatus
```
the status field will change to 'idle' once the garbage collection task has finished (press 'ctrl-c' to exit the watch process)

POST MAINTENANCE

Once the manual cleanup and housekeeping tasks have completed, we can restarted the schedulers.

Restart the maintenance window scheduler:
```
# dpnctl start maint
```
Restart the backup scheduler:
```
# dpnctl start sched
```
Check the status of both schedulers:
```
# dpnctl status
```
we need to confirm that the 'maintenance window scheduler' is 'enabled' and the 'backup scheduler' is 'up'