NetBackup and Oracle RMAN troubleshooting and data gathering
This post simple provides a listing of log files needed in order to troubleshoot Veritas NetBackup + Oracle RMAN isssues.
Log file description:
Logfile | Description |
---|---|
bphdb | Resides on both server and client. Records communication between the client, ie. the Oracle API (RMAN), and the NBU server itself. Contains requests for data, lists of files backed up and restored. |
obackup_tape | Obackup was the old API before we switched to RMAN. It is possible to revert to Obackup to backup an Oracle database in case of RMAN issues. Contains record of tape requests and which tapes were used. |
bporaexp | Log which captures Oracle export information. If not using Oracle Hotbackup or Coldbackup, then a backup of a tablespace or the entire database can be taken to a flat file instead, using Oracle export. Log contains extensive information on Oracle users, which Oracle tables were backed up and when. |
bporaimp | Log which captures Oracle import information, relating to restoration of either a subset of Oracle tablespaces or the entire export database flat file. |
bporaexp64 | Same as above, but for 64-bit Oracle. |
obporaimp64 | Same as above, but for 64-bit Oracle. |
bp.conf | Central NBU configuration file, found on either client, media server or master server. |
dbclient | Client log which goes hand-in-hand with the Server-side bphdb log. Records communication and requests between the NBU server and the Oracle API (RMAN). Depending on the verbosity, will also show the actual RMAN scripts being run. |
bpbrm | NBU server-side backup and restore manager log. Records error status, and whether backup/restore was successful or failed. |
bptm | NBU media-server tape management and tape request log. Shows which tapes were loaded and when, and into which drives. Also contains record of any media errors. Can be cross-referenced with bpbrm log to refer to tape activity for specific backups/restores. |
bprestore | Logs information when the command-line version of NBU restore is used. |
online_util | NBU client-side log. |
bpbkar | Client log used for standard (non-RMAN) issues. Shows backup status, and is usually cross-referenced with the client bpcd log. |
bpcd | NBU client request log. Contains details of polling events from the server, and logs when a backup/restore request is initiated. |
bpfis | No longer used. |
bppfi | No longer used. |
tar | NBU uses GNU tar as its data format. Log file contains debug information from the actual tar binary recorded during restore operations. Can be used to diagnose tape/data corruption issues. |
Data required for generic NBU+RMAN/Oracle issues
Master Server
- Gather:
Add
VERBOSE=5
to/usr/openv/netbackup/bp.conf
Run/usr/openv/netbackup/bin/goodies/support >support.out
mkdir /usr/openv/netbackup/logs/bphdb mkdir /usr/openv/netbackup/logs/obackup_tape
- Plus, if NOT using RMAN, for 32-bit Oracle:
mkdir /usr/openv/netbackup/logs/bporaexp mkdir /usr/openv/netbackup/logs/bporaimp
- Or, if NOT using RMAN and for 64-bit oracle:
mkdir /usr/openv/netbackup/logs/bporaexp64 mkdir /usr/openv/netbackup/logs/bporaimp64
Media Server
- Gather:
Add
VERBOSE=5
to/usr/openv/netbackup/bp.conf
- Plus, if NOT using RMAN, for 32-bit Oracle:
mkdir /usr/openv/netbackup/logs/bporaexp mkdir /usr/openv/netbackup/logs/bporaimp
- Or, if NOT using RMAN and for 64-bit oracle:
mkdir /usr/openv/netbackup/logs/bporaexp64 mkdir /usr/openv/netbackup/logs/bporaimp64
Client or Alternate Client
- Gather:
Add
VERBOSE=5
to/usr/openv/netbakcup/bp.conf
mkdir /usr/openv/netbackup/logs/dbclient mkdir /usr/openv/netbackup/logs/dbcd mkdir /usr/openv/netbackup/logs/bphdb mkdir /usr/openv/netbackup/logs/dbclient
- Plus, if NOT using RMAN, for 32-bit Oracle:
mkdir /usr/openv/netbackup/logs/bporaexp mkdir /usr/openv/netbackup/logs/bporaimp
- Or, if NOT using RMAN but for 64-bit oracle:
mkdir /usr/openv/netbackup/logs/bporaexp64 mkdir /usr/openv/netbackup/logs/bporaimp64
Logs required for advanced/complex RMAN backup issues
Master Server
mkdir /usr/openv/netbackup/logs/bpbkar/bpbrm
Media Server
mkdir /usr/openv/netbackup/logs/bptm
Client or Alternate Client
mkdir /usr/openv/netbackup/logs/online_util mkdir /usr/openv/netbackup/logs/bpbkar mkdir /usr/openv/netbackup/logs/bpfis mkdir /usr/openv/netbackup/logs/bppfi
Data required for advanced/complex RMAN restore issues.
Master server
mkdir /usr/openv/netbackup/logs/bprestore mkdir /usr/openv/netbackup/logs/bprd mkdir /usr/openv/netbackup/logs/bpbrm mkdir /usr/openv/netbackup/logs/bptm mkdir /usr/openv/netbackup/logs/tar
Media Server
None
Client or Alternate Client
None
Note
- ALL log directories must have permissions of 777 to enable writing.
- NBU daemons must always be restarted after changing bp.conf. Verbosity level 5 is a good starting point as it gives us a good level of detail, but not so much as to confuse the picture. We may later need to increase the verbosity to hone in on a particular daemon, or for the duration of a particular backup/restore. In NBU versions later than 3.5, daemon verbosity can be changed dynamically.
- Only required if requested by Veritas engineer upon escalation of an issue to them.
- For all log files produced under the /usr/openv/netbackup/logs subdirectories, the following applies:
- For problems spanning several days, gather all logs under these directories.
- For problems relating to a specific date or dates, gather only those logs for the relevant dates.