NetBackup and Oracle RMAN troubleshooting and data gathering

Published   Read time 4 min(s) 15 sec(s)   (3667 views).  NetBackup  

This post simple provides a listing of log files needed in order to troubleshoot Veritas NetBackup + Oracle RMAN isssues.

Log file description:

LogfileDescription
bphdb Resides on both server and client. Records communication between the client, ie. the Oracle API (RMAN), and the NBU server itself. Contains requests for data, lists of files backed up and restored.
obackup_tape Obackup was the old API before we switched to RMAN. It is possible to revert to Obackup to backup an Oracle database in case of RMAN issues. Contains record of tape requests and which tapes were used.
bporaexp Log which captures Oracle export information. If not using Oracle Hotbackup or Coldbackup, then a backup of a tablespace or the entire database can be taken to a flat file instead, using Oracle export. Log contains extensive information on Oracle users, which Oracle tables were backed up and when.
bporaimp Log which captures Oracle import information, relating to restoration of either a subset of Oracle tablespaces or the entire export database flat file.
bporaexp64 Same as above, but for 64-bit Oracle.
obporaimp64 Same as above, but for 64-bit Oracle.
bp.conf Central NBU configuration file, found on either client, media server or master server.
dbclient Client log which goes hand-in-hand with the Server-side bphdb log. Records communication and requests between the NBU server and the Oracle API (RMAN). Depending on the verbosity, will also show the actual RMAN scripts being run.
bpbrm NBU server-side backup and restore manager log. Records error status, and whether backup/restore was successful or failed.
bptm NBU media-server tape management and tape request log. Shows which tapes were loaded and when, and into which drives. Also contains record of any media errors. Can be cross-referenced with bpbrm log to refer to tape activity for specific backups/restores.
bprestore Logs information when the command-line version of NBU restore is used.
online_util NBU client-side log.
bpbkar Client log used for standard (non-RMAN) issues. Shows backup status, and is usually cross-referenced with the client bpcd log.
bpcd NBU client request log. Contains details of polling events from the server, and logs when a backup/restore request is initiated.
bpfis No longer used.
bppfi No longer used.
tar NBU uses GNU tar as its data format. Log file contains debug information from the actual tar binary recorded during restore operations. Can be used to diagnose tape/data corruption issues.

Data required for generic NBU+RMAN/Oracle issues

Master Server

  • Gather: Add VERBOSE=5 to /usr/openv/netbackup/bp.conf Run /usr/openv/netbackup/bin/goodies/support >support.out 
    mkdir /usr/openv/netbackup/logs/bphdb
mkdir /usr/openv/netbackup/logs/obackup_tape
  • Plus, if NOT using RMAN, for 32-bit Oracle: 
    mkdir /usr/openv/netbackup/logs/bporaexp
mkdir /usr/openv/netbackup/logs/bporaimp
  • Or, if NOT using RMAN and for 64-bit oracle: 
    mkdir /usr/openv/netbackup/logs/bporaexp64
mkdir /usr/openv/netbackup/logs/bporaimp64

Media Server

  • Gather: Add VERBOSE=5 to /usr/openv/netbackup/bp.conf
  • Plus, if NOT using RMAN, for 32-bit Oracle: 
    mkdir /usr/openv/netbackup/logs/bporaexp
mkdir /usr/openv/netbackup/logs/bporaimp
  • Or, if NOT using RMAN and for 64-bit oracle: 
    mkdir /usr/openv/netbackup/logs/bporaexp64
mkdir /usr/openv/netbackup/logs/bporaimp64

Client or Alternate Client

  • Gather: Add VERBOSE=5 to /usr/openv/netbakcup/bp.conf 
    mkdir /usr/openv/netbackup/logs/dbclient
mkdir /usr/openv/netbackup/logs/dbcd
mkdir /usr/openv/netbackup/logs/bphdb
mkdir /usr/openv/netbackup/logs/dbclient
  • Plus, if NOT using RMAN, for 32-bit Oracle: 
    mkdir /usr/openv/netbackup/logs/bporaexp
mkdir /usr/openv/netbackup/logs/bporaimp
  • Or, if NOT using RMAN but for 64-bit oracle: 
    mkdir /usr/openv/netbackup/logs/bporaexp64
mkdir /usr/openv/netbackup/logs/bporaimp64

Logs required for advanced/complex RMAN backup issues

Master Server

mkdir /usr/openv/netbackup/logs/bpbkar/bpbrm

Media Server

mkdir /usr/openv/netbackup/logs/bptm

Client or Alternate Client

mkdir /usr/openv/netbackup/logs/online_util
mkdir /usr/openv/netbackup/logs/bpbkar
mkdir /usr/openv/netbackup/logs/bpfis
mkdir /usr/openv/netbackup/logs/bppfi

Data required for advanced/complex RMAN restore issues.

Master server

mkdir /usr/openv/netbackup/logs/bprestore
mkdir /usr/openv/netbackup/logs/bprd
mkdir /usr/openv/netbackup/logs/bpbrm
mkdir /usr/openv/netbackup/logs/bptm
mkdir /usr/openv/netbackup/logs/tar

Media Server

None

Client or Alternate Client

None

Note

  1. ALL log directories must have permissions of 777 to enable writing.
  2. NBU daemons must always be restarted after changing bp.conf. Verbosity level 5 is a good starting point as it gives us a good level of detail, but not so much as to confuse the picture. We may later need to increase the verbosity to hone in on a particular daemon, or for the duration of a particular backup/restore. In NBU versions later than 3.5, daemon verbosity can be changed dynamically.
  3. Only required if requested by Veritas engineer upon escalation of an issue to them.
  4. For all log files produced under the /usr/openv/netbackup/logs subdirectories, the following applies:
    • For problems spanning several days, gather all logs under these directories.
    • For problems relating to a specific date or dates, gather only those logs for the relevant dates.