Solaris tape device files and utilities

Updated 21 Apr 2001 Read time 26 min(s) 11 sec(s) (3025 views). Solaris

Solaris tape device files are rather complex in terms of the way they are created and viewed. The level at which users interact with these drivers is by and large limited to the symbolic links the operating system creates; they actually reside in /devices, though /dev houses the generally used files.

Many support calls I have worked are on Solstice Backup (aka Legato NetWorker) and Veritas NetBackup and can actually be handled if one understands the basics of tape devices at the operating system level. The purpose of this document is to prepare people for troubleshooting at the basic level, to help ensure that all necessary steps have been taken prior to engaging the issue at the application level.

Fundamental communication

Every tape device attached to a host must exist at its own unique SCSI id number on a particular controller. Usually, there are small wheels or tickers on the back of devices that allow this to be set with relative ease. On larger libraries that have front LCD displays, configuration of robotics and drive ID's can be done here.

Cabling and termination of libraries and drives requires the same considerations that any other SCSI device looks for; it is important to make sure that differential and single-ended devices are terminated properly, and that cables are within specifications.

**NOTE** Never plug SCSI devices in hot. The system and the peripheral to be added should both be quiesced prior to doing so or else interfaces may be blown. Removing SCSI devices can be done while the system is up, though it is preferable that it not be.

Once the hardware is attached, it is often useful to probe the controllers from the prom level to make sure that the devices have been picked up by the system (that hardware connectivity has been established at the SCSI level). If the probe-scsi-all command shows the devices, all is well. If it does not, but returns to a prompt, then it doesn't see them at all. If it hangs, then you may have a SCSI ID conflict or other connectivity problem.

If the system is already up and one does not think connectivity has been established, the iostat utility can be used to get information about attached disks and tape drives. The most beneficial options for our purposes are 'E' (report error statistics on each device) and 'n' (display names in descriptive format (c#t#t# or rmt respectively for each device). Command output from an iostat -En looks as follows:

c1t5d2          Soft Errors: 0 Hard Errors: 0 Transport Errors: 0 
Vendor: SEAGATE  Product: ST31200W SUN1.05 Revision: 8724 Serial No: 00328177 
Size: 1.05GB 
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 
Illegal Request: 0 Predictive Failure Analysis: 0 

c1t5d3          Soft Errors: 0 Hard Errors: 0 Transport Errors: 0 
Vendor: SEAGATE  Product: ST31200W SUN1.05 Revision: 8566 Serial No: 00675596 
Size: 1.05GB 
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 
Illegal Request: 0 Predictive Failure Analysis: 0 

rmt/0           Soft Errors: 2 Hard Errors: 0 Transport Errors: 0 
Vendor: QUANTUM  Product: DLT7000          Revision: 2560 Serial No: q`

Both Legato NetWorker (Solstice Backup) and Veritas NetBackup have their own utilities for probing for devices. These are obviously run from init 3, and can serve a similar function to probe-scsi.

NetWorker's utility can be run like this:

/etc/LGTOuscsi/inquire

with output as noted below:

root@smurf-lgto# /etc/LGTOuscsi/inquire
scsidev@0.2.0:SEAGATE ST32171W SUN2.1G7462|Disk, /dev/rdsk/c0t2d0s2
scsidev@0.3.0:SEAGATE ST32171W SUN2.1G7462|Disk, /dev/rdsk/c0t3d0s2
scsidev@0.4.0:QUANTUM VK2275J  SUN2.1G8817|Disk, /dev/rdsk/c0t4d0s2
scsidev@0.5.0:QUANTUM VK2275J  SUN2.1G8817|Disk, /dev/rdsk/c0t5d0s2
scsidev@0.8.0:IBM     DNES30917SUN9.0GSAD0|Disk, /dev/rdsk/c0t8d0s2

On NetBackup, one can issue:

/usr/openv/volmgr/bin/sgscan

and the ouptut is as follows:

root@smurf-nbu# /usr/openv/volmgr/bin/sgscan
/dev/sg/c0t0l0: Disk (/dev/rdsk/c0t0d0): "FUJITSU MAG3091L SUN9.0G"
/dev/sg/c0t1l0: Disk (/dev/rdsk/c0t1d0): "FUJITSU MAG3091L SUN9.0G"
/dev/sg/c0t6l0: Cdrom: "TOSHIBA XM6201TASUN32XCD"

**NOTE** Libraries (robotics) will not be seen, and cannot be administered or manipulated, at the operating system level. One's best indication of whether or not a library is recognised comes from analysing probe output. Backup software must be used to achieve any sort of comprehensive control over such devices.

Solaris uses the st (SCSI tape) device driver to talk to any drives that are attached to a host. The configuration file that it uses is the /kernel/drv/st.conf file. Edits to st.conf should be done with great care as infrequently as possible, as modification of a given entry could render a device unreachable.

**NOTE** If changes are made, the system should be rebooted for the modifications to take; If this is utterly impossible, unloading and reloading the st module may work (more often than not it won't, though):
# rem_drv st
# add_drv st
Any return except for a prompt means you have to reboot. The most common error encountered here will state that the module has been loaded but failed to attach. Again, this necessitates a reboot -r.

st.conf

The st.conf file is broken up into four general sections:

introductory section -- talks about natively supported devices and so forth
tape_config_list -- lists most devices Solaris is aware of. This is where entries for additional drives are added. The entries take the form of SCSI inquiry strings and other pertinent information. Some have not been tested by Sun officially, a fact that is noted in the comments within st.conf. An example from this section looks like:
```
		
#tape-config-list=
#       "Emulex  MTO2",         "Emulex MT02 QIC-11/QIC-24",    	"MT02",
#       "ANRITSU DMT2120",      "Unisys 1/2\" Reel",            		"ANRITSU",
#       "ARCHIVE Python 28454", "Archive Python 4mm Helical Scan","ARCH_Python",
#       "ARCHIVE Python 29279", "Archive Python 4mm DAT Stacker","ARCH_Python",
#       "ARCHIVE Python 28388", "Archive/Conner CTDX004 4mm DAT", "Conner2-DAT",
#       "ARCHIVE VIPER 150",    "Archive QIC-150",              "QIC150",
#       "ARCHIVE VIPER 255",    "Archive QIC-525",              "QIC150",
#       "DEC     DLT2",         "DEC DLT",                      "DLT2",
#       "DEC     TZ87",         "DEC DLT",                      "DLT2",
#       "EXABYTE EXB-2501",     "Exabyte EXB-2501 QIC",         "EXB-2501",
```
**NOTE** More often than not, many of the problems encountered by customers who are unable to create device files for their drives can be solved by making sure the line for tape_config_list is uncommented. Failure to do so prevents Solaris from reading the entries below it and, by extension, reading the device-specific entries below that. The comma at the end of each line is also important. The last entry in this section should end with a semicolon, thereby introducing the following section.
device-specific properties (data-property-name) -- this section tells the st driver how to specifically interact with a given drive, including what densities to use,whatbehavioral bits have been set, and other such information. Many entries must be obtained from third party sources (drive manufacturers, etc.) as Solaris only supports certain drives natively. For more detail on each field, see the st man page. Broadly speaking, the entries take the following form:
```
#ANRITSU        =       1,0x25,0,0x41d,4,0x00,0x02,0x03,0x03,1;
#ARCH_Python    =       1,0x2c,0,0xde39,4,0x00,0x8c,0x8c,0x8c,3;
#CLASS_3490     =       1,0x24,0,0x1c43d,1,0x00,0;
#C3490          =       1,0x24,0,0x1c43d,4,0x00,0x00,0x00,0x01,2;
#Conner2-DAT    =       1,0x2c,0,0xde39,1,0x00,0;
#DAT            =       1,0x34,0,0x0439,1,0x00,0;
#DLT2           =       1,0x38,0,0xd639,4,0x17,0x18,0x80,0x81,2;
#DLT-data       =       1,0x38,0,0xD639,4,0x80,0x81,0x82,0x83,2;
#DLT7k-data     =       1,0x38,0,0x1D639,4,0x82,0x83,0x84,0x85,2;
#EXB-2501       =       1,0x28,1024,0x442,1,0x00,0;
```
**NOTE** Entries in the data-property-name section must terminate with a semicolon. Failure to do so when adding non-native entries will cause problems.
recognized targets -- the bottom of the st.conf file has entries for all acceptable SCSI ID targets that drives may possess. Typically, they are from 0 - 6 (uncommented) and 8 - 15 (available if needed; use depends on SCSI capabilities; must be uncommented). The entries look like this:
```
name="st" class="scsi"
       target=3 lun=0;

name="st" class="scsi"
       target=4 lun=0;

#
#       In case there are wide tape drives, one can use these targets
#
#name="st" class="scsi"
#       target=8 lun=0;
```

Device files

To view tape device files on Solaris hosts, go to the /dev/rmt directory. Herein you will find a series of files comprised of numbers and letters. They will look something like this:

root@smurf-nbu# ls /dev/rmt
0           0m          1cn         1ub         2lb
0b          0mb         1h          1ubn        2lbn
0bn         0mbn        1hb         1un         2ln
0c          0mn         1hbn        2           2m
0cb         0n          1hn         2b          2mb
0cbn        0u          1l          2bn         2mbn
0cn         0ub         1lb         2c          2mn
0h          0ubn        1lbn        2cb         2n
0hb         0un         1ln         2cbn        2u
0hbn        1           1m          2cn         2ub
0hn         1b          1mb         2h          2ubn
0l          1bn         1mbn        2hb         2un
0lb         1c          1mn         2hbn        sg.conf.all
0lbn        1cb         1n          2hn         sg.links.all
0ln         1cbn        1u          2l          st.conf.all

These entries are just symbolic links to real character (device) files. They are named in this way for purely aesthetic reasons, as the actual device paths are extremely long and ugly. For example, a long listing of the device file 0mbn would show the following:

lrwxrwxrwx   1 root     other         65 Jun 20 15:18 0mbn -> ../../devices/sbus@1f,0/espdma@e,8400000/esp@e,8800000/st@1,0:mbn

If one were to do a long listing of the device path, the output would look like this:

root@smurf-nbu# ls -la ../../devices/sbus@1f,0/espdma@e,8400000/esp@e,8800000/st@1,0:mbn
crw-rw-rw-   1 root     sys       33, 77 Jun 29 03:37 ../../devices/sbus@1f,0/espdma@e,8400000/esp@e,8800000/st@1,0:mbn

The c in the permissions mask indicates that the file is a character file. The permissions are read and write for owner, read/write for the group 'sys', and read/write for for all others. This file is rarely manipulated or examined directly, as the vast majority of applications and user interactions take place at the link level. Ergo, it is important to understand what the /dev/rmt entries mean instead of concentrating on the character file itself.

Breaking down the pieces

The numbers in front of the series of one to three letters stands for the logical device number each tape drive holds. The first one found by the system is given the designation 0, the second is 1, the third is 2, and so forth. These numbers do NOT correspond to SCSI i.d. Numbers, as can be seen in the long listing above (the st@1,0 means it is SCSI tape device 0 at SCSI i.d. 1, LUN 0). For our example, we will use files associated with device 0.

Key terms used in forthcoming explanations:

rewinding -- tape will be rewound after last end-of-file (EOF) mark is written to the ufsdump or other backup
non-rewinding -- tape will be left positioned where last EOF was written. This facilitates continued archives being created on a piece of media without having to space out to the end of the previous backup first.
Berkeley [close] style -- indicates the way in which EOF spacing will be handled by Solaris. This is different from how System V (or AT&T) close style operates. Essentially, it means that when reading past an end-of-file mark, the drive returns the first record of the next file, and when closing the no rewind device, it skips a tape space forward. Device files used by Solaris will be Berkeley style almost all the time. If you want more information, read SRDB 15385.
density -- Different tape drives use different density settings. Basically, this parameter informs the system about the length of tapes being used. A particular tape drive can use different density device files; they are not limited to only one density.
xompression: self-evident -- refers to the algorithmic ratio used by the driver in compacting data for storage on removable media. There are different levels of compression in Solaris device files.

**NOTE** Not all devices are capable of using compression drivers. Customers should consult the product notes for their particular drives to see what they can and cannot do. Additionally, certain drives do not use compression settings in the normal manner; For instance, DLT drives use 'c' for compression, 'h' for uncompressed, and so forth. Again, referring to device documentation will elucidate specific requirements.

Now, on to the explanations. Beside each device file listed below is a dissection of what the letters mean. All these device files contain different attributes but refer to the same physical tape drive, 0, and can be used at will:

0	This assumes a standard rewinding device file with no compression.
0b	Berkeley style, rewinding
0bn	Berkeley style, non-rewinding.
0c	SysV, compression, rewinding
0cb	Berkeley style, rewinding
0cbn	Berkeley style, compression, non-rewinding
0cn	SysV, compression, non-rewinding
0h	SysV, high density, rewinding
0hb	high density, Berkeley, rewinding
0hbn	high density, Berkeley, non-rewinding
0hn	SysV, high density, non-rewinding
0l	SysV, low density, rewinding
0lb	low density, Berkeley, rewinding
0lbn	low density, Berkeley, non-rewinding
0ln	SysV, low density, non-rewinding
0m	SysV, medium density, rewinding
0mb	medium density, Berkeley, rewinding
0mbn	medium density, Berkeley, non-rewinding. Very commonly used device.
0mn	SysV, medium density, non-rewinding
0n	SysV, no compression, non-rewinding
0u	SysV, ultra density, rewinding
0ub	ultra density, Berkeley, rewinding
0ubn	ultra density, Berkeley, non-rewinding
0un	SysV, ultra density, non-rewinding

&nbp;

Similar entries for other logical device numbers will be found as well. Whenever the system creates tape device files, the full spectrum of possible device files is created. This can be seen above, in the first example, where entries for devices 0, 1, and 2 are found.

**NOTE** The number found in /dev/rmt is not necessarily indicative of a unique device. If a boot -r is done, device numbers in /dev/rmt may be incremented. The sure way to know is to do a long listing of the /dev/rmt device file and check the 'st @ <#>' parameter.

How to create tape device files

There are two ways that tape device files are created.

One is through the re-probing done when a boot -r is executed. The buses on a host are rescanned and the device paths compared to those already existing in the /etc/path_to_inst file. If an entry exists for particular devices, the number for that device may be incremented by one. In other words, if two new tape drives are added to a brand new system, entries 0 and 1 will exist. If a boot -r is done with only those two drives attached, /dev/rmt may now contain entries for devices 0,1,2 and 3. Device files for 0 and 1 will be unusable, almost as if they are stale.
A second way to create device files can be done if a boot -r is not possible. It involves a sort of 'soft' probe of the SCSI buses for new hardware, and a user-supplied series of commands for building device files and symbolic links. These commands are implicitly run when a boot -r is executed, but users can run them instead of bringing a system all the way down. They should be issued in succession:
- drvconfig -- The default operation of drvconfig is to create the /devices directory tree that describes, in the filesystem namespace, the hardware layout of a particular machine. Hardware devices present on the machine and powered on as well as pseudo-drivers are represented under /devices. Normally this command is run automatically after a new driver has been installed and the system has been rebooted. It can also be run as a probe if the system can't go down.
- tapes --tapescreates symbolic links in the /dev/rmt directory to the actual tape device special files under the /devices directory tree. 'tapes' searches the kernel device tree to see what tape devices are attached to the system. For each tape drive found, the following steps are performed:
  1. The /dev/rmt directory is searched for a /dev/rmt/n entry that is a symbolic link to the /devices special node of the current tape drive. If one is found, this determines the logical controller number of the tape drive.
  2. The rest of the special devices associated with the drive are checked, and incorrect symbolic links are removed and necessary ones added.
  3. If none are found, a new logical controller number is assigned (the lowest-unused number), and new symbolic links are created for all the special devices associated with the drive.
  **NOTE** The tapes command does not remove links to non-existent devices; these must be removed by hand.
- devlinks -- devlinks creates symbolic links from the /dev directory tree to the actual block- and character-special device nodes under the /devices directory tree. The links are created according to specifications found in the table-file (by default etc/devlink.tab).
After running these commands, device files should exist in the /dev/rmt directory that reflect a new date and time stamp. This does not always work as well as an actual boot -r, as the latter forces a full probe.

**NOTE** In Solaris 8, the devfsadm command replaces the aforementioned sequence. Running it by itself will achieve the same thing as running drvconfig, disks, tapes, audlinks, links, and devlinks all at once. There is actually a daemon that runs full time in the background, and can actually perform the same tasks when it cycles without manual intervention.

Solaris tape utilities and commands

Quite a few utilities exist within the Solaris operating system that can add value to your troubleshooting efforts. Several of the most widely used are detailed below. For additional information, refer to manual pages or the brief overviews given in the Solaris Operating Environment (System Administration I) class manual for each, as only a limited number of operands and arguments for each are covered here.

mt -- One of the first things one should become familiar with in regard to device files on Solaris is the mt command. It stands for 'magnetic tape', and provides a useful way in which to interact with system-attached tape drives. The mt command works by allowing users to manipulate drives directly through their associated device files. The syntax for this command is as follows:
```
mt -f(t) <device file> <command>
```
Some of the more common commands used with mt are listed below. Entering the first three letters of any of them does the same thing as entering the whole word.
1. status (stat) -- This is used to determine whether or not basic connectivity to a drive exists. It returns the status of a given device, and will generally provide one of two types of output, error and online. The unsuccessful message usually looks like this:
```
# mt -f /dev/rmt/0mbn stat
/dev/rmt/0mbn: no tape loaded or drive offline
```
  This is always returned unless there is a tape in the drive. If a tape is sensed, the output looks like this:
```
# mt -f /dev/rmt/0mbn status
 Quantum DLT7000 tape drive:
    sense key(0x0)= No Additional Sense   residual= 0   retries= 0	
   file no= 0   block no= 0
```
  Such output at least tells you that a drive is there and is operational from this level.
2. fsf -- This is an acronym for 'file skip forward'. Issuing this command will space the driveheadoutnfilemarks on tape, and requires a specifier that denotes the number ofspacesoutonedesires. An example of the command's syntax is:
```
# mt -f /dev/rmt/0mbn fsf 20
```
3. rewind (rew) -- Self explanatory; it rewinds tapes that have been loaded to a particular drive. This is usually run, logically, after one has spaced out onto the tape. It takes no further arguments.
4. erase -- Erases the contents of an entire tape.
  **NOTE** There is no way to recover the contents after this is done.
tar -- The Tape ARchive utility can be used to bundle multiple files into a single tarfile. It can also be used to extract files from said tarfiles. This is a useful utility to be familiar with, as it can be employed to ascertain whether or not a tape device is writable to and readable from and can thus serve as a troubleshooting tool. There are three main uses for the tar command that are germane to this document. They all perform a function that can aid in honing in on problems. The first of these creates a tar archive:
```
tar cvf ^lt;tarfile (destination file)> <source^gt;
```
where 'destination file' is the name of the file you would like to create, and 'source' is the directory or files that will be included in the archive. The 'cvf' options stand for 'create', 'verbose', and 'file' (this option tells tar that you will be providing a tarfile name for it to write to. When this command is issued, the source files will be written to standard out (verbosely) as the archive is being created:
```
$ tar cvf ./tartest.tar ./*
a ./raidcode.txt 45K
a ./rmlog 2104K
a ./rmlog.log 79K
a ./rmloginfo 208K
```
A file called tartest.tar will now exist in your working directory.
**NOTE** For the purposes of this TOI archives were created within standard UNIX directories, but the device file of a drive can be used in place of a tarfile; this will allow you to create your archive on tape.
To check that the files one specified were bundled into the archive, the tar utility can be given a 't' option (table of contents). The syntax for using this is:
```
tar tvf <tarfile>
```
The output will appear as follows:
```
$ tar tvf tartest.tar
-rw-r--r-- 114294/10   45530 Nov 17 22:03 2000 ./raidcode.txt
-rw-r--r-- 114294/10  2153880 Apr  8 15:11 2001 ./rmlog
-rw------- 114294/10   80765 Apr  8 16:44 2001 ./rmlog.log
-rwxrwxrwx 114294/10  212992 Nov 10 00:30 2000 ./rmloginfo
```
If it comes back as expected, you will know that your tar archive was created correctly. Note that the files displayed by the 'tvf' are the same as those in the 'cvf'. The third and final example of tar shown herein is utilized to extract files from a previously written archive. If you create one on tape, you can now test to see if the device read works by seeing if you can get it extracted. The syntax for this function is:
```
tar xvf <absolute path of tarfile> .
```
If I create a directory called /tmp/blah, then cd to it and execute the following command, you can see how the files are put there:
```
$ tar xvf /home/fooRM6/tartest.tar .
x ./raidcode.txt, 45530 bytes, 89 tape blocks
x ./rmlog, 2153880 bytes, 4207 tape blocks
x ./rmlog.log, 80765 bytes, 158 tape blocks
x ./rmloginfo, 212992 bytes, 416 tape blocks
```
Trying the preceding operations to a potentially problematic tape drive will clarify for you how compromised communication with that device really is. Successfully creating and extracting tar archives will allow you to proceed to the next level of troubleshooting, that of the backup application itself.
**CRITICAL TAR NOTES**
- It is very important to remember that the Solaris version of tar does exactly what you tell it to do. Ergo, if you give an absolute pathname for the 'source', that directory path will be hardcoded into the archive. In other words if I had used the path '/home/foo/RM6/*' instead of the './*' that I used, whomever wanted to extract the archive would have to have the directory path '/home/foo/RM6' on their machine. If they do not, a checksum error will be given. The lesson is ALWAYS USE RELATIVE PATHNAMES FOR THE SOURCE.
- Conversely, when you go to extract an archive, you should issue the command to do so from the location you want the files to be extracted to. Solaris tar does sourcing in relative terms, and does not work well otherwise. GNU tar is far more flexible in this sense.
- Bear in mind that tar does not care what exists in a directory. That is to say, it will overwrite any identically named files when an extract is performed to a specific location. If you or customers want to save versions of existing files, rename them before extracting.
dd -- This utility goes by many names, including 'disk dump' and 'direct dump'.What ever you call it, always give it the respect it deserves; dd does not care what exists on the destination side of your command, and will, by virtue of the level it operates at, overwrite very fundamental parts of the configuration in place. A good definition of the utility can be gleaned from the man page:
"dd copies the specified input file to the specified output with possible conversions. The standard input and output are used by default. The input and output block sizes may be specified to take advantage of raw physical I/O. Sizes are specified in bytes; a number may end with k, b, or w to specify multiplication by 1024, 512, or 2, respectively. Numbers may also be separated by x to indicate multiplication. dd will read the input one block at a time, using the specified input block size; it then will process the block of data actually returned, which could be smaller than the requested block size. dd will apply any conversions that have been specified and write the resulting data to the output in blocks of the specified output block size."
The main arguments you'll ever use with this utility are 'if' (input file), 'bs' (block size), and 'of' (output file). A typical dd command lookslike
```
dd if=<file> bs=<block size> of=<file or device file>
```
and will provide the following type of output:
```
# dd if=/etc/system bs=64k of=/dev/rmt/0mbn
0+1 records in
0+1 records out
```
The command above wrote the /etc/system file in 64k blocks to the tape drive controlled by /dev/rmt/0mbn. The '0+1' indicates no full records, one partial record at the given block size. To see if one can pull the fileofftape and read it, the following sequence of command can be issued:
```
mt -f /dev/rmt/0mbn rew
```
(after writing the dd from the last command,your are nowonefilemark out on tape, so you have to get back to the beginning).
```
# dd if=/dev/rmt/0mbn bs=64k of=/dev/null
0+1 records in
0+1 records out
```
The same number of full and partial records was returned at the given block size, which means you can go to the output file (/dev/null was used for purposes of example, so no file was created) and read the output.
**NOTE** The 'k' in '64k' is important; without it, the system defaults to bytes and output from the same command (sans the 'k') would look like this:
```
# dd if=/etc/system bs=64 of=/dev/rmt/0mbn
33+1 records in
33+1 records out
```
In summary, be VERY careful when using dd as it can really wreak havoc on a system if used carelessly. People have overwritten filesystems, vtocs, and other things of this sort. It is not recommended as a primary method by which to troubleshoot drives, but can be employed if necessary. Read the man page for a more thorough explanation of its capabilities.
ufsdump/ufsrestore -- Solaris provides means by which to perform backups and recoveries that do not require any additional software. The ufsdump and ufsrestore utilities are native to the operating system, and are actually quite effective for performing stores and restores. It is included here as a possible alternative or workaround for customers who you determine are having serious problems with the backup application they normally run. Additionally, this should be the recommended way that customers back up their / filesystem irrespective of what other software they typically use. Theufsdump command is generally used in a fairly simple way. It is issued with a number of arguments, including the level of backup (0-9, with 9 being the most granular incremental), additional options, a tape device file to write to, and the source to be written:
```
ufsdump <level><options> <dumpfile> <source>
```
It is a good idea to unmount filesystems and check them before backing them up. If only a directory is to be stored, this is obviously not a consideration. A typical ufsdump command and its output could look like this:
```
# ufsdump 0uf /dev/rmt/0 /etc/
  DUMP: Writing 32 Kilobyte records
  DUMP: Date of this level 0 dump: Sat Apr 14 17:57:23 2001
  DUMP: Date of last level 0 dump: the epoch
  DUMP: Dumping /dev/rdsk/c0t0d0s0 (smurf-sol8:/) to /dev/rmt/0.
  DUMP: Mapping (Pass I) [regular files]
  DUMP: Mapping (Pass II) [directories]
  DUMP: Estimated 44988 blocks (21.97MB).
  DUMP: Dumping (Pass III) [directories]
  DUMP: Dumping (Pass IV) [regular files]
  DUMP: Tape rewinding
  DUMP: 44158 blocks (21.56MB) on 1 volume at 277 KB/sec
  DUMP: DUMP IS DONE
```
The 'u' option instructs ufsdump to create an entry in the /etc/dumpdates file, where records of backups are kept. Adding an 'f' tells the utility that you will specify a dumpfile where you would like the backup stored. In most cases, this is a tape device file. Ufsdump will automatically rewind the tape when it's done. Using ufsrestore can be slightly more complicated, depending upon precisely what it is one intends to restore. For our purposes here, a very basic iteration of the command has been detailed simply to provide a sense of its usage. Like ufsdump, the syntax is straightforward:
```
ufsrestore <options> <dumpfile>
```
The ufsrestore utility requires that one of five options exist in the command string it is given. Without one there, you will get usage errors. The options it requires are:
- i -- interactive; sets up a text interface for users to pick files to be restored
- r -- recursive; dump back whatever is found on the tape
- R -- this stands for 'resume' and allows operations to continue from where they left off in the event of an interruption
- t -- table of contents; performs the same function as in tar in that it displays the files in a dump. It may be modified by pointing the utility to specific files, otherwise it takes root as the default.
- x -- extract files; this assumes a particular series of files that you want to restore, and is used only with specific modifiers.
In restoring the /etc/ filesystem from the previous example, the following command could be given, and the following output produced:
```
# ufsrestore rvf /dev/rmt/0
  Verify volume and initialize maps
  Media block size is 64
  Dump   date: Sat Apr 14 17:57:23 2001
  Dumped from: the epoch
  Level 0 dump of a partial file system on potsie:/etc/
  Label: none
  Begin level 0 restore
  Initialize symbol table.
  Extract directories from tape
  Calculate extraction list.
  ./lost+found: (inode 3) not found on volume
  ./var: (inode 5888) not found on volume
  ./usr: (inode 41216) not found on volume
  ./platform: (inode 123648) not found on volume
  Make node ./etc 	
  Make node ./etc/default
  Make node ./etc/cron.d
  Make node ./etc/dfs
  etc., etc. 
```
^*After the 'Make node' section, you again get a list of files and directories that were not found but contain a reference:
```
  ./bin: (inode 87) not found on volume
  ./dev: (inode 212008) not found on volume
  ./devices: (inode 212009) not found on volume
  ./kernel: (inode 341559) not found on volume
```
^*The utility then extracts files, creates symbolic links and ends with:
```
  Add links
  Set directory mode, owner, and times.
  Check the symbol table.
  Check pointing the restore
```
The man pages for both of these command can provide information about all options and uses. Suffice it to say, if you get to this level in troubleshooting a tape drive issue, customers are in fairly deep trouble and should use this as a last ditch attempt to get the data to tape.

To summarise

A lot can be tried prior to attacking a backup application directly. Most of the time you will find that the issues that come to us can be resolved by trying certain things at the operating system level. Knowledge of the software products is obviously necessary, but hopefully this introduces other angles from which to resolve tape problems.