Sun SAM-FS / QFS 4.0 Concepts

This article provides a list of things to know, product features and HSM concepts for the SUN SAM-FS/QFS 4.0 product base.

Thing To Know

  • What’s the package name?
    Table 1 — Sun SAM-FS/QFS package names
    SAM versionPackage names
    3.5.0-xx LSCsamfs, LSCqfs
    4.0.xx SUNWsamfs, SUNWqfs
  • What’s the current version? 4.0 (this is always changing)
  • Where do I get it? sun.com
  • Websites for reference:
  • What's licensed with SAM-FS? EVERYTHING
  • Locations to remember
    • /etc/opt/SUNWsamfs — configuration files (archiver.cmd, defaults.conf, mcf, etc)
    • /opt/SUNWsamfs — binaries, MAN pages, example files, docs, JRE binaries
    • /opt/SUNWsamfs/sbin/info.sh — generate a SAMreport (required for escalation to Sun)

SAM 4.0 Changes

  • Shared writer
  • File system quotas for Sun QFS
  • ACL (Access Control Lists)
  • Archive scheduling
  • Disk Archiving — SAM-FS can now be configured to archive to a remote host’s magnetic disk
  • System Error Facility (SEF) — Allows capture and compilation of log sense pages of peripheral tape devices
  • File system daemon sam-fsd
  • md devices on QFS (not DiskSuite)
  • Show stage requests waiting for media
  • Operator notification when exported or manually mounted media is requested
  • samfsconfig command and man pages
  • New option to archive copy commands — '-M' option specifies metadata only
  • Stager daemon — sam-stagerd
  • samunhold command and man pages — Assists administration of SANergy file sharing on QFS
  • Support for new devices
    • Libraries — ADIC Scalar 1000 AIT, Exabyte x80, IBM 3584 UltraScalable, StorageTek L20, L40, L80.
    • Drives — IBM 3580 (LTO), Quantum SDLT220 (SuperDLT), AIT in ADIC using DAS/ACI interface.
  • samcheck command and man pages — File system block usage identifier
  • Support for ACSLS 6.0
  • Change to the drive selection algorithm for StorageTek Passthru automated libraries
  • New mount option — 'hwm_archive' was added to direct FS to start archiver when high threshold is reached

Differences

UFSSAM-FS
  • Typically one disk partition
  • File size maximum = 1 Terabyte
  • Maximum number of files limited by inode number
  • File system size is static (generally)
  • Up to 252 disk partitions
  • File size maximum = 264 –1
  • Maximum number of files = 255–1
  • File system can be grown dynamically (even when at 100%)

SAM-FS specific commands

Although SAM-FS file systems can be navigated using standard UNIX commands, these commands do not recognize the extended attributes of files in SAM-FS file systems. In order to accurately view and manipulate these files, there are SAM-FS variants of some commands that access the extended inode information. These commands include sls, sfind, squota, sdu, and ssum

There are also special commands to manipulate archiving, staging, and releaser settings. These commands include archive, stage, segment, and release, among others. These commands are also documented in the SAM-FS cheat sheet, with examples for common procedures

inodes

  • Has more information than UFS inodes
  • Stored on disk in .inodes, NOT on tape
  • Uses dual storage DAU (disk allocation units)

SAM-FS uses a patented “dual” storage allocation unit mechanism. UFS uses a standard static block size of 8192 bytes, but allows for fragmentation at 1024 bytes. The term DAU refers to the amount of disk space allocated rather than the conventional 8192 file system block size. SAM-FS uses two DAUs rather than the standard file system block size used by UFS. The small DAU is 4096 bytes and the large DAU is 16384 bytes (default), 32768 bytes, or 65536 bytes. The configurable large DAU allows good access performance for large files without incurring storage utilization problems for small files.

Primary functions - Archive

Archive — write copies of files to near-line or off-line media (tape, magnetic disk, WORM optical)

Placing a file in a SAM-FS file system triggers the archiving operation. SAM-FS automatically makes one archive copy to a piece of removable media. The archiver’s activities are governed by directives defined in the archiver.cmd file. By setting parameters in this file, the system administrator can customize the archiving process to meet varying requirements of the user community. SAM-FS provides the ability to create archive sets. Archive sets are groupings of data that match a particular criterion such as minimum size, maximum size, owner, group, or directory location. The archive sets control how long to wait before archiving the data, the destination of the archive copy, and how long to keep the copy archived.

Up to four copies of a file may be dynamically and automatically written to removable media. The archiver may group archived files together with other files depending on file sizes.

Files can be released leaving a stub on the disk cache. This allows you to open the file and read it without staging it onto disk. When you modify files, all existing archive copies are marked stale. The archiver will schedule new archive copies based on the directives set in the archiver.cmd file. When the new archive copies are created, SAM-FS no longer points to the stale archive copies although the stale archive copies continue to reside on removable media.

Primary functions - Release

Release — maintain the on-line disk space of the file system

The SAM-FS releaser manages the usage of the disk cache. Two threshold values manage the cache. When disk space exceeds the high threshold value (for example 90%), the releaser automatically releases archived files from the disk until the available disk space reaches the low threshold (for example 70%). This defines the hysteresis (i.e. lag) window for on-line storage. You configure the initial disk thresholds with the mount(1M) command. You can change the thresholds at any time through the command interfaces. The user may release a file immediately with the release(1M) command. The super user may set the “never release” flag so that the file always remains on the disk cache. When the “partial release” flag is set, the file is released, but the first portion of the file is retained on-line. Applications such a filemgr(1) read only the beginning of the file.

Primary functions - Stage

Stage — retrieve archive copies of files back to on-line disk as required

Once a file is released, staging is the process of retrieving file data back to disk cache. When you access a file that has been released, staging occurs. SAM-FS organizes stage requests by media type and VSN. When the system receives new requests, it queues them dynamically and puts them in the most logical position based on media type and VSN. For a sequential read of a near-line file, the read operation tracks along directly behind the staging operation. This means that the stage does not have to be complete before the system begins returning data to the user. SAM-FS satisfies the user’s request by supplying that portion of the file back on line as needed to respond to the request. The user has the option to stage the file immediately with the stage(1) command. There’s an option to wait or not wait for the stage to complete. Using the stage(1) command, the user may also specify which archive copy to retrieve.

Primary functions — Associateve staging

Associative Staging — if any files with an “a” are referenced, all others will be staged

SAM-FS functions like a traditional file system, retrieving or storing files as requested by users or applications. In some situations this file-by-file model may not be appropriate such as when a project consists of multiple files, or text files linked to graphics files. One approach to this problem is to have the user construct lists of related files and manually execute the staging, or use the HSM software to process the list of files. This approach requires significant intervention by the user and may be prone to errors.

Users can set the Associative Staging attribute to a file OR a directory. When the Associative Staging attribute is set, accessing that file stages every file in the requested file’s directory (that also has the attribute enabled).

Primary Functions — Direct access & pre-staging

Direct Access and Pre-Staging — If file is marked for direct access, it will be read directly from the removable media device

To facilitate the efficient use of on-line storage and provide quick access to near-line data, users can mark a file with the “never stage” attribute. This means SAM-FS will access the file directly from the removable media. SAM-FS does not retrieve file data back into disk cache and the file remains off-line. Direct access allows you to efficiently access large near-line databases.

SAM-FS can retrieve a record with direct access in much less time than with systems which first stage the file to on-line storage.

SAM-FS provides support for direct access on files which reside on tape or optical disk.

For applications which need to access large portions of the data, you can pre-stage the file to disk. Although not a requirement on the part of the user, pre-staging may provide improved usage of device resources by allowing stage requests for files resident on the same archive media to be batched together.

File Attributes

  • Stored in the SAM-FS inode (.inodes)
  • Use sls to view extended file attributes
    • sls –D
    • sls -2
  • Set with user commands archive, release, stage

SAMreport

  • The SAMreport is your best friend if you’re not sure where to go
  • Collect by running info.sh (in /opt/SUNWsamfs/sbin/). Drops file in /tmp/SAMreport. These can get pretty big, so it may be in more than one piece
  • If you’re troubleshooting a SAM-FS problem, and not a system problem, this will be better than an explorer for you