Building a QFS shared file system tutorial
This article provides the steps necessary to create a QFS Shared file system
- Install, patch, and configure Sun QFS standalone
- Learn what processes run in QFS standalone environment, compare to SAM-QFS
- Learn shared filesystem concepts
- Successfully create a shared SAN filesystem
- Verify functionality
- Learn server failover concepts
- Successfully switch server and client roles between filesystem host
- Unconfigure environment
For this tutorial, we will use pinky as the server, and perky as the client in the shared filesystem configuration. This document is not meant to replace any training, and will be very basic. It is the readers responsibility to have access to the installation and configuration guides for SAM-FS/QFS (either hard-copy or online from docs.sun.com).
In the interests of saving time, I am assuming that the SAM-QFS package and product patch have already been installed, but be aware of what package names are for SAM-FS vs QFS standalone.
Configuring shared filesystems
- Verify that both systems have visibility to the storage that will be used for the shared filesystem (in this tutorial I am using a Sun StorEdge T3+ array):
s4u-sam# format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c0t0d0 /pci@1f,0/pci@1,1/ide@3/dad@0,0 1. c1t40d0 /pci@1f,0/pci@1/pci@3/SUNW,qlc@5/fp@0,0/ssd@w216000c0ff8015c6,0 2. c1t40d1 /pci@1f,0/pci@1/pci@3/SUNW,qlc@5/fp@0,0/ssd@w216000c0ff8015c6,1 3. c1t40d2 /pci@1f,0/pci@1/pci@3/SUNW,qlc@5/fp@0,0/ssd@w216000c0ff8015c6,2 4. c4t1d0 /pci@1f,0/pci@1/pci@3/SUNW,qlc@4/fp@0,0/ssd@w50020f2300000720,0 5. c4t1d1 /pci@1f,0/pci@1/pci@3/SUNW,qlc@4/fp@0,0/ssd@w50020f2300000720,1 Specify disk (enter its number):In this configuration, we will use
c4t1d0for metadata, and
c4t1d1for actual filesystem data. See SunQFS Filesystem Administrators Guide, page 5 for more information on metadata.
- Create a new file called
/etc/opt/SUNWsamfs/. Without this file, the daemons won't start and operations will fail.
s4u-sam# cat /etc/opt/SUNWsamfs/LICENSE.4.0 08b7f2b480c8a7c1c09598faaa720b88 8048cd8e0001935ed02f3985feb74671
- Configure the mcf file for the new filesystem.
Note: The shared flag in the example below. See SunQFS Filesystem Administrators Guide, page. 42 for more information on configuring mcf file
s4u-sam# cat /etc/opt/SUNWsamfs/mcf # Please verify these disk partitions are truly not in use samfs1 20 ma samfs1 - shared /dev/dsk/c4t1d0s0 21 mm samfs1 - /dev/rdsk/c4t1d0s0 /dev/dsk/c4t1d1s0 22 mr samfs1 - /dev/rdsk/c4t1d1s0
- Configure the
hosts.familynamefile for the QFS server. This file is responsible for defining the clients and servers that are allowerd to access the filesystem. See SunQFS Filesystem Administrators Guide, page 100 for more information on the hosts file
s4u-sam# cat /etc/opt/SUNWsamfs/hosts.samfs1 # HOST NAME HOST IP SERVER PRIORITY NOT USED SERVER HOST pinky 10.1.0.4 1 - server perky 10.1.0.234 2 -
- Send a HUP signal to
sam-fsddaemon on the server. This will make
sam-fsdreread the configuration files.
sammkfsto create the filesystem.
s4u-sam# sammkfs -S samfs1
- Configure the services file for tcp communications between server and any clients. Add the following line to the /etc/inet/services file (note the port number- this will be used again later):
samsock.samfs1 7105/tcp # QFS samfs1 portSend a HUP signal to inetd daemon to reread the services file.
- Verify that sam-sharefsd daemon is now running for this filesystem.
s4u-sam# ps -ef | grep sam root 2046 2038 0 20:54:30 ? 0:00 sam-sharefsd samfs1 root 2038 1 0 20:54:16 ? 0:00 /usr/lib/fs/samfs/sam-fsd root 3248 3182 0 13:15:17 pts/4 0:00 grep sam
- Create mount point for filesystem, and edit the /etc/vfstab file to add the new shared filesystem. The entry must have the shared option at the end of the line.
samfs1 - /samfs1 samfs - yes shared,bg
- Mount the filesystem, either through mountall command, or mount it manually
s4u-sam# mount samfs1 /samfs1
- The server is now ready to go. The steps will be the same on the client, specifying pinky as the filesystem server in the /etc/SUNWsamfs/hosts.samfs1 file.
- Once the client is configured, mount the filesystem on perky as well.
s4u-sam# mount samfs1 SAM-FS: Shared server is not responding SAM-FS: Shared server responded
- We now have successfully created a shared filesystem. You can verify the IP connection between the two hosts using netstat.
s4u-sam# netstat -a | grep sam pinky.32850 perky.samsock.samfs1 24820 0 48632 0 ESTABLISHED 30001761010 stream-ord 30001c25a18 00000000 /var/opt/SUNWsamfs/uds/Fsd
As long as both machines in this scenario are configured to communicate with the metadata device (both physical connectivity and through the mcf file), both machines are eligible to be servers for the filesystem.
Live Failover (when server is up)
On the metadata server, run the samsharefs command to define the new server.
s4u-sam# samsharefs -s perky samfs1 # # Host file for family set 'samfs1' # # Version: 3 Generation: 2 Count: 2 # Server = host 0/pinky, length = 80 # Pending Server = host 1/perky #
Failover will take about 60 seconds. During this time, samsharefs commands won't come back, but eventually you'll see the new server established.
Now on either machine we can see who the new server is.
s4u-sam# samsharefs samfs1 # # Host file for family set 'samfs1' # # Version: 3 Generation: 3 Count: 2 # # Server = host 1/perky, length = 80 pinky 10.1.0.4 1 - perky 10.1.0.234 2 - server
Server Failover when server is down
Typically, if the metadata server is having issues, you'll want to have it either completely disconnected or powered off to ensure that it does not try to regain control.
Issue an init 0 to whichever box has server access to the filesystem (use samsharefs command to determine this). Then, use the samsharefs command to force failover to one of the clients that can be used as an alternate server.
s4u-sam# samsharefs -R -s perky samfs1 # # Host file for family set 'samfs1' # # Version: 3 Generation: 4 Count: 2 # # Server = host 1/perky, length = 80 perky 10.1.0.234 2 - server pinky 10.1.0.4 1 -
These steps are specific for QFS standalone. There are additional procedures for failover on SAM-QFS configurations. These steps are discussed in the Sun SAM-QFS File System Administrators Guide, page 125.
When mounting the filesystem (manually or through vfstab), there are several shared filesystem specific options that can be used. These options are beyond the basic scope of this lab, but are worth knowing. These can be foind in the Sun SAM-QFS File System Administrators Guide page 132.