Quick Sun StorEdge A5000 Troubleshooting Guide

This article provides a quick overview of the Sun StorEdge A5x00 storage array with basic outlines and a troubleshooting aide.

A5000 key components

  • Hot pluggable units:
    • power supplies
    • front panel module
    • front and rear fans
    • ib's
    • gbics
    • disk drives
  • Units not hot pluggable:
    • ac power filter
    • interconnect assembly
    • backplane

Basic drive architecture for fibre channels drives

Dual port loop interactions:

  • fibre channel domains for loop a and b are independent,
  • common processor services both a and b loop.
  • there exist some loop events that can effect the other loop.
  • Some drive failures can cause failures on both loops.

General Info

  • Maximum enclosures per loop:
    • A5000 -- 4 per loop
    • A5100 -- 3 per loop
    • A5200 -- 3 per loop
  • Fibre topologies:
    • point to point - used by SSA's
    • fc-al - (arbitrated loop) A5x00 and T3's
    • fabric - not implimented by sun at this time
  • minimum config:
    • A5000 -- (5 disks) f3, f6, r0, r3, r6
    • A5100/A5200 -- (7 disks) f0, f5, f10, r0, r3, r6, r10
  • hub configurations:
    • do not use ports 0 and 6 until required.
    • Start host connections at port 4 and work right
    • start photon connections at port 3 and work left
  • disk speeds and transfer rate:
    • 7200 rpm -- 12.75mps
    • 10000 rpm -- 17.9 mps

A5x00 drivers

  • socal -- the socal driver recieves requests for Sbus FC-100 activity from any driver and maps the requests and responses to the standard FC-AL format. SUNWluxl (32-bit), SUNWluxlx (64-bit)
  • sf -- the sf driver executes SCSI commands through the Sbus FC-100 host bus adapter. It maps these requests to the format required by the socal driver. It also providescommand throttling services for the host adapter. SUNWluxd, SUNWluxdx (64-bit)
  • ifp -- the ifp driver recieves requests for PCI FC-100 activityfrom any driver and maps the requests and responses to the standard FC-AL format. It executes SCSI commands through the PCI FC-100 host bus adapter. It also provides command throttling services for the host adapter. SUNWifp,SUNWifpx (64-bit)
  • ssd -- the ssd driver provides all the standard disk driver capabilities. SUNWssad
  • ses -- the ses driver provides access to the enclosure services SUNWses, SUNWsesx (64-bit)

Device addressing

The physical path is identified through the hosts device tree.

/devices/sbus@a,0/SUNW,socal@2,0/sf@1,0/ssd@w21000020370b34ca,0

where:

pathdescription
sbus@a,0 sbus slot
SUNW,socal@2,0 slot on io board that contains the hba
sf@1,0 slot on the hba that contains the gbic (1 is left, 0 is right)
ssd@w21000020370b34c data path w21 indicates ib:a w22 indicates ib:b
ssd@w21000020370b34ca the 'a' defines the photon soc port. 1 or 9 = a0 2 or a = a1 3 or b = b0 4 or c = b1
/pci@4,2000/SUNW,ifp@1/ses@w5080020000006e3b,0

where:

pathdescription
pci@4,2000 pci slot
SUNW,ifp@1 hba (replaces the sbus socal and sf)
sf@1,0 hba gbic slot (PCI FC-100 only has one)
ses@w5080020000006e3b data path

Locating a disk by it's AL_PA number

There are two means of locating a disk by it's al_pa.

  1. Use the table below,
  2. or use the luxadm -e dump_map <boxname> command.

Example using luxadm

# luxadm -e dump_map a|more
Pos AL_PA ID Hard_Addr Port WWN Node WWN Type
0   2  7c 2 20020800207e147d 200c0800207e147d 0xff (Unknown Type)
1   d2 d d2 508002000002bbb9 508002000002bbb8 0xd (SES device)
2   ef 0 ef 210000203713121a 200000203713121a 0x0 (Disk device)----------------
3   e8 1 e8 2100002037131529 2000002037131529 0x0 (Disk device)
4   e4 2 e4 2100002037131098 2000002037131098 0x0 (Disk device)
5   e2 3 e2 210000203713126e 200000203713126e 0x0 (Disk device) Front Backplane
6   e1 4 e1 21000020370ed14a 20000020370ed14a 0x0 (Disk device)
7   e0 5 e0 21000020371313e6 20000020371313e6 0x0 (Disk device)
8   dc 6 dc 21000020371311d4 20000020371311d4 0x0 (Disk device)----------------
9   b5 1d b5 508002000002bbba 508002000002bbb8 0xd (SES device)
10  cd 10 cd 21000020370ee53b 20000020370ee53b 0x0 (Disk device)--------------
11  cc 11 cc 21000020371315c6 20000020371315c6 0x0 (Disk device)
12  cb 12 cb 210000203713176a 200000203713176a 0x0 (Disk device)
13  ca 13 ca 210000203713172b 200000203713172b 0x0 (Disk device)  Rear backplane
14  c9 14 c9 2100002037131543 2000002037131543 0x0 (Disk device)
15  c6 16 c6 21000020370e6c7f 20000020370e6c7f 0x0 (Disk device)--------------
OrderDeviceBox ID 0Box ID 1Box ID 2Box ID 3
    AL_PA SEL_ID AL_PA SEL_ID AL_PA SEL_ID AL_PA SEL_ID
1 IB node D2 0D 9E 2D 5A 4D 29 6D
2 Front Disk 0 EF 0/0 B2 20/32 72 40/64 3A 60/96
3 Front Disk 1 E8 01/1 B1 21/33 71 41/65 39 61/97
4 Front Disk 2 E4 02/2 AE 22/34 6E 42/66 36 62/98
5 Front Disk 3 E2 03/3 AD 23/35 6D 43/67 35 63/99
6 Front Disk 4 E1 04/4 AC 24/36 6C 44/68 34 64/100
7 Front Disk 5 E0 05/5 AB 25/37 6B 45/69 33 65/101
8 Front Disk 6 DC 06/6 AA 26/38 6A 46/70 32 66/102
9 Front Disk 7 DA 07/7 A9 27/39 69 47/71 31 67/103
10 Front Disk 8 D9 08/8 A7 28/40 67 48/72 2E 68/104
11 Front Disk 9 D6 09/9 A6 29/41 66 49/73 2D 69/105
12 Front Disk 10 D5 0A/10 A5 2A/42 65 4A/74 2C 6A/106
13 IB node B5 1D 75 3D 45 5D 01 7D
14 Rear Disk 0 CD 10/16 98 30/48 55 50/80 25 70/112
15 Rear Disk 1 CC 11/17 97 31/49 54 51/81 23 71/113
16 Rear Disk 2 CB 12/18 90 32/50 53 52/82 1F 72/114
17 Rear Disk 3 CA 13/19 8F 33/51 52 53/83 1E 73/115
18 Rear Disk 4 C9 14/20 88 34/52 51 54/84 1D 74/116
19 Rear Disk 5 C7 15/21 84 35/53 4E 55/85 1B 75/117
20 Rear Disk 6 C6 16/22 82 36/54 4D 56/86 18 76/118
21 Rear Disk 7 C5 17/23 81 37/55 4C 57/87 17 77/119
22 Rear Disk 8 C3 18/24 80 38/56 4B 58/88 10 78/120
23 Rear Disk 9 BC 19/25 7C 39/57 4A 59/89 0F 79/121
24 Rear Disk 10 BA 1A/26 7A 3A/58 49 5A/90 08 7A/122

Troubleshooting tools

  • FPM -- front panel module.
    • enable and disable fru's and non-fru parts
    • check fru and non fru status
    • check node count - what is node count counting? # of disks, # of hosts (NOT HBA's), and # of ib's.
  • luxadm -- SUNWluxop
  • explorer
  • PFP -- photon fun pack. Works very well with explorer data. No longer being developed, replaced by STORtools.
        commands:
    finchk*        infobase*      miscchk*       the-man*
    disklist*      getall*        lineget*       patchchk*      
    diskrev*       hbarev*        maploop*       
    encrev*        hwchk*         messagechk*

    to use pfp with explorer data; unpack the tar file and set the following 2 variables

    EXPLORER_DATA [data directory]
    EXPLORER_OUTPUT [command output directory

    then run getall

  • lbf -- also part of STORtools common patterns:
    7e7e7e7e - checks cables and gbics
    aa55aa55 - checks boards (ib, hba, centerplanes) and drive ports
    4a4a4a4a - checks boards (ib, hba, centerplanes) and drive ports
    b5b5b5b5 - checks boards (ib, hba, centerplanes) and drive ports

    usage:

    lbf -t 7e7e7e7e -n 10000 /devices/sbus@a,0/SUNW,socal@2,0:0
  • STORtools -- diagnostic software for FC-AL storage devices

Basic drive architecture for fibre channel drives

System Messages

  • HBA attach message (socal)
    Jan 12 08:11:15 shamu unix: SUNW,socal0 is /sbus@2,0/SUNW,socal@d,10000
    socal0 = socal instance sbus@2,0 = sbus number socal@d = sbus slot number
  • Loop attach (sf)
    Jan 12 08:11:15 shamu unix: sf0 is /sbus@2,0/SUNW,socal@d,10000/sf@0,0
    sf0 = sf instance sbus@2,0 = sbus number socal@d = sbus slot number sf@0,0 = port on hba
  • Disk attach (ssd)
    Jan 12 08:11:15 shamu unix: ssd0 is /sbus@e,0/SUNW,socal@d,10000/sf@1,0/ssd@w22000020370ee9b0,0
    ssd0 = ssd instance sbus@2,0 = sbus number socal@d = sbus slot number sf@0,0 = port on hba 22000020370ee9b0 = disk port's world wide number
  • Enclosure services attach (ses)
    Jan 12 08:11:15 shamu unix: ses10 is /sbus@e,0/SUNW,socal@d,10000/sf@1,0/ses@w5080020000022b33,0
    sbus@2,0 = sbus number socal@d = sbus slot number sf@0,0 = port on hba 5080020000022b33 = ib port world wide number