Sun Management Center - A Technical Overview

The Sun Management Center software is an advanced Systems Management Software solution from Sun Microsystems .

Sun Management Center (SunMC) is an open, extensible system monitoring and management solution that uses Java software and Simple Network Management Protocol (SNMP) to provide an integrated and comprehensive enterprise-wide management of Sun products and their subsystems, components and peripheral devices.

This article provides a technical overview for troubleshooting of the SunMC product

Installation, Upgrade and Removing SunMC

System Requirements

Table 1 — Installation requirements
System Ultra 1 (Ultra 2 recommended)
128mb memory (256 recommended)
128mb swap (200 recommended)
Storage 100mb — /opt/SUNWsymon
50mb — /var/opt/SUNWsymon
Operating Systems Solaris 2.5.1
Solaris 2.6
Solaris 2.7
Patches Solaris 2.6
105181-15
105210-19
105284-25
105490-07
105568-13
105633-21
105669-07
106040-12
Solaris 2.7
106980-05
107078-10
107607-01
107636-01

Installing

Use the following steps to install SunMC:

  1. Download the latest release of SunMC from http://www.sun.com/software/sunmanagementcenter/download/ (latest 2.1)
  2. Uncompress and untar the package
  3. go to the directory where the package resides, and run the install script:
    # cd /opt/sunmc2.1
    # ./es-inst
  4. when prompted for the agent port, change it from 161 to 1161, or simply disable snmpdx (recommended to change the port to 1161)
  5. add sun-mc users to /etc/group and /var/opt/SUNWsymob/cfg/esusers if the installation do not add them

Uninstalling

To uninstall SunMC, following these steps:

  1. Change to the package install directory:
    # cd /opt/SUNWsymon/sbin
  2. Stop all SyMON services:
    # ./es-stop -a
  3. Run the uninstall script:
    # ./es-uninstall
  4. Remove configuration files:
    # rm -rf /var/opt/SUNWsymon

Upgrading

  1. It is recommended to follow the uninstall procedure first with one change:
    1. Instead of removing the /var/opt/SUNWsymon folder, just rename it so that configuration files can still be retrieved.
  2. now follow the steps for normal installation

Product Info

Components

SunMC consists of three components:

Functionality

Table 2 — SunMC components
ComponentSizePlatform
Console 25mb SPARC, Windows 95/98, WinNT
Server 45mb SPARC only
Agent Layer 25mb SPARC only

Support Hardware

  • All SPARC platforms (inc Enterprise E10000)
  • Printers
  • Gateways

Functionality

SunMC provides the following functionality:

  • GUI based monitoring console
  • alarm manager
  • performance and data management
  • hardware configuration manager
  • process monitoring
  • discovery (automatically adds new machines on the network)
  • log file monitoring

Architecture overview

Console layer

  • login access via unix id & password
  • GUI is written in java
  • interface to the server layer
  • hierarchy view on the left side
  • topology view on the right side
  • double click on a host to bring up the host details view, where alarms are set.
    • Host details view contains 7 tabs
      1. info — basic system info
      2. browser — hierarchy view of hardware, OS, local apps, remote systems
      3. alarms — current and unacknowledged alarms
      4. processes — monitored processes (empty by default)
      5. log view — view of /var/adm/messages or SunVTS diag files
      6. applications
      7. hardware — hardware info and views
    • attribute editor
      • brought up by selecting an item in the hierarchy view and clicking on attributes button or right clicking and selecting attribute editor from the menu.
      • Contains 5 tabs
        1. info
        2. alarm — set alarm thresholds
        3. actions — set actions for alarms
          • alarm events are filtered by severity, state, ack/unack
          • alarm events are sorted by start time, severity, and state
        4. refresh
        5. history

Server layer

  • supports the console layer
  • receives asynchronous alarms
  • dissimulates alarms to interested clients
  • interface between console and agent configurations

Agent layer

  • daemon that facilitates monitoring
  • interfaces with the monitoring modules

Modules overview

SunMC modules define:

  • data collection
  • data storage
  • alarm parameters
  • alarm rules
  • alarm actions

All module customisations are made in /var/opt/SUNWsymon/cfg. These files can be copied to other systems for easy administration.

Module examples

  • file scanning module
    • specify a file(s) to watch and regex to look for
  • kernel reader
    • swap
    • cpu
    • load
    • processes
    • disk space
    • network errors
  • process monitoring module
    • monitor specified process(s)

Setting Alarms

SunMC alarms are set from the host detail view:

  • choose (or load a module) from the hierarchy view on the left
    • the right frame will display module info in a table format
    • select a cell to alarm on, for example:
      filesystem usage -> kb used
    • click on the attributes button and select the alarms tab
    • enter thresholds, for example:
      critical threshold < 90
    • select the actions tab and enter an action if needed, for example:
      email [userid]
      • custom alarm actions
        • alarm action scripts must be stored in /var/opt/SUNWsymon/bin folder
        • email.sh is the only factory script included

SunMC daemons

  • SunMC Server
    start with /opt/SUNWsymon/sbin/es-start -s
  • Configuration Server — provides SunMC security and authenticates console logins
    starts with /opt/SUNWsymon/sbin/es-start -f
  • Topology Agent — provides SunMC domain management
    start with /opt/SUNWsymon/sbin/es-start -p
  • Trap Handler — receives alarm notifications (communicates through snmp traps)
    start with /opt/SUNWsymon/sbin/es-start -t
  • Event Management Agent — maintains event info and alarms
    start with /opt/SUNWsymon/sbin/es-start -e
  • Agent
    start with /opt/SUNWsymon/sbin/es-start -a

GUI view of daemons and laters

  • Console layer
    • console
    • requires that the server layer is running on the server you login to.
  • Server layer
    • configration server
    • topology
    • server
    • trap handler
    • event manager
  • Agent layer
    • agent

Troubleshooting

source tree

  • /opt/SUNWsymon — factory scripts, binaries, default config
  • /var/opt/SUNWsymon — custom scripts, binaries and configurations

resolving port issues

  • file: /var/opt/SUNWsymon/cfg/domain-config.x
  • look for and change the snmpPort setting, for example:
    agent = { snmpPort = 161 }
  • change 161 to 1161
  • stop and start symon agent
    # /opt/SUNWsymon/sbin/es-stop -A
    # /opt/SUNWsymon/sbin/es-start -A

checking port conflicts

  • Run netstat -an | grep [port number]
  • port 161 is the most common conflict. Recommendation is to configure all ports +1000 (ie: 161 = 1161)
  • that way a netstat -an | grep 116 will show all sunMC ports

checking log files

  • In /var/opt/SUNWsymon/log/ directory:
    • agent.log — err, warn, info, debug, trace
    • agentStatus.log — status
    • alarms.log
    • cfgserver.log — err, warn, info, debug, trace
    • event.log — err, warn, info, debug, trace
    • eventStatus.log — status
    • history.log
    • predictive.failure
    • server.log — err, warn, info, debug, trace, data, snmp
    • topology.log — err, warn, info, debug, trace
    • topologyStatus.log — status
    • trap.log — err, warn, info, debug, trace
  • to view the log files use the es-run command with the ctail or ccat commands. For example:
    # /opt/SUNWsymon/sbin/es-run ctail -f /var/opt/SUNWsymon/log/server.log
    Note: You must specify the absolute log path

trouble shooting agent problems

  • run the agent in interactive mode:
    # /opt/SUNWsymon/sbin/es-start -ai

checking for processes

  • Determine which process running:
    # ps -ef | grep esd
     root  588     1  0   Dec 04 ?       18:39 esd - init topology -dir /var/opt/SUNWsymon
     root  586     1  3   Dec 04 ?       90:55 esd - init agent -dir /var/opt/SUNWsymon
     root  587     1  0   Dec 04 ?        3:53 esd - init trap -dir /var/opt/SUNWsymon
     root  589     1  0   Dec 04 ?        6:25 esd - init cfgserver -dir /var/opt/SUNWsymon
     root  590     1  0   Dec 04 ?        3:39 esd - init event -dir /var/opt/SUNWsymon
    

enable creation of core files

  • set the ESDEBUG environment variable to true and repostart the agent:
    # ESDEBUG=true
    # export ESDEBUG
    # /opt/SUNWsymon/sbin/es-start -A
  • Core files located in /var/opt/SUNWsymon/cfg

exporting the display

  • the new java gui is not entirely functional on a remote display and should not be expected to work properly
  • it should not be expected to work at all using exceed. Use the windows console instead

cannot login

  • user id is not in the /var/opt/SUNWsymon/cfg/esusers file

cannot set alarms

  • user id is not in /etc/group with the esdomadm and esadm groups

Further information

The following links will provide additional information on SunMC: