Title and Copyright Information
About This Manual
New and Changed Features
Related Documents
Reader's Comments
1    Introduction to Performance and Availability
1.1    Performance Terminology and Concepts
1.2    High-Performance Configurations
1.2.1    CPU Resources
1.2.2    Memory Resources
1.2.3    Disk Storage    RAID Technology    SCSI Concepts    Data Paths    Transmission Methods    SCSI Bus Speeds    SCSI Bus Length and Termination
1.2.4    Network Subsystem
1.3    High-Availability Configurations
2    Planning a High-Performance and High-Availability Configuration
2.1    Identifying a Resource Model for Your Workload
2.2    Identifying Performance and Availability Goals
2.3    Choosing System Hardware
2.3.1    CPU Configuration
2.3.2    Memory and Swap Space Configuration    Determining Your Physical Memory Requirements    Choosing a Swap Space Allocation Mode    Determining Swap Space Requirements
2.3.3    I/O Bus Slot Capacity
2.3.4    Support for High-Performance Disk Storage
2.3.5    Support for High-Performance Network
2.4    Choosing Disk Storage Hardware
2.4.1    Fast Disks
2.4.2    Solid-State Disks
2.4.3    Devices with Wide Data Paths
2.4.4    High-Performance Host Bus Adapters
2.4.5    DMA Host Bus Adapters
2.4.6    RAID Controllers
2.4.7    Fibre Channel
2.4.8    Prestoserve
2.5    Choosing How to Manage Disks
2.5.1    Using a Shared Pool of Storage for Flexible Management
2.5.2    Striping Data or Disks to Distribute I/O
2.5.3    Using Parity RAID to Improve Disk Performance
2.6    Choosing a High-Availability Configuration
2.6.1    Using a Cluster for System Availability
2.6.2    Using RAID for Disk Data Availability
2.6.3    Using Redundant Networks
2.6.4    Using Redundant Power Supplies and Systems
3    Monitoring Systems and Diagnosing Performance Problems
3.1    Obtaining Information About System Events
3.1.1    Using Event Manager
3.1.2    Using DECevent
3.2    Using System Accounting and Disk Quotas
3.3    Continuously Monitoring Performance
3.3.1    Using Performance Manager
3.3.2    Using Performance Visualizer
3.4    Gathering Performance Information
3.5    Profiling and Debugging Kernels
3.6    Accessing and Modifying Kernel Subsystems
3.6.1    Displaying the Subsystems Configured in the Kernel
3.6.2    Displaying Current Subsystem Attribute Values
3.6.3    Displaying Minimum and Maximum Attribute Values
3.6.4    Modifying Attribute Values at Run Time
3.6.5    Modifying Attribute Values at Boot Time
3.6.6    Permanently Modifying Attribute Values
3.6.7    Displaying and Modifying Kernel Variables by Using the dbx Debugger
4    Improving System Performance
4.1    Steps for Configuring and Tuning Systems
4.2    Tuning Special Configurations
4.2.1    Tuning Internet Servers
4.2.2    Tuning Large-Memory Systems
4.2.3    Tuning NFS Servers
4.3    Checking the Configuration by Using the sys_check Utility
4.4    Solving Common Performance Problems
4.4.1    Application Completes Slowly
4.4.2    Insufficient Memory or Excessive Paging
4.4.3    Insufficient Swap Space
4.4.4    Processes Swapped Out
4.4.5    Insufficient CPU Cycles
4.4.6    Disk Bottleneck
4.4.7    Poor Disk I/O Performance
4.4.8    Poor AdvFS Performance
4.4.9    Poor UFS Performance
4.4.10    Poor NFS Performance
4.4.11    Poor Network Performance
4.5    Using the Advanced Tuning Guidelines
5    Tuning System Resource Allocation
5.1    Tuning Process Limits
5.1.1    Increasing System Tables and Data Structures
5.1.2    Increasing the Maximum Number of Processes
5.1.3    Increasing the Maximum Number of Threads
5.2    Tuning Program Size Limits
5.2.1    Increasing the Size of a User Process Stack
5.2.2    Increasing the Size of a User Process Data Segment
5.3    Tuning Address Space Limits
5.4    Tuning Interprocess Communication Limits
5.4.1    Increasing the Maximum Size of a System V Message
5.4.2    Increasing the Maximum Size of a System V Message Queue
5.4.3    Increasing the Maximum Number of Messages on a System V Queue
5.4.4    Increasing the Maximum Size of a System V Shared Memory Region
5.4.5    Increasing the Maximum Number of Shared Memory Regions Attached to a Process
5.4.6    Modifying Shared Page Table Sharing
5.5    Tuning the Open File Limits
5.5.1    Increasing the Maximum Number of Open Files
5.5.2    Increasing the Maximum Number of Open File Descriptors
6    Managing Memory Performance
6.1    Virtual Memory Operation
6.1.1    Physical Page Tracking
6.1.2    File System Buffer Cache Memory Allocation    Metadata Buffer Cache Memory Allocation    Unified Buffer Cache Memory Allocation    AdvFS Buffer Cache Memory Allocation
6.1.3    Process Memory Allocation    Process Virtual Address Space Allocation    Virtual Address to Physical Address Translation    Page Faults
6.1.4    Page Reclamation    Modified Page Prewriting    Reclaiming Memory by Paging    Reclaiming Memory by Swapping
6.2    Configuring Swap Space for High Performance
6.3    Gathering Memory Information
6.3.1    Monitoring Memory by Using the vmstat Command
6.3.2    Monitoring Memory by Using the ps Command
6.3.3    Monitoring Swap Space Usage by Using the swapon Command
6.3.4    Monitoring the UBC by Using the dbx Debugger
6.4    Tuning to Provide More Memory to Processes
6.4.1    Reducing the Number of Processes Running Simultaneously
6.4.2    Reducing the Static Size of the Kernel
6.4.3    Decreasing the Borrowed Memory Threshold
6.4.4    Decreasing the Size of the AdvFS Buffer Cache
6.4.5    Decreasing the Memory for AdvFS Access Structures
6.4.6    Decreasing the Size of the Metadata Buffer Cache
6.4.7    Decreasing the Size of the namei Cache
6.4.8    Increasing the Memory Reserved for Kernel malloc Allocations
6.5    Tuning Paging and Swapping Operation
6.5.1    Increasing the Paging Threshold
6.5.2    Increasing the Rate of Swapping
6.5.3    Decreasing the Rate of Swapping
6.5.4    Enabling Aggressive Task Swapping
6.5.5    Limiting the Resident Set Size to Avoid Swapping
6.5.6    Increasing Modified Page Prewriting
6.5.7    Decreasing Modified Page Prewriting
6.5.8    Increasing the Size of the Page-In and Page-Out Clusters
6.5.9    Increasing the Swap I/O Queue Depth for Page Ins and Swap Outs
6.5.10    Decreasing the Swap I/O Queue Depth for Page Ins and Swap Outs
6.5.11    Increasing the Swap I/O Queue Depth for Page Outs
6.5.12    Decreasing the Swap I/O Queue Depth for Page Outs
6.6    Reserving Physical Memory for Shared Memory
6.6.1    Tuning the Kernel to Use Granularity Hints
6.6.2    Modifying Applications to Use Granularity Hints
7    Managing CPU Performance
7.1    Gathering CPU Performance Information
7.1.1    Monitoring CPU Usage by Using the ps Command
7.1.2    Monitoring CPU Statistics by Using the vmstat Command
7.1.3    Monitoring the Load Average by Using the uptime Command
7.1.4    Checking CPU Usage by Using the kdbx Debugger
7.1.5    Checking Lock Usage by Using the kdbx Debugger
7.2    Improving CPU Performance
7.2.1    Adding Processors
7.2.2    Using the Class Scheduler
7.2.3    Prioritizing Jobs
7.2.4    Scheduling Jobs at Offpeak Hours
7.2.5    Stopping the advfsd Daemon
7.2.6    Using Hardware RAID to Relieve the CPU of I/O Overhead
8    Managing Disk Storage Performance
8.1    Guidelines for Distributing the Disk I/O Load
8.2    Monitoring the Distribution of Disk I/O
8.3    Displaying Disk Usage by Using the iostat Command
8.4    Managing LSM Performance
8.4.1    LSM Features
8.4.2    Basic LSM Disk, Disk Group, and Volume Guidelines    Initializing LSM Disks as Sliced Disks    Sizing the rootdg Disk Group    Sizing Private Regions    Making Private Regions in a Disk Group the Same Size    Organizing Disk Groups    Mirroring the Root File System    Mirroring Swap Devices    Saving the LSM Configuration
8.4.3    LSM Mirrored Volume Configuration Guidelines    Placing Mirrored Plexes on Different Disks and Buses    Using Multiple Plexes in a Mirrored Volume    Choosing a Read Policy for a Mirrored Volume    Using a Symmetrical Plex Configuration    Using Hot Sparing for Mirrored Volumes
8.4.4    Dirty-Region Logging Configuration Guidelines    Configuring Log Plexes    Using the Correct Log Size    Placing Logging Subdisks on Infrequently Used Disks    Using Solid-State Disks for DRL Subdisks    Using a Nonvolatile Write-Back Cache for DRL
8.4.5    LSM Striped Volume Configuration Guidelines    Increasing the Number of Disks in a Striped Volume    Distributing Striped Volume Disks Across Different Buses    Choosing the Correct LSM Stripe Width
8.4.6    LSM RAID 5 Configuration Guidelines    Using RAID 5 Logging    Using the Appropriate Strip Width    Using Hot Sparing for RAID 5 Volumes
8.4.7    Gathering LSM Information    Displaying Configuration Information by Using the volprint Utility    Monitoring Performance Statistics by Using the volstat Utility    Tracking Operations by Using the voltrace Utility    Monitoring Events by Using the volwatch Script    Monitoring Events by Using the volnotify Utility
8.5    Managing Hardware RAID Subsystem Performance
8.5.1    Hardware RAID Features
8.5.2    Hardware RAID Products
8.5.3    Hardware RAID Configuration Guidelines    Distributing Storage Set Disks Across Buses    Using Disks with the Same Data Capacity    Choosing the Correct Hardware RAID Stripe Size    Mirroring Striped Sets    Using a Write-Back Cache    Using Dual-Redundant Controllers    Using Spare Disks to Replace Failed Disks
8.6    Managing CAM Performance
9    Managing File System Performance
9.1    Gathering File System Information
9.1.1    Displaying File System Disk Space
9.1.2    Checking the namei Cache with the dbx Debugger
9.2    Tuning File Systems
9.2.1    Increasing the Size of the namei Cache
9.2.2    Delaying vnode Deallocation
9.2.3    Delaying vnode Recycling
9.2.4    Increasing Memory for the UBC
9.2.5    Increasing the Borrowed Memory Threshold
9.2.6    Increasing the Minimum Size of the UBC
9.2.7    Improving Large File Caching Performance
9.2.8    Disabling File Read Access Time Flushing
9.2.9    Caching Only File System Metadata with Prestoserve
9.3    Managing Advanced File System Performance
9.3.1    AdvFS Features
9.3.2    AdvFS I/O Queues
9.3.3    AdvFS Access Structures
9.3.4    AdvFS Configuration Guidelines    Configuring File Domains    Configuring Filesets for High Performance    Distribute the AdvFS I/O Load    Improving the Transaction Log Performance    Forcing Synchronous Writes    Preventing Partial Data Writes    Enabling Direct I/O    Configuring an AdvFS root File system    Striping Files    Using AdvFS Quotas    Consolidating I/O Transfers
9.3.5    Gathering AdvFS Information    Monitoring AdvFS Performance Statistics by Using the advfsstat Command    Identifying Disks in an AdvFS File Domain by Using the advscan Command    Checking AdvFS File Domains by Using the showfdmn Command    Displaying AdvFS File Information by Using the showfile Command    Displaying the AdvFS Filesets in a File Domain by Using the showfsets Command    Monitoring the Bitmap Metadata Table
9.3.6    Tuning AdvFS    Increasing the Size of the AdvFS Buffer Cache    Increasing the Number of AdvFS Buffer Hash Chains    Increasing the Memory for Access Structures    Increasing Data Cached in the Ready Queue    Increasing the AdvFS Smooth Sync Cache Timeout Value    Specifying the Maximum Number of I/O Requests on the Device Queue    Disabling the Flushing of Modified mmapped Pages
9.3.7    Improving AdvFS Performance    Defragmenting a File Domain    Decreasing the I/O Transfer Size    Moving the Transaction Log    Balancing a Multivolume File Domain    Migrating Files Within a File Domain
9.4    Managing UFS Performance
9.4.1    UFS Configuration Guidelines    Modifying the File System Fragment and Block Sizes    Reducing the Density of inodes    Allocating Blocks Sequentially    Increasing the Number of Blocks Combined for a Cluster    Using MFS    Using UFS Disk Quotas    Increasing the Number of UFS and MFS Mounts
9.4.2    Gathering UFS Information    Displaying UFS Information by Using the dumpfs Command    Monitoring UFS Clustering by Using the dbx Debugger    Checking the Metadata Buffer Cache by Using the dbx Debugger
9.4.3    Tuning UFS    Increasing the Size of the Metadata Buffer Cache    Increasing the Size of the Metadata Hash Chain Table    Increasing the UFS Smooth Sync Cache Timeout Value    Delaying UFS Cluster Flushing    Increasing the Number of Blocks Combined for Read-Ahead    Increasing the Number of Blocks Combined for a Cluster    Defragmenting a File System
9.5    Managing NFS Performance
9.5.1    Gathering NFS Information    Displaying NFS Information by Using the nfsstat Command    Displaying Idle Thread Information by Using the ps Command
9.5.2    Improving NFS Performance    Using Prestoserve to Improve NFS Server Performance    Configuring Server Threads    Configuring Client Threads    Modifying Cache Timeout Limits    Decreasing Network Timeouts    Using NFS Protocol Version 3
10    Managing Network Performance
10.1    Gathering Network Information
10.1.1    Monitoring Network Statistics by Using the netstat Command
10.1.2    Checking Socket Listen Queue Statistics by Using the sysconfig Command
10.2    Tuning the Network Subsystem
10.2.1    Improving the Lookup Rate for TCP Control Blocks
10.2.2    Increasing the Number of TCP Hash Tables
10.2.3    Tuning the TCP Socket Listen Queue Limits
10.2.4    Increasing the Number of Outgoing Connection Ports
10.2.5    Modifying the Range of Outgoing Connection Ports
10.2.6    Disabling Use of a PMTU
10.2.7    Increasing the Number of IP Input Queues
10.2.8    Enabling mbuf Cluster Compression
10.2.9    Enabling TCP Keepalive Functionality
10.2.10    Improving the Lookup Rate for IP Addresses
10.2.11    Decreasing the TCP Partial-Connection Timeout Limit
10.2.12    Decreasing the TCP Connection Context Timeout Limit
10.2.13    Decreasing the TCP Retransmission Rate
10.2.14    Disabling Delaying the Acknowledgment of TCP Data
10.2.15    Increasing the Maximum TCP Segment Size
10.2.16    Increasing the Transmit and Receive Buffers for a TCP Socket
10.2.17    Increasing the Transmit and Receive Buffers for a UDP Socket
10.2.18    Increasing the Size of the ARP Table
10.2.19    Increasing the Maximum Size of a Socket Buffer
10.2.20    Preventing Dropped Input Packets
11    Managing Application Performance
11.1    Gathering Profiling and Debugging Information
11.2    Improving Application Performance
11.2.1    Using the Latest Operating System Patches
11.2.2    Using the Latest Version of the Compiler
11.2.3    Using Parallelism
11.2.4    Optimizing Applications
11.2.5    Using Shared Libraries
11.2.6    Reducing Application Memory Requirements
11.2.7    Controlling Memory Locking
1-1    Moving Instructions and Data Through the Memory Hardware
1-2    Physical Memory Usage
1-3    Configuration with Potential Points of Failure
1-4    Fully Redundant Cluster Configuration
6-1    UBC Memory Allocation
6-2    Memory Allocation During High File System Activity and No Paging Activity
6-3    Memory Allocation During Low File System Activity and High Paging Activity
6-4    Virtual Address Space Usage
6-5    Virtual-to-Physical Address Translation
6-6    Paging and Swapping Attributes
6-7    Paging Operation
9-1    AdvFS I/O Queues
1-1    Memory Management Hardware Resources
1-2    RAID Level Performance and Availability Comparison
1-3    SCSI Bus Speeds
1-4    SCSI Bus and Segment Lengths
2-1    Resource Models and Possible Configuration Solutions
2-2    High-Performance System Hardware Options
2-3    High-Performance Disk Storage Hardware Options
2-4    High-Performance Disk Storage Configuration Solutions
2-5    High-Availability Configurations
3-1    Tools for Continuous Performance Monitoring
3-2    Kernel Profiling and Debugging Tools
4-1    Internet Server Tuning Guidelines
4-2    Large-Memory System Tuning Guidelines
4-3    NFS Server Tuning Guidelines
4-4    Advanced Tuning Guidelines
5-1    Default Values for the maxusers Attribute
5-2    IPC Limits Tuning Guidelines
6-1    Default Values for vm_page_free_target Attribute
6-2    Virtual Memory and UBC Monitoring Tools
6-3    Memory Resource Tuning Guidelines
6-4    Paging and Swapping Tuning Guidelines
7-1    CPU Monitoring Tools
7-2    Primary CPU Performance Improvement Guidelines
8-1    Disk I/O Distribution Monitoring Tools
8-2    LSM Disk, Disk Group, and Volume Configuration Guidelines
8-3    LSM Mirrored Volume Guidelines
8-4    Dirty-Region Logging Guidelines
8-5    LSM Striped Volume Guidelines
8-6    LSM RAID 5 Volume Guidelines
8-7    LSM Monitoring Tools
8-8    Hardware RAID Subsystem Configuration Guidelines
9-1    General File System Tuning Guidelines
9-2    AdvFS Configuration Guidelines
9-3    AdvFS Monitoring Tools
9-4    AdvFS Tuning Guidelines
9-5    AdvFS Performance Improvement Guidelines
9-6    UFS Configuration Guidelines
9-7    UFS Monitoring Tools
9-8    UFS Tuning Guidelines
9-9    NFS Monitoring Tools
9-10    NFS Performance Guidelines
10-1    Network Monitoring Tools
10-2    Network Tuning Guidelines
11-1    Application Profiling and Debugging Tools
11-2    Application Performance Improvement Guidelines