Title and Copyright Information
About This Manual
New and Changed Features
Related Documents
Reader's Comments
Part 1 -- Introduction to System Tuning
1    Introduction to System Tuning
1.1    Hardware Configuration
1.1.1    Hardware Configuration Overview
1.2    Performance Terminology and Concepts
1.3    Disk Storage Resources
1.3.1    RAID Technology
1.3.2    SCSI Concepts    Data Paths    SCSI Bus Speeds    Transmission Methods    Extending UltraSCSI Bus Segments    SCSI Bus Length and Termination
1.3.3    Fibre Channel    Fibre Channel Topologies    Point-to-Point Topology    Fabric Topology    Arbitrated Loop Topology    Fibre Channel Topology Comparison    Zoning    Switch Zoning Versus Selective Storage Presentation    Types of Zoning    Zoning Example    Cascaded Switches
1.4    Network Resources
1.4.1    Network Subsystem
1.4.2    Using Redundant Networks
1.4.3    NetRAIN
1.4.4    Routing
1.4.5    LAG Interface
1.5    File System Resources
1.5.1    Using AdvFS    Using the UBC
1.5.2    Using NFS
1.6    Memory Resources
1.6.1    Paging and Swapping
1.6.2    Caching Data
1.7    CPU Resources
1.8    Identifying a Resource Model for Your Workload
1.9    Most Commonly Tuned Subsystems
2    Gathering System and Performance Information
2.1    Methodology Approach to Solving Performance Problems
2.2    Obtaining Information About System Events
2.2.1    Using Event Manager
2.2.2    Using DECevent
2.2.3    Using Compaq Analyze
2.2.4    Using System Accounting and Disk Quotas
2.3    Primary Tools for Gathering Information
2.3.1    Gathering Hardware Information Using the hwmgr Utility
2.3.2    Gathering System Information by Using the collect Utility    Configuring collect to Automatically Start on System Reboot    Plotting collect Datafiles
2.3.3    Checking the Configuration by Using the sys_check Utility
2.4    Secondary Tools for Gathering Information
2.4.1    Gathering Locking Statistics by Using the lockinfo Utility
2.4.2    Gathering CPU Usage and Process Statistics by Using the sched_stat Utility
2.4.3    Displaying Network and NFS Statistics by Using the nfsstat Utility
2.4.4    Gathering Information by Using the tcpdump Utility
2.4.5    Monitoring Network Statistics by Using the netstat Command    Input and Output Errors and Collisions    Device Driver Errors    Memory Usage    Socket Connections    Dropped or Lost Packets    Retransmissions, Out-of-Order Packets, and Bad Checksums    Routing Statistics    Protocol Statistics
2.4.6    Gathering NFS Server Side Information Using ps axlmp
2.4.7    Gathering NFS Client Side Information Using nfsiod
2.4.8    Monitoring Incoming Network Traffic to an NFS Server by Using the nfswatch Command
2.5    Additional Tools for Monitoring Performance
2.6    Gathering Profiling and Debugging Information
3    Displaying and Modifying Kernel Subsystem Attributes
3.1    Operating System Support for Attributes
3.2    Displaying Attribute Values
3.3    Modifying Attribute Values
3.3.1    Current Value
3.3.2    Permanent Value
Part 2 -- Tuning by Application
4    Tuning Oracle
4.1    Monitoring Oracle Statistics
4.2    Improving the Performance of the gettimeofday( ) Function
4.3    Choosing and Enabling IPC Communication Protocols
4.4    Tuning Recommendations
4.4.1    Modifying Virtual Memory Attributes    Disabling Shared Memory    Allocating Shared Memory    Modifying the rad_gh_regions Attribute    Modifying the gh_chunks Attribute    Modifying the Percentage of Physical Memory the UBC is Using    Modifying the Percentage of Memory the UBC is Borrowing    Modifying the Percentage of Memory the UBC Can Use For a Single File    Modifying the UBC Threshold    Modifying the Percentage of Pages that Must be Dirty    Modifying the Swap Allocation Mode
4.4.2    Modifying the Advanced File System Attribute
4.4.3    Modifying the Virtual File System Attribute
4.4.4    Modifying Interprocess Communication Attributes    Modifying the System V Shared Regions    Modifying the System V Maximum Size of Shared Memory Region    Modifying the System V Minimum Size of Shared Memory Region    Modifying the Shared Memory Regions that Can be Used at One Time    Modifying the Shared Memory Regions that Can be Attached at One Time
4.4.5    Modifying Internet Attributes    Modifying the Send Buffer Size for the UDP Sockets    Modifying the Receive Buffer Size for the UDP Sockets    Modifying the Number of Times a System can make Outgoing Connections
4.4.6    Modifying Process Attributes    Modifying the Per Process Stack Size    Modifying the Maximum Size of the User Process Stack Size    Modifying the Per Process Data Size    Modifying the Maximum Size of the Per Process Data Size    Modifying the Per Process Address Size    Modifying the Maximum Per Process Address Size    Modifying the Maximum Number of Processes    Modifying the Maximum Number of Threads    Modifying the Space Allocated to System Tables
4.4.7    Modifying the Real-Time Attribute
4.4.8    Modifying Reliable Datagram Attributes    Modifying the Maximum Number of Objects in the RDG    Modifying the Maximum Size of the RDG Message    Modifying the Maximum Number of Messages in the RDG    Modifying the Maximum Number of Sessions within the RDG Table    Modifying the Maximum Number of Pages Wired For Message Packets
4.4.9    Modifying the Memory Channel Attribute
5    Tuning Network File Systems
5.1    Monitoring NFS Statistics
5.2    Detecting Poor NFS Performance
5.3    Performance Benefits and Tradeoffs
5.4    NFS Configuration
5.4.1    Configuring Server Threads
5.4.2    Configuring Client Threads
5.4.3    Modifying Cache Timeout Limits
5.5    NFS Retransmissions
5.5.1    Decreasing Network Timeouts
5.6    Tuning NFS Servers
5.6.1    Modifying NFS Server Side Attributes    Write Gathering    Improving NFS Server Response Time to Client Write Requests    Specifying the Amount of Time in Seconds the Server will Delay the Write    Increasing the NFS Send and Receive Buffer Size
5.7    Tuning NFS Clients
5.7.1    Modifying NFS Client Side Attributes    Improving Read Performance    Controlling How Long Before the Client will Start Transmitting    Directory Name Lookup Cache (DNLC)    Negative Name Cache Lookups (NNC)    Specifying File Consistency Across NFS Clients    Changing the NFS Client Behavior When Fetching File Attributes
6    Tuning Internet Servers
6.1    Improving Internet Server Performance
6.1.1    Configuring Hardware
6.1.2    Configuring Memory and Swap Space
6.1.3    Logging IP Addresses
6.1.4    Monitoring Network Statistics
6.1.5    Monitoring Socket Statistics
6.1.6    Monitoring Virtual Memory Statistics
6.1.7    Gathering Configuration Information
6.2    Primary Tuning Recommendations
6.2.1    Modifying Internet Attributes    Increasing the Size of the TCP Hash Table    Disabling PMTU Discovery    Increasing the Number of Outgoing Connection Ports
6.2.2    Modifying Process Attributes    Increasing the Size of System Tables and Data Structures    Increasing the Number of Processes per User    Increasing the Number of Threads per User    Increasing the User Process Data Segment Size Limits    Increasing the User Process Address Space Limits
6.2.3    Modifying Socket Attributes    Increasing the Maximum Number of Pending TCP Connections    Increasing the Minimum Number of Pending TCP Connections    Enabling the mbuf Cluster Compression
6.3    Advanced Tuning Recommendations
6.3.1    Modifying Generic Attributes
6.3.2    Modifying Internet Attributes    Increasing the Number of TCP Hash Table    Increasing the Number of Hash Buckets    Modifying the TCP Partial Connection Timeout Limit    Decreasing the Rate of TCP Retransmissions    Enabling TCP Keepalive Functionality    Increasing the TCP Connection Context Timeout Rate    Modifying the Range for Outgoing Connection Ports    Increasing the Number of IP Input Queues    Increasing the Maximum Length of the IP Input Queue
6.3.3    Modifying Network Attributes    Increasing the Number of Output Packets Before Packets are Dropped    Reducing Screening Cache Misses    Reducing the Screening Buffer Drops
6.3.4    Modifying Socket Attributes
6.3.5    Modifying Virtual Memory Attributes
7    Managing Application Performance
7.1    Improving Application Performance
7.1.1    Using the Latest Operating System Patches
7.1.2    Using the Latest Version of the Compiler
7.1.3    Using Parallelism
7.1.4    Optimizing Applications
7.1.5    Using Shared Libraries
7.1.6    Reducing Application Memory Requirements
7.1.7    Controlling Memory Locking
Part 3 -- Tuning by Component
8    Managing System Resource Allocation
8.1    Tuning Process Limits
8.1.1    Increasing System Tables and Data Structures
8.1.2    Increasing the Maximum Number of Processes
8.1.3    Increasing the Maximum Number of Threads
8.2    Tuning Program Size Limits
8.2.1    Increasing the Size of a User Process Stack
8.2.2    Increasing the Size of a User Process Data Segment
8.3    Tuning Address Space Limits
8.4    Tuning Interprocess Communication Limits
8.4.1    Increasing the Maximum Size of a System V Message
8.4.2    Increasing the Maximum Size of a System V Message Queue
8.4.3    Increasing the Maximum Number of Messages on a System V Queue
8.4.4    Increasing the Maximum Size of a System V Shared Memory Region
8.4.5    Increasing the Maximum Number of Shared Memory Regions Attached to a Process
8.4.6    Modifying Shared Page Table Sharing
8.5    Tuning the Open File Limits
8.5.1    Increasing the Maximum Number of Open Files
8.5.2    Increasing the Maximum Number of Open File Descriptors
8.6    Aurema ARMTech Suite
9    Managing Disk Storage Performance
9.1    Guidelines for Distributing the Disk I/O Load
9.2    Monitoring the Distribution of Disk I/O
9.2.1    Displaying Disk Usage by Using the iostat Command
9.3    Managing Storage with LSM
9.3.1    LSM Features
9.4    Managing Hardware RAID Subsystem Performance
9.4.1    Hardware RAID Features
9.4.2    Hardware RAID Products
9.4.3    Hardware RAID Configuration Guidelines    Distributing Storage Set Disks Across Buses    Using Disks with the Same Data Capacity    Choosing the Correct Hardware RAID Stripe Size    Mirroring Striped Sets    Using a Write-Back Cache    Using Dual-Redundant Controllers    Using Spare Disks to Replace Failed Disks
9.5    Managing CAM Performance
10    Managing Network Performance
10.1    Gathering Network Information
10.1.1    Checking Socket Listen Queue Statistics by Using the sysconfig Command
10.2    Tuning the Network Subsystem
10.2.1    Improving the Lookup Rate for TCP Control Blocks
10.2.2    Increasing the Number of TCP Hash Tables
10.2.3    Tuning the TCP Socket Listen Queue Limits
10.2.4    Increasing the Number of Outgoing Connection Ports
10.2.5    Modifying the Range of Outgoing Connection Ports
10.2.6    Disabling PMTU Discovery
10.2.7    Increasing the Number of IP Input Queues
10.2.8    Enabling mbuf Cluster Compression
10.2.9    Enabling TCP Keepalive Functionality
10.2.10    Improving the Lookup Rate for IP Addresses
10.2.11    Decreasing the TCP Partial-Connection Timeout Limit
10.2.12    Decreasing the TCP Connection Context Timeout Limit
10.2.13    Decreasing the TCP Retransmission Rate
10.2.14    Disabling Delaying the Acknowledgment of TCP Data
10.2.15    Increasing the Maximum TCP Segment Size
10.2.16    Increasing the Transmit and Receive Buffers for a UDP Socket
10.2.17    Increasing the Maximum Size of a Socket Buffer
10.2.18    Preventing Dropped Input Packets
11    Managing File System Performance
11.1    Tuning Caches
11.1.1    Monitoring Cache Statistics
11.1.2    Tuning the namei Cache
11.1.3    Tuning the UBC
11.1.4    Tuning the Metadata Buffer Cache
11.1.5    Tuning AdvFS Access Structures
11.2    Tuning AdvFS
11.2.1    AdvFS Configuration Guidelines    Storing Data Using RAID1 or RAID5    Forcing a Synchronous Write Request or Enabling Persistent Atomic Write Data Logging    Enabling Direct I/O    Using AdvFS to Distribute Files    Striping Data    Defragmenting a File Domain    Decreasing the I/O Transfer Size    Moving the Transaction Log
11.2.2    Monitoring AdvFS Statistics    Displaying AdvFS Performance Statistics    Displaying Disks in an AdvFS File Domain    Displaying AdvFS File Domains    Displaying AdvFS File Information    Displaying the AdvFS Filesets in a File Domain
11.2.3    Tuning AdvFS Queues
11.3    Tuning UFS
11.3.1    UFS Configuration Guidelines    Modifying the File System Fragment and Block Sizes    Reducing the Density of inodes    Set Rotational Delay    Increasing the Number of Blocks Combined for a Cluster    Using MFS    Using UFS Disk Quotas    Increasing the Number of UFS and MFS Mounts
11.3.2    Monitoring UFS Statistics    Displaying UFS Information    Monitoring UFS Clustering    Displaying the Metadata Buffer Cache
11.3.3    Tuning UFS for Performance    Adjusting UFS Smooth Sync and I/O Throttling    Delaying UFS Cluster Writing    Increasing the Number of Blocks in a Cluster    Defragmenting a File System
11.4    Tuning NFS
12    Managing Memory Performance
12.1    Virtual Memory Operation
12.1.1    Physical Page Tracking
12.1.2    File-System Buffer Cache Memory Allocation    Metadata Buffer Cache Memory Allocation    Unified Buffer Cache Memory Allocation
12.1.3    Process Memory Allocation    Process Virtual Address Space Allocation    Virtual Address to Physical Address Translation    Page Faults
12.1.4    Page Reclamation    Modified Page Prewriting    Reclaiming Memory by Paging    Reclaiming Memory by Swapping
12.2    Configuring Swap Space for High Performance
12.3    Monitoring Memory Statistics
12.3.1    Displaying Memory by Using the vmstat Command
12.3.2    Displaying Memory by Using the ps Command
12.3.3    Displaying Swap Space Usage by Using the swapon Command
12.3.4    Displaying the UBC by Using the dbx Debugger
12.4    Tuning to Provide More Memory to Processes
12.4.1    Reducing the Number of Processes Running Simultaneously
12.4.2    Reducing the Static Size of the Kernel
12.4.3    Increasing the Memory Reserved for Kernel malloc Allocations
12.5    Modifying Paging and Swapping Operations
12.5.1    Increasing the Paging Threshold
12.5.2    Managing the Rate of Swapping
12.5.3    Enabling Aggressive Task Swapping
12.5.4    Limiting the Resident Set Size to Avoid Swapping
12.5.5    Managing Modified Page Prewriting
12.5.6    Managing Page-In and Page-Out Clusters Sizes
12.5.7    Managing I/O Requests on the Swap Partition
12.6    Reserving Physical Memory for Shared Memory
12.6.1    Tuning the Kernel to Use Granularity Hints
12.6.2    Modifying Applications to Use Granularity Hints
12.7    Improving Performance with Big Pages
12.7.1    Using Big Pages
12.7.2    Determining when a Memory Object uses Big Pages
13    Managing CPU Performance
13.1    Monitoring CPU Performance Information
13.1.1    Monitoring the Load Average by Using the uptime Command
13.1.2    Checking CPU Usage by Using the kdbx Debugger cpustat Extension
13.1.3    Checking Lock Usage by Using the kdbx Debugger lockstat Extension
13.2    Improving CPU Performance
13.2.1    Adding Processors
13.2.2    Using the Class Scheduler    Class Scheduler Overview    Related Utilities    Invoking the Class Scheduler    Planning Class Scheduling    Configuring Class Scheduling    Creating and Managing Classes    Creating a Class    Managing Identifier Types Within Classes    Enabling the Class Scheduler    Adding Members to a Class    Deleting Members From a Class    Other Class Management Options    Using the runclass Command    Using the Class Scheduling Graphical Interface    Creating or Modifying a Database
13.2.3    Prioritizing Jobs
13.2.4    Scheduling Jobs at Offpeak Hours
13.2.5    Stopping the advfsd Daemon
13.2.6    Using Hardware RAID to Relieve the CPU of I/O Overhead
1-1    Mapping Out Your Hardware Configuration
1-2    Point-to-Point Topology
1-3    Fabric Topology
1-4    Arbitrated Loop Topology
1-5    A Simple Zoned Configuration
1-6    Meshed Resilient Fabric with Four Cascaded Switches
1-7    Single Interface Configuration
1-8    Mulitple Interfaces
1-9    Physical Memory Usage
1-10    Moving Instructions and Data Through the Memory Hardware
11-1    Striping Data
11-2    AdvFS I/O Queues
12-1    UBC Memory Allocation
12-2    Memory Allocation During High File-System Activity and No Paging Activity
12-3    Memory Allocation During Low File-System Activity and High Paging Activity
12-4    Process Virtual Address Space Usage
12-5    Virtual-to-Physical Address Translation
12-6    Paging and Swapping Attributes
12-7    Paging Operation
1-1    RAID Level Performance and Availability Comparison
1-2    SCSI Bus Speeds
1-3    SCSI Bus and Segment Lengths
1-4    Fibre Channel Fabric and Arbitrated Loop Comparison
1-5    Type of Zoning Supported by Switches
1-6    Memory Management Hardware Resources
1-7    Resource Models and Possible Configuration Solutions
2-1    Tools for Continuous Performance Monitoring
2-2    Application Profiling and Debugging Tools
4-1    Tools to Detect Poor Oracle Application Performance
5-1    Tools to Detect Poor NFS Performance
5-2    Potential NFS Problems and Solutions
5-3    NFS Tuning Guidelines
5-4    NFS Server Tuning Guidelines
5-5    Identifying Your Network Card Type
6-1    Tools for Monitoring Network Statistics
7-1    Application Performance Improvement Guidelines
8-1    Default Values for the maxusers Attribute
8-2    IPC Limits Tuning Guidelines
9-1    Disk I/O Distribution Monitoring Tools
9-2    Hardware RAID Subsystem Configuration Guidelines
10-1    Network Monitoring Tools
10-2    Network Tuning Guidelines
11-1    Tools to Display Cache Information
11-2    When to Change the Values of the Namei Cache Related Attributes
11-3    When to Change the Values of the UBC-Related Attributes
11-4    AdvFS Configuration Guidelines
11-5    Tools to Display AdvFS Information
11-6    UFS Configuration Guidelines
11-7    Tools to Display UFS Information
11-8    UFS Tuning Guidelines
12-1    Default Values for vm_page_free_target Attribute
12-2    Tools to Display Virtual Memory and UBC
12-3    Memory Resource Tuning Guidelines
13-1    CPU Monitoring Tools
13-2    Primary CPU Performance Improvement Guidelines