This is taken from IBM Systems Magazine - AIX

ASO Can Quickly Boost 

Performance for Free

ASO can quickly boost deliver performance for free

In 2011, IBM announced the Active System Optimizer (ASO) enhancement, available only on recent AIX levels for POWER7. The intent was to automatically apply best-practices performance tweaks to individual LPARs. Some of those tweaks include improvements to memory affinity—specifically associating targeted workloads to a specific core or set of cores and determining if memory can be relocated for higher affinity to cache and cores. An in-LPAR solution, ASO runs in the background of each LPAR it’s tuning. Keep in mind it doesn’t support active memory sharing or LPAR migration.

Getting Started

ASO is installed by default in a standard installation as part of the bos.aso fileset. Running lslpp –l | grep bos.aso shows the installed level set. The three ASO prerequisites are:

1. AIX V6 TL08 or AIX V7
2. The bos.aso fileset must be installed
3. A POWER7/7+ server

To check the prerequisites:

# oslevel -s

7100-01-05-1228

This output shows the system is at: AIX V7 TL01 SP5.

# lslpp -l | grep aso
  bos.aso                   7.1.1.15  COMMITTED  Active System Optimizer
  bos.aso                   7.1.1.15  COMMITTED  Active System Optimizer
# lsconf | grep ^Processor
Processor Type: PowerPC_POWER7
Processor Implementation Mode: POWER 7
Processor Version: PV_7_Compat
Processor Clock Speed: 3000 MHz

This indicates the system can run ASO without problems.

To use ASO, first start the subsystem, using startsrc –s aso. You can then monitor and control it using the asoo command. The following commands can be very useful:

# lssrc -s aso
Subsystem         Group            PID          Status 
 aso                              5373980      active

The ASO subsystem is started and is running as PID 5373980.

# ps -ef | grep 5373980
    root  5373980  3604670   0   Jan 02      -  1:14 /usr/sbin/aso

The program being run is /usr/sbin/aso.

asoo –L   List the current and reboot values
# asoo -L
NAME                  CUR    DEF    BOOT   MIN    MAX     UNIT       TYPE 
aso_active             0      0      0      0      1      boolean      D

#asoo -op aso_active=1 Permanently turn ASO on (0 is disabled and is the default)

To check what’s happening in the ASO logs; their location is specified in /etc/syslog.conf and defaults to:

aso.notice /var/log/aso/aso.log      rotate size 128k time 7d
aso.info    /var/log/aso/aso_process.log rotate size 1024k

The aso.log records the on/off status and the reasons ASO goes into hibernation. Similarly, aso_process.log records the same information as well as details of any actions ASO takes and the processes it modifies.

My test LPAR is on a Power 720 and has an entitlement of 0.5, VPs set to 2 and 4GB of memory. The lssrad command shows I only have one socket, so there’s little to tune with respect to affinity.

# lssrad -av
REF1   SRAD        MEM      CPU
0
          0    3699.81      0-11

Although the ASO subsystem is running, we still need to enable ASO as follows:

# asoo -o aso_active=1
Setting aso_active to 1

The log now shows:

Jun 16 17:32:46 testlpar aso:notice aso[5373980]: ASO enabled by tunable
Jun 16 17:32:46 testlpar aso:notice aso[5373980]: [HIB] Current number of system
 virtual cpus too low (2 cpus)
Jun 16 17:32:46 testlpar aso:notice aso[5373980]: [HIB] Increase system virtual cpus to
 at least 3 cpus to run ASO. Hibernating.

When planning where and when to run ASO, keep in mind it needs at least three virtual processors (VPs) to run. At this point, ASO was still enabled, so I disabled it again and used Dynamic LPAR to add another VP to the LPAR.
# lsdev -C | grep proc
proc0      Available 00-00       Processor
proc4      Available 00-04       Processor
proc8      Available 00-08       Processor
# asoo -o aso_active=1
Setting aso_active to 1


The log now shows:
Jun 16 17:39:08 testlpar aso:notice aso[5373980]: ASO enabled by tunable


Unfortunately, the test system wasn’t very busy so ASO decided to hibernate:
LOG
Jun 16 17:40:53 testlpar aso:notice aso[5373980]: [HIB] Used entitlement per unfolded vCPU
 is below threshold (1% of a core).
Jun 16 17:40:53 testlpar aso:notice aso[5373980]: [HIB] ASO will hibernate until used
 entitlement is at least 30% of a core per unfolded vCPU


Internal Analysis

On a busy system, ASO takes some time to profile and analyze the running workloads to dynamically tune the system for those workloads. Once it’s running, it needs no further interaction. ASO will examine AIX kernel data about processes and threads, and the POWER7 hardware performance counters. Based on that data, ASO makes tuning decisions. It will try to improve cache and memory affinity by moving workloads to cores that help them improve their affinity needs. It takes into account Workload Manager (WLM) and workload partition (WPAR) resource sets as well as constraints. It does this by looking at the workload over minutes, and setting scheduler resource allocation domain (SRAD) and resource set (RSET) rules in order to change the CPU a workload is running on. All changes are logged and tracked with ASO logging both the expected and actual gain seen.

By default, ASO enables cache and memory affinity optimization, large page optimization, and memory prefetch. These restricted tunables should only be modified at the request of IBM Support.

To improve the cache and memory performance of workloads, ASO uses one of three types of optimization: cache affinity, aggressive cache affinity or memory affinity. For cache affinity, it tries to reduce chip-to-chip cache movement. For a workload assigned cores across two sockets, ASO tries to place the workload so it fits within a socket using an RSET. For aggressive cache affinity, it tries to reduce the number of chips used and may compress the workload onto fewer cores to do so. For memory affinity, ASO tries to ensure that local memory is used. This often involves migrating pages for workloads to DIMMs closer to the core being used, thus reducing remote memory traffic. It uses CPU and memory RSETs to do this.

Debugging

In the unlikely event you experience problems with ASO, the perfpmr.sh scripts don’t collect the ASO logs. So, you must set ASO into debug mode:
asoo –o debug_level=3


You’ll then need to recreate the problem and reset ASO to normal logging:
asoo –o debug_level=0


Finally, forward the aso_debug.out file to IBM Support.

A Fine Tuning Tool

Free with AIX and easy to use, ASO offers great benefits for tuning. It monitors workloads on a busy system and dynamically moves them to gain the best affinity within an LPAR. With it, administrators can avoid making WLM and RSET changes to attain affinity.

It can be turned off immediately if performance results are less than expected. It’s particularly useful for multi-threaded, long-running processes on larger systems, especially the Power 770 and above, which have multiple nodes. However, any of the multi-socket systems could benefit. Try ASO on test LPARs to gauge its potential benefits for performance.