Blog
i can

July 08, 2014

IBM i 7.2 Improved Temporary Storage Tracking (Part 3)

I’m continuing the series of blogs on improved temporary storage tracking in IBM i 7.2 –part 1 covered the changes made to provide improved tracking and part 2 covered the changes made so you can better understand temporary consumption at a system level. You may want to read these blogs if you have not done so already.

This week, I want to review the enhancements that allow you to better view and understand where the temporary storage is being used from a job-specific perspective. If you have discovered a problem with temporary storage consumption on your IBM i partition, you generally detect the issue at the system level but will need to identify the root cause. The problem is often caused by a job (or jobs) that is consuming the storage.

 

Active Jobs

Using Navigator for i to view Active Jobs (or active jobs by memory pool, or active jobs by subsystem), the temporary storage used column will be displayed by default. Simply click on the column heading (twice) to sort the table by the jobs that are using the most temporary storage. (Yes, I’ve already asked the Navigator development team to add a way to select the sort order with just one click….)

Also, a reminder about using the Active Jobs interface on Navigator – if you want to see updated values displayed in the table, don’t forget to click the “refresh” button – the table is not dynamically updated. (Yes, I’ve already asked the Navigator development team to add dynamic updates….)

The screen capture below shows the Active Jobs display sorted by the temporary storage used column.

Dawn1

 

Work with Active Jobs (WRKACTJOB)

For those of you who prefer to use the green-screen interface to manage i, you will appreciate the enhancement to WRKACTJOB to provide temporary storage as a column. You can simply use WRKACTJOB and press F11 two times to display the new column. You can sort on the column in the usual way with your cursor placed on the column heading and press F16. In addition, if you prompt the WRKACTJOB command, there is a new value, *TMPSTG, on the sequence (SEQ) parameter that you can use to immediately display the view with the jobs sorted by the temporary storage column.

The screen capture below shows the WRKACTJOB display sorted by the temporary storage column.

Dawn2

 

Work with Job

There is also a minor change to a job’s run attributes. Prior to 7.2, you could work with a job and see the maximum amount of temporary storage allowed and the current amount used. 7.2 enhances this by including peak usage by that particular job.

Dawn3

Interestingly, this support is only on the green screen WRKJOB (option 3) and not on the job properties GUI. (Yes, I’ve already asked the Navigator development team to add the peak temporary storage used to the GUI….)

As an aside, most of you should already know that you configure the maximum temporary storage allowed by a job on the class object. In 7.2, the MAXTMPSTG parameter on the class object is now in megabytes rather than kilobytes. Read the blog Jobs Exceeding their CPU or Storage Limits are now Held on why you should start setting the maximum temporary storage value.

 

Temporary Storage Details – buckets per job

In part 2, I wrote about how you can view the temporary storage details to find a table of all of the temporary storage buckets along with their size information. That article noted that there is a bucket for each active job; the temporary storage details view is another way to find the jobs that are using the temporary storage in addition to Navigator’s Active Jobs task.

Remember you can use filters to subset what you see, so you could add a filter on the “Job Status” column to only show those entries that start with “*” – this is a simple way to filter out the global entries and only see the temporary storage information for jobs. You can then sort the bucket current size column to find the job(s) using the most temporary storage. The nice thing about using this approach is that you also can find jobs that have ended but still have temporary storage allocated.

If you discover active jobs that are consuming an unexpectedly large amount of temporary storage, you can take proactive actions by holding or ending them. While you can see if ended jobs had temporary storage that was not released, you cannot take any actions upon those jobs. Diagnostics or debugging will be necessary to determine what the job was doing when it ended that prevented the system from freeing storage.

 

It looks like I will have at least two more blogs on related topics for temporary storage tacking in the 7.2 release. In future blogs, I will discuss support added to Collection Services and how to set up notification.

 

 

June 25, 2014

PowerVM Processor Virtualization 101 Part 2

Chris Francois completes blog part 2 on PowerVM Processor Virtualization 101. Read PowerVM Processor Virtualization 101 part 1 if you missed it. 

This is the second installment in a two-part blog series to provide basic introduction to the PowerVM processor virtualization terminology and acronyms. The first part was heavy on terminology. This part will tie the terminology together and provide you with a better understanding of the trade-offs involved with PowerVM LPAR configuration as it pertains to processor virtualization. The IBM Redbooks publication, “IBM PowerVM Virtualization Introduction and Configuration” provides much greater detail. For an in-depth treatment of this material from an IBM i perspective, see “Under the Hood: POWER7 Logical Partitions.”

PowerVM implements serial sharing of physical processors, with entitled capacity commitments enforced through periodic intervals of time called “dispatch windows.” The sharing is serial because the physical processor is dispatched exclusively to one partition at a time, regardless of the processor's thread context (e.g., SMT4). Entitled capacity represents a claim on a fraction of physical processor dispatch time in the dispatch window. For example, assuming the dispatch window period is 10 milliseconds, 2.0 processor units entitled capacity is a claim on 20 milliseconds of physical processor dispatch time. These entitled capacity claims are “use it or lose it”; every dispatch window the LPAR's entitled capacity commitment is replenished without regard to history. For POWER8, PowerVM requires a minimum of 0.05 processor unit and a maximum of 1.0 processor unit per virtual processor. The total current entitled capacity of all shared processor LPARs cannot exceed the number of processors assigned to the physical shared processor pool, and the current number of dedicated processors cannot exceed the balance of licensed processors in the platform. This is a roundabout way of saying that while there can be more virtual processors than physical processors (up to 20 times), the entitled capacity can never be overcommitted.

The differences between a shared and dedicated processor LPAR goes beyond the fact that a dedicated processor LPAR has a fixed ratio (i.e., 1:1) of entitled capacity units to virtual processors. A shared processor LPAR can be configured for uncapped sharing mode, meaning that it is able to use excess shared processor pool capacity above and beyond its entitled capacity. For an uncapped LPAR, the uncapped weight offers some control over the relative distribution of excess shared processor pool capacity among competing uncapped LPARs. A dedicated processor LPAR can be configured for processor sharing, meaning that the operating system can chose to allow the LPAR's idle virtual processor[s] to be temporarily donated to the physical shared processor pool. Oftentimes, this is an effective way to increase the excess shared pool capacity available to uncapped LPARs, and normally the performance impact on the donating LPAR is negligible, as the physical processor is returned to the donating LPAR upon demand.

The other major implementation differences mainly impact performance:

  • • Resource Isolation – While POWER systems and PowerVM provide secure data isolation between LPARs, the physical nature of serially sharing physical processors can impact the effectiveness of processor caches and other hardware resources. For a dedicated processor LPAR, the operating system has greater control over processor sharing and associated performance impacts.
  • • Processor Affinity – The association between a virtual processor and its underlying physical processor are not set in stone, but for a dedicated processor LPAR, the associations are much more durable than for a shared processor LPAR. The VCPU of a shared processor LPAR may be dispatched to any physical processor of the shared processor pool, whereas the VCPU of a dedicated processor LPAR is generally dispatched to the same physical processor. Architecturally, virtual to physical processor associations can change at any time, but for a dedicated processor LPAR, they tend to remain constant during partition activation. Exceptions are processor DLPAR, Live Partition Mobility, and Dynamic Platform Optimizer operations. Software optimizations based on processor affinity are generally more effective for dedicated processor LPARs than for shared processor LPARs.
  • • VCPU Latency – Shared processor LPARs can incur entitlement delays, which are the result of entitled capacity being exhausted during the dispatch window, and have a greater potential for VCPU dispatch delays, which are the result of over subscription of the physical shared processor pool at any moment. Dedicated processor LPARs don't experience entitlement delays, and VCPU dispatch delays are generally negligible.
  • • I/O Latency – Interrupts from I/O adapters assigned to shared processor LPARs are routed to any physical processor of the shared processor pool. Sometimes the interrupt can be handled directly, but sometimes it must be forwarded to a VCPU of its assigned LPAR. This forwarding can be a source of latency which does not occur for the interrupts of I/O adapters assigned to dedicated processor LPARs.

There you have it...PowerVM processor virtualization in a nutshell. PowerVM's flexible, industrial strength processor virtualization supports a range of options and features to maximize the utility of the Power Systems platform. For more in-depth coverage, the website, “Server virtualization with PowerVM” is a comprehensive source for this and other PowerVM topics.

References

IBM i 7.2 and POWER8

PowerVM Processor Virtualization 101

Under the Hood: POWER7 Logical Partitions

IBM PowerVM Virtualization Introduction and Configuration

Live Partition Mobility 

Dynamic Platform Optimizer – Affinity and Beyond for IBM Power

 

 

June 19, 2014

PowerVM Processor Virtualization 101

I’d like to thank Chris Francois for writing this blog. Chris is one of the leading experts on IBM i running on the POWER processor. Chris has been a commercial OS kernel programmer for nearly 25 years, and is currently a lead developer of IBM i Licensed Internal Code. Chris joined IBM Rochester in 1994 during the AS/400’s migration to the processor family ultimately to be known as POWER.

In addition to authoring this blog, Chris is also the author of a recent article on IBM i developerWorks, IBM i 7.2 and POWER8.

 

This is the first installment in a two-part blog series to provide basic introduction to the PowerVM processor virtualization terminology and acronyms. The first part will be heavy on terms without much explanation; the second part will be just the opposite. The IBM Redbooks publication, “IBM PowerVM Virtualization Introduction and Configuration” provides much greater detail. For an in-depth treatment of this material from an IBM i perspective, see “Under the Hood: POWER7 Logical Partitions.” Let’s get started...

The IBM i operating system executes in a system virtual machine (VM) implemented by the PowerVM hypervisor on a Power Systems server. PowerVM supports 1,000 concurrent instances of VMs, which are called logical partitions (LPARs). The hardware resources available to the LPAR are specified in the LPAR configuration, and include the number of virtual processor cores (VCPUs), MBs of mainstore, etc., as well as resource virtualization attributes and qualifiers. For instance, the LPAR may use shared processors, in which case the processing units defines the fractional share of physical processor dispatch time to which the partition is entitled, or the partition may use dedicated processors, in which case the processing units is implicitly the full share of physical processor dispatch time corresponding to the number of VCPUs configured. The term entitled capacity (EC) is sometimes used interchangeably with processing units, especially to avoid confusion with [virtual] processors.

There are a number of LPAR configuration parameters associated with processors. Some of them are fixed for the duration of the partition activation (i.e., partition IPL), and some may be changed while the partition is active. Static LPAR parameters include the minimum and maximum number of VCPUs, the processing mode (Shared or Dedicated), the processor sharing mode, and the Processor Compatibility Mode (PCM). A shared processor LPAR in uncapped sharing mode is often simply referred to as an uncapped partition, as its virtual processor dispatch time is not limited, or capped, by its entitled processor units. Dynamic LPAR parameters include the current number of VCPUs, the current processor units for a shared processor LPAR (SPLPAR), the weight for an uncapped partition, and the processor-sharing attribute for a dedicated processor LPAR. The term donation attribute is often used in place of processor sharing attribute to underscore that sharing is the result of a dedicated processor LPAR temporarily donating unused processors to the physical shared processor pool.

Many of the parameters for a dedicated processor LPAR are shown in Figure 1 below, which is the Processors properties tab of a partition profile as rendered by the HMC. Figure 2 illustrates the parameters for a shared processor LPAR.

06182014DawnMay1

Figure 1 – Dedicated Processor Sharing Mode Attributes

06182014DawnMay2

Figure 2 – Shared Processor Sharing Mode Attributes

Of all of these parameters, Processor Compatibility Mode is probably least familiar to IBM i audiences. The short explanation is that PCM makes the virtualized processor appear as an instance of the indicated generation of POWER processor. So for example, POWER8 mode supports SMT8 and implements Version 2.07 of the Power Instruction Set Architecture (ISA), POWER7 mode supports SMT4 and implements Power ISA 2.06, POWER6 mode supports SMT2 implements Power ISA 2.03, and so forth. On a POWER8 server, PCM can be configured for POWER8, POWER7, POWER6 and POWER6+.

One of the benefits of the Technology Independent Machine Interface (TIMI) is that IBM i applications are generally insulated from the Power ISA version. As a result, PCM has not been relevant for most IBM i users. Now that IBM i supports Live Partition Mobility between POWER7 and POWER8 Power Systems servers, PCM will become more important, because it must be set to a mode supported on both the source and target systems.

In the next part, we’ll look at how these partition configuration parameters interrelate, and the flexibility that PowerVM processor virtualization can offer in support of scale-up and consolidation roles, on the same platform, at the same time.

 

References

Under the Hood: POWER7 Logical Partitions

IBM PowerVM Virtualization Introduction and Configuration

Live Partition Mobility

IBM i 7.2 and POWER8

June 10, 2014

7.2 Improved Temporary Storage Tracking (Part 2)

Last week, I wrote about the underlying changes made in the 7.2 release of the IBM i operating system for improved temporary storage tracking.

This week, I want to review the system status enhancements that allow you to better view and understand where the temporary storage is being used from a systemwide perspective.

Work with System Status (WRKSYSSTS)

Prior to the 7.2 release, WRKSYSSTS showed temporary storage as “Current unprotect used” and “Maximum unprotect” values. These have simply been reworded to be “Current temporary used” and “Peak temporary used,” as the following screen capture shows:

Fig1

System Status (the Navigator Way)

Using Navigator for i to view System Status, on the Disk Space tab, you will find the current and maximum temporary storage used values; these are not new and have been on the GUI System Status for some time.

What is new with the 7.2 release is the “Temporary Storage Details” button on this view, as you can see on the screen capture below.

Fig2

When you click the “Temporary Storage Details” button, it will bring up a table of information regarding the temporary storage buckets that I wrote about last week. The example below shows the table of the temporary storage buckets that is displayed and the columns of information: the current bucket size, the bucket peak size, and the bucket limit size. (I’ll write about setting bucket limits in a future blog.) In addition, for any temporary storage bucket that is associated with a job, you will see information about that job, including the job name, user name, and job number along with the job status. As is the case with the Navigator GUI, all of these columns can be sorted, so it can be very easy to find the largest temporary storage bucket.

Fig3

In addition to the capability to link the temporary storage buckets through the System Status interface, IBM i Services has a service for temporary storage – SYSTMPSTG. The temporary storage service provides a programmatic interface to access the temporary storage buckets and provides the same information as the GUI does. The SYSTMPSTG view contains one row for every temporary storage bucket that is tracking some amount of temporary storage across the system.

 

 

June 03, 2014

7.2 Improved Temporary Storage Tracking (Part 1)

I will write at least three blogs on the temporary storage tracking enhancements in the 7.2 release - there is simply too much to cover in one blog. As you may recall, I briefly wrote about “New accounting method and tools for tracking use of temporary storage” in IBM i 7.2 – Systems Management Improvements.

Temporary storage does not persist beyond the current IPL – when the partition is IPLed, all temporary storage is released. During the life of the IPL, temporary storage is (generally) associated with the job that allocates it and any temporary storage not freed when the job ends is automatically released. However, there are some types of temporary storage that are not associated with a job or storage may be allocated by one job and released by another. Prior to 7.2, if you want to find out which jobs are consuming the most temporary storage it can be a tedious process; you can use Work with System Activity (WRKSYSACT and F11) to display storage allocated and dealloacted by each job, but these values also include permanent storage and they are deltas, not totals. You can look at the job (via work with job) to see the actual temporary storage used, but you have to examine each job individually. I summarized some of the challenges when I wrote the blog Understanding Disk Space Usage

In the 7.2 release, IBM has made many improvements to address these difficulties. The most significant is how temporary storage is tracked. How temporary storage is allocated (to a job, independent of a job, shared between jobs) does not change, but how the system keeps track of where temporary storage is used is improved. The system now has a concept of temporary storage buckets that can tell you exactly where all the temporary storage is being used. 

There is a description of the temporary storage buckets in the 7.2 Knowledge Center – it’s hidden away under Database topics, IBM i Services, Storage Services (and I’ll write more about the storage services in a future blog…). I’ve copied that text here:

There are two types of temporary storage buckets:

• global buckets that are used to track temporary storage that is scoped to all jobs on the system

• job buckets that are used to track temporary storage that is scoped to a single job

Each bucket has a bucket number. Global buckets managed by the licensed internal code have bucket numbers from 1 to 4095. Global buckets managed by IBM i Work Management have bucket numbers from 4096 to 65535. Job buckets have numbers greater than 65535.

A job temporary storage bucket is assigned when the job starts and does not change for the life of the job. A job temporary storage bucket will normally be empty after the associated job ends and all working storage for the job is deleted or freed. If the job temporary storage bucket is empty after the job ends, the bucket becomes available to be associated with a new job. If the job associated with the job buckets ends and some temporary objects tracked to that job are not deleted, the job bucket will show a status of *ENDED as well as the date and time that the job ended. These job buckets identify jobs that are not deleting all of their temporary storage when the job ends.

Statistics for each job bucket indicate the current amount of storage (in bytes) used for temporary storage tracked by the bucket, the storage limit (in bytes) for disk storage used for temporary storage tracked by the bucket, and the peak amount of disk storage (in bytes) used for temporary storage tracked by the bucket. For job buckets, the storage limit will reflect the MAXTMPSTG value of the class (*CLS) object specified when the job was submitted; a null value is returned if the job has a MAXTMPSTG value of *NOMAX.

These changes for managing temporary storage were driven by requirements from clients small and large; IBM i does a great job at managing storage for you, but if something went wrong, it was challenging to identify the root cause. The changes in 7.2 should make it much easier to determine where your temporary storage is being consumed.

Look forward to future blogs where I’ll write more about how you can now view and manage your temporary storage usage much more effectively with the 7.2 release.