IBM i 7.2 - Batch Model

In the IBM i 7.2 release, there is a new function called “Batch Model”; the 7.2 announcement stated: “The Batch Model has been added, providing the ability to change environment variables, modeling how those changes would impact the batch window.” Batch Model is a utility that can help you understand your batch workloads and can assist in predicting the performance result of hardware changes (processors or disk) or an increasing workload on your batch window.

You will find Batch Model under Sizing within the Performance tasks in Navigator, as the following image shows. You must have the IBM i Performance Tools licensed program product (5770-PT1, option 1) installed in order for these tasks to be available.

Batch model runs over Collection Services data; you identify the timeframe that represents your batch application when creating the batch model. Once the model is created, you may need to calibrate the model; this is needed if the system didn’t create the model as you require. For example, your batch application may consist of several jobs and you must ensure those jobs and linkages (parent/child relationship) are correct.

Once your batch model has been correctly calibrated, you then use the Change Batch Model function to change various configuration components of your batch application. You can change the system model (processor) configuration, the disk configuration and the workload job characteristics (move, copy, delete jobs from the model. Once these changes are made, you analyze your batch model to get the results. Once the analysis is complete, you can investigate the results with the Performance Data Investigator. The Performance Data Investigator has a few ways to look at your modeled results, comparing the measured workload you started with to the results of the modeled workload.

One example is the resource utilization overview, as the following image shows:

You can use Batch Model on Collection Services data from the 6.1 or 7.1 releases. You do need to have a 7.2 partition to access the Navigator interface for Batch Model, but you can save your Collection Services data from 6.1 or 7.1 onto that 7.2 partition and select that older performance data when creating your batch model. This is very handy for sizing exercises if you are planning a HW or SW upgrade. Note: Support for the POWER8 hardware models is not yet available but will be coming soon in a PTF.

The development team that worked on creating Batch Model has provided an article, “How to use the Batch Model performance tool” on developerWorks, where you can find a more in-depth review of this tool.

Some of you may be familiar a tool called Batch/400. Batch/400 was a utility written many years ago as an as-is tool to help predict the impact of changes on a batch application; it had a command called BCHMDL. While I was writing this article, I thought perhaps I might be able to find some historical references for the Batch/400 utility. It turned out to be a bit difficult to find much documentation about it. An Internet search on “Batch/400” revealed “Batch 400 American Stout brewed by Swamp Head Brewery.” An Internet search on “BCHMDL” told me about the British Columbia Hockey League … with a couple results more along the line of what I was really looking for.

 

New IBM i 7.2 TCP/IP Configuration Information Commands

Two new commands have been provided in IBM i 7.2 to assist clients with handling TCP/IP configuration information.

The new Retrieve TCP/IP Information (RTVTCPINF) command will gather related TCP/IP configuration items (DB files, data areas, validation lists, environment variables, IFS files, etc.) for the systems TCP/IP processing as well as supported TCP/IP servers. 

A full system backup of this information is still recommended by using the Save System (SAVSYS) command. The RTVTCPINF command will take a snapshot that ensures all related TCP/IP configuration items are saved together as a single “working set.” This helps to avoid problems where items that must be processed together as a single unit get out of sync, something that can easily happen when related items reside in multiple places on the system. The RTVTCPINF command is automatically run and saved into the QUSRSYS library before the QUSRSYS library is saved, similar to the Save System Information (SAVSYSINF) command.

The Update TCP/IP Information (UPDTCPINF) command is available to restore the selected type of TCP/IP configuration information. This can be used to recover from problems that are reported against some TCP/IP configuration item, or to quickly back out a TCP/IP configuration change to a last known working configuration. It can also be useful to transport a working TCP/IP configuration to another system.

For more details, read the article, “New TCP/IP Configuration Recovery Enhancements.” 

 

IBM i 7.2 Improved Temporary Storage Tracking (Part 3)

I’m continuing the series of blogs on improved temporary storage tracking in IBM i 7.2 –part 1 covered the changes made to provide improved tracking and part 2 covered the changes made so you can better understand temporary consumption at a system level. You may want to read these blogs if you have not done so already.

This week, I want to review the enhancements that allow you to better view and understand where the temporary storage is being used from a job-specific perspective. If you have discovered a problem with temporary storage consumption on your IBM i partition, you generally detect the issue at the system level but will need to identify the root cause. The problem is often caused by a job (or jobs) that is consuming the storage.

 

Active Jobs

Using Navigator for i to view Active Jobs (or active jobs by memory pool, or active jobs by subsystem), the temporary storage used column will be displayed by default. Simply click on the column heading (twice) to sort the table by the jobs that are using the most temporary storage. (Yes, I’ve already asked the Navigator development team to add a way to select the sort order with just one click….)

Also, a reminder about using the Active Jobs interface on Navigator – if you want to see updated values displayed in the table, don’t forget to click the “refresh” button – the table is not dynamically updated. (Yes, I’ve already asked the Navigator development team to add dynamic updates….)

The screen capture below shows the Active Jobs display sorted by the temporary storage used column.

 

Work with Active Jobs (WRKACTJOB)

For those of you who prefer to use the green-screen interface to manage i, you will appreciate the enhancement to WRKACTJOB to provide temporary storage as a column. You can simply use WRKACTJOB and press F11 two times to display the new column. You can sort on the column in the usual way with your cursor placed on the column heading and press F16. In addition, if you prompt the WRKACTJOB command, there is a new value, *TMPSTG, on the sequence (SEQ) parameter that you can use to immediately display the view with the jobs sorted by the temporary storage column.

The screen capture below shows the WRKACTJOB display sorted by the temporary storage column.

 

Work with Job

There is also a minor change to a job’s run attributes. Prior to 7.2, you could work with a job and see the maximum amount of temporary storage allowed and the current amount used. 7.2 enhances this by including peak usage by that particular job.

Interestingly, this support is only on the green screen WRKJOB (option 3) and not on the job properties GUI. (Yes, I’ve already asked the Navigator development team to add the peak temporary storage used to the GUI….)

As an aside, most of you should already know that you configure the maximum temporary storage allowed by a job on the class object. In 7.2, the MAXTMPSTG parameter on the class object is now in megabytes rather than kilobytes. Read the blog Jobs Exceeding their CPU or Storage Limits are now Held on why you should start setting the maximum temporary storage value.

 

Temporary Storage Details – buckets per job

In part 2, I wrote about how you can view the temporary storage details to find a table of all of the temporary storage buckets along with their size information. That article noted that there is a bucket for each active job; the temporary storage details view is another way to find the jobs that are using the temporary storage in addition to Navigator’s Active Jobs task.

Remember you can use filters to subset what you see, so you could add a filter on the “Job Status” column to only show those entries that start with “*” – this is a simple way to filter out the global entries and only see the temporary storage information for jobs. You can then sort the bucket current size column to find the job(s) using the most temporary storage. The nice thing about using this approach is that you also can find jobs that have ended but still have temporary storage allocated.

If you discover active jobs that are consuming an unexpectedly large amount of temporary storage, you can take proactive actions by holding or ending them. While you can see if ended jobs had temporary storage that was not released, you cannot take any actions upon those jobs. Diagnostics or debugging will be necessary to determine what the job was doing when it ended that prevented the system from freeing storage.

 

It looks like I will have at least two more blogs on related topics for temporary storage tacking in the 7.2 release. In future blogs, I will discuss support added to Collection Services and how to set up notification.

 

 

7.2 Improved Temporary Storage Tracking (Part 2)

Last week, I wrote about the underlying changes made in the 7.2 release of the IBM i operating system for improved temporary storage tracking.

This week, I want to review the system status enhancements that allow you to better view and understand where the temporary storage is being used from a systemwide perspective.

Work with System Status (WRKSYSSTS)

Prior to the 7.2 release, WRKSYSSTS showed temporary storage as “Current unprotect used” and “Maximum unprotect” values. These have simply been reworded to be “Current temporary used” and “Peak temporary used,” as the following screen capture shows:

System Status (the Navigator Way)

Using Navigator for i to view System Status, on the Disk Space tab, you will find the current and maximum temporary storage used values; these are not new and have been on the GUI System Status for some time.

What is new with the 7.2 release is the “Temporary Storage Details” button on this view, as you can see on the screen capture below.

When you click the “Temporary Storage Details” button, it will bring up a table of information regarding the temporary storage buckets that I wrote about last week. The example below shows the table of the temporary storage buckets that is displayed and the columns of information: the current bucket size, the bucket peak size, and the bucket limit size. (I’ll write about setting bucket limits in a future blog.) In addition, for any temporary storage bucket that is associated with a job, you will see information about that job, including the job name, user name, and job number along with the job status. As is the case with the Navigator GUI, all of these columns can be sorted, so it can be very easy to find the largest temporary storage bucket.

In addition to the capability to link the temporary storage buckets through the System Status interface, IBM i Services has a service for temporary storage – SYSTMPSTG. The temporary storage service provides a programmatic interface to access the temporary storage buckets and provides the same information as the GUI does. The SYSTMPSTG view contains one row for every temporary storage bucket that is tracking some amount of temporary storage across the system.

 

 

7.2 Improved Temporary Storage Tracking (Part 1)

I will write at least three blogs on the temporary storage tracking enhancements in the 7.2 release - there is simply too much to cover in one blog. As you may recall, I briefly wrote about “New accounting method and tools for tracking use of temporary storage” in IBM i 7.2 – Systems Management Improvements.

Temporary storage does not persist beyond the current IPL – when the partition is IPLed, all temporary storage is released. During the life of the IPL, temporary storage is (generally) associated with the job that allocates it and any temporary storage not freed when the job ends is automatically released. However, there are some types of temporary storage that are not associated with a job or storage may be allocated by one job and released by another. Prior to 7.2, if you want to find out which jobs are consuming the most temporary storage it can be a tedious process; you can use Work with System Activity (WRKSYSACT and F11) to display storage allocated and dealloacted by each job, but these values also include permanent storage and they are deltas, not totals. You can look at the job (via work with job) to see the actual temporary storage used, but you have to examine each job individually. I summarized some of the challenges when I wrote the blog Understanding Disk Space Usage. 

In the 7.2 release, IBM has made many improvements to address these difficulties. The most significant is how temporary storage is tracked. How temporary storage is allocated (to a job, independent of a job, shared between jobs) does not change, but how the system keeps track of where temporary storage is used is improved. The system now has a concept of temporary storage buckets that can tell you exactly where all the temporary storage is being used. 

There is a description of the temporary storage buckets in the 7.2 Knowledge Center – it’s hidden away under Database topics, IBM i Services, Storage Services (and I’ll write more about the storage services in a future blog…). I’ve copied that text here:

There are two types of temporary storage buckets:

• global buckets that are used to track temporary storage that is scoped to all jobs on the system

• job buckets that are used to track temporary storage that is scoped to a single job

Each bucket has a bucket number. Global buckets managed by the licensed internal code have bucket numbers from 1 to 4095. Global buckets managed by IBM i Work Management have bucket numbers from 4096 to 65535. Job buckets have numbers greater than 65535.

A job temporary storage bucket is assigned when the job starts and does not change for the life of the job. A job temporary storage bucket will normally be empty after the associated job ends and all working storage for the job is deleted or freed. If the job temporary storage bucket is empty after the job ends, the bucket becomes available to be associated with a new job. If the job associated with the job buckets ends and some temporary objects tracked to that job are not deleted, the job bucket will show a status of *ENDED as well as the date and time that the job ended. These job buckets identify jobs that are not deleting all of their temporary storage when the job ends.

Statistics for each job bucket indicate the current amount of storage (in bytes) used for temporary storage tracked by the bucket, the storage limit (in bytes) for disk storage used for temporary storage tracked by the bucket, and the peak amount of disk storage (in bytes) used for temporary storage tracked by the bucket. For job buckets, the storage limit will reflect the MAXTMPSTG value of the class (*CLS) object specified when the job was submitted; a null value is returned if the job has a MAXTMPSTG value of *NOMAX.

These changes for managing temporary storage were driven by requirements from clients small and large; IBM i does a great job at managing storage for you, but if something went wrong, it was challenging to identify the root cause. The changes in 7.2 should make it much easier to determine where your temporary storage is being consumed.

Look forward to future blogs where I’ll write more about how you can now view and manage your temporary storage usage much more effectively with the 7.2 release.

 

 

Run an HTTP Server in its own Subsystem

One of the enhancements in the IBM i 7.2 release is the ability to configure an HTTP server to specify the subsystem in which it runs. Given that more and more work is driven through a Web server, I’m sure many of you will agree this is a great enhancement. The IBM i 7.2 announcement has the following:

The ability to run HTTP server jobs in a custom subsystem. Jobs are no longer tied to QHTTPSVR. This ensures HTTP server jobs have their own dedicated memory pool as needed.

But it doesn’t tell you how to do it, nor does it give you a clue where to find that information. I wasn’t able to find it in the IBM i 7.2 Knowledge Center (until I learned enough to know what specifically to search for). Even the HTTP Server section on “What’s new for IBM i 7.2” did not mention the capability to configure the subsystem the server is to run in.

I was determined to figure this out without asking an IBMer, so I went to the Internet Configurations with Navigator for i to see if I could find it on the GUI. There was nothing obvious there, so I started to explore and opened up the directive index – nothing related to subsystems under “S”…. a search on “sub” however, did find “HTTPSubsystemDesc” and there were the four directives for the subsystem configuration information:

• • HTTPRoutingData - specifies the user defined routing data for HTTP server jobs.
• • HTTPStartJobDesc - specifies the user created job description that defines how HTTP server jobs should be run.
• • HTTPStartJobQueue - specifies the user created job queue to which the HTTP server jobs will be submitted.
• • HTTPSubsystemDesc - specifies the user created subsystem that the HTTP server runs in.

Now that I knew the directive names, I could find documentation in the Knowledge Center. There is an interesting interplay between the various directives and the behavior of where the jobs run. Copied directly from the Knowledge Center:

Note: To make HTTP server run in subsystem other than QHTTPSVR, at least HTTPStartJobQueue directive is required to be specified and the desired subsystem is MUST in active status before starting HTTP server. If only HTTPSubsystemDesc directive is specified, only the specified subsystem is started and HTTP server jobs still run under QHTTPSVR. If only HTTPStartJobQueue is specified but the desired subsystem is not active at that moment, the HTTP server jobs will not be started until the subsystem is started.

Once I had found the basic information on the directives, I did an Internet search on “HTTPSubsystemDesc”, where I found the IBM technote, How to run HTTP Apache in its own subsystem. Documentation! Success!

The general approach is to create the class, the subsystem description, job queue and job queue entry, and the routing entry as you would when you are setting up your own subsystems. The magic is then modifying the HTTP server configuration with the four directives that specify the subsystem configuration information. Once you have the subsystem configuration in place, you can start your HTTP server as you typically would and it will run in the customized subsystem.

Now you know how to do this configuration and this blog gives the Internet one more article that will hopefully help others when they do a search to figure out how to do this.

 

 

IBM i 7.2 – Systems Management Improvements

IBM i 7.2 is finally announced!

It’s going to be great to start writing about all the things in this release. Steve Will’s blog, Announcing IBM i 7.2, reviews the overall announcement; I want to briefly review some of the systems management enhancements (and there are a lot of them!). I’ll write more in-depth blogs on these topics and more over the next several months.

I’ve taken some text directly from the announcement letter (in italics) and have added some additional information:

New accounting method and tools for tracking use of temporary storage

IBM i generally does a great job of automatically managing the temporary storage on the system, but if you have an issue with a storage leak, tracking down the problem can be somewhat challenging. The enhancements in 7.2 improve the accounting of temporary storage that will make it easier to determine where the temporary storage is being used. One simple but very useful change is the temporary storage used by a job has been added as a column in the Active Jobs view in Navigator as well as on WRKACTJOB.

I’ll be speaking on this topic at COMMON in the “Manage Work Better with Better Work Management” presentation.

Navigator has some wonderful enhancements in 7.2:

IBM Navigator for i increases the effectiveness of system management activities by adding the ability to manage and compare PTFs between systems, and to monitor, analyze, and predict workload impact on the system. Performance Data Investigator (PDI) allows further monitoring, detection, and adjustment of specific performance issues. Using the collection data as a base, PDI can model future workloads.

• • PTF support has been added to Navigator – you can view the PTFs and PTF groups that are installed on your partition; you can install, remove and clean up PTFs from the GUI as well. And as the announcement stated, you can now do the compare and update of PTFs via the Web console.
• • A major new function called Batch Model has been added to the performance tasks. Batch model uses Collection Services data collected while the application is running and provides an interface to model a variety of changes and see the effect on that application. For example, you could model how the application’s performance would be affected if the disk configuration were changed.
• • System Monitor support has been added to Navigator as well. These System Monitors are very similar to the system monitors that existed in the IBM i System Navigator product as part of Management Central. The System Monitors in Navigator use the Performance Data Investigator to display the monitor graph data. I’ll be talking about this new support in detail at my “Monitoring System Health and Performance of IBM i” at COMMON.
• • Finally, Collection Services has many new metrics, as is typical for a new release. The Performance Data Investigator has many new charts of performance metrics. Details will be saved for a future blog. For those of you at COMMON, I’ll be speaking on these topics as well.

You can read more about these Navigator enhancements in Tim Rowe’s blog at blogs.systemideveloper.com.

The ability to run HTTP server jobs in a custom subsystem. Jobs are no longer tied to QHTTPSVR. This ensures HTTP server jobs have their own dedicated memory pool as needed.

I think this one is pretty self-explanatory – and a very welcome enhancement!

New PTF improvements for tracking of Security Audit Log

The PTF install history is now tracked in the Security Audit Log. This includes tracking changes to system based on PTF activity on the system and will track both PTF operations and/or PTF object changes.

More immediate Apply options for PTFs

IBM has added support for conditional immediate PTFs. Conditional immediate PTFs allow an immediate apply PTF which supersedes a delayed PTF to be immediately applied if the superseded PTF has already been applied to the system.

The release also has many security and networking enhancements – there’s just too much to cover in this one blog.

The upcoming COMMON conference will be a great place to learn more on the 7.2 enhancements. The COMMON sessions from IBMers will have titles and abstracts updated this coming week to reflect sessions with 7.2 content. If you haven’t registered yet, there’s still time to do so.

You can find the 7.2 documentation in the IBM Knowledge Center.

I have a lot of new blog articles to write!