Move My i

Today is a major announcement day for IBM i with Technology Refresh 4 (TR4). Steve's “You and i” blog reviews the content of this announcement.

Last year, IBM provided support to suspend an i partition, which I previously wrote about in “Suspend my i”. 

The most significant function in IBM i TR4 is Live Partition Mobility. With Live Partition Mobility, you can take an active, running IBM i partition and move it from one frame to another frame, without disruption to your applications. I'm not going to discuss more about what mobility is and why you might want to use it, as Steve covered that in his blog. What I do want to write about is a little more detail on just what you need to accomplish this.

There are both hardware and software prerequisites:

• POWER7 tower/rack system for both the source and destination systems
  • Firmware service pack 730_xx, 740_xx, or later
  • IBM Hardware Management Console V7R7.5.0M0, or later
• All I/O must be virtual and supported by VIOS; it can be Virtual SCSI (VSCSI), Virtual Fibre Channel (NPIV) or Virtual Ethernet
• The configuration must use only external storage and that storage must be accessible to both the source and destination systems
• Both source and destination systems must be on the same Ethernet network
• IBM i 7.1 TR4 PTF group – SF99707 level 4
• PowerVM Enterprise Edition on both the source and target system
• VIOS 2.2.1.5 (FP25 SP3) on both the source and target system

There are a few restrictions that are too detailed to review in this blog; the Live Partition Mobility documentation in the IBM i developerWorks site reviews the complete set of prerequisites, requirements and restrictions.

The actual move of a partition is initiated from the HMC. IBM Systems Director VMControl has relocation, which you can imagine will ultimately support IBM i as well.

In general, applications and the operating system are unaware that the partition is moved from one system to another. There are some exceptions to this, such as Collection Services; when the partition is starting to run on the target system, the Collection Services collector job will cycle the collection so correct hardware information is recorded on the target system.

There are two new work management exit points that are part of the IBM i suspend and mobility support. These exit points allow to you run a program before and after a partition is suspended or moved. (These exit points are the same, regardless of whether you are suspending a partition or moving a partition). 

            Work with Registration Information

 

 Type options, press Enter.

  5=Display exit point  8=Work with exit programs

 

              Exit

   Exit         Point

 Opt Point         Format  Registered Text

 _  QIBM_QWC_RESUME    RSMS0100   *YES   Resume system

 _  QIBM_QWC_SUSPEND   SSPS0100   *YES   Suspend system  

The suspend exit is called before the partition is suspended or moved and is called two twice. The first time it is called to check to see if the operation is allowed. The second call is to prepare for the move or suspend operation.

The resume exit is called after the move is complete or after the system has been restarted from a suspended state. Applications can use these exit points to monitor and accept or refuse move or suspend requests. Running an exit program after a move or suspend also allows an application to refresh any copies of any system-specific information, log information about mobility requests for licensing or other purposes, adjust heart beat rates or whatever actions an application may require.

For more information on these exit points, refer to the “What's New”for APIs in the IBM i 7.1 Information Center.

The following messages may be logged to the QSYSOPR message queue as part of the suspend or move request:

• CPI09A5 - Partition suspend request in progress.
• CPI09A6 - Partition not ready for suspend request.
• CPI09A7 - Partition suspend request canceled.
• CPI09A8 - Partition resumed after migration.
• CPI09A9 - Partition resumed from hibernation.

The system serial number will change when a partition has been moved. The model and processor feature may also change.

The documentation on enhancements added to IBM i via Technology Refreshes can be found on the IBM developerWorks web site.

Other sources for documentation on Live Partition Mobility for IBM i include:

• Partition mobility overview for HMC
  
• IBM PowerVM Live Partition Mobility
  
• API Guide (For registering system exits)
• Licensing information
• Software Knowledge Base

          Search for “Live Partition Mobility” to find any usage notes.

 

IBM Improves the Handling of the Inactivity Timeout

The inactive job time-out (QINACTITV) system value defines the minimum amount of time that an interactive job can be inactive before an action is taken. The action that is taken is defined by the inactive job message queue (QINACTMSGQ) system value. A related system value, time interval before disconnected job ends (QDSCJOBITV), defines how long a job can be disconnected before the job is ended.

The handling of inactive job time out (QINACTITV) is being changed with a 7.1 PTF (the other system values are unaffected) to address two long-standing problems.

1. Accuracy. Prior to this change, a job can be inactive for up to twice the specified interval before an action is taken. For inactivity intervals measured in hours, this level of uncertainty is simply too high.
   
   
2. Performance. A large number of jobs can be ended at once when the subsystems check for inactivity. This can affect overall system performance at the time the jobs are being ended.

The change is available only for the 7.1 release with PTF SI46398. This is a delayed PTF, so it requires an IPL to put in on or to take it off.

First of us, let's review this change as it pertains to accuracy.

Prior to this change, the subsystem would wait for the QINACTITV number of minutes between the times it checked for inactive interactive jobs. Jobs can be inactive for nearly two intervals before the inactivity is detected; if a job runs just after the system checks for inactivity, at the next check it will not be inactive for the full interval, so the timeout will not occur until the following interval. For example, if you have QINACTITV set to 60 minutes, jobs may need to be inactive for nearly 120 minutes for the inactivity to be detected.

The PTF improves the timeliness in which inactivity is identified.

The QINACTITV function is still enforced by the subsystem code periodically checking for inactivity, but with the change, the subsystem tries to check more often to improve the accuracy. In most cases, the subsystem will now check for inactivity every 10 minutes and will keep track of how long a job has been inactive. If the QINACTITV system value is less than 15 minutes, the subsystem will continue to use the value specified by QINACTITV. (Checking for inactivity every 10 minutes only helps for the cases where QINACTITV is greater than or equal to 15 minutes).

A job can still be inactive for longer than the QINACTITV system value. For any given QINACTITV, you can work out an exact value for how long a job can be inactive before an action is taken. The job becomes inactive sometime during the 10 minutes between checks. So, in the simple case where the QINACTITV value is evenly divisible by 10, a job can be inactive for up to 10 minutes plus the QINACTITV value. But the QINACTITV value does not have to be evenly divisible by 10 and the subsystem cannot take action until the next time it checks for inactivity. For an QINACTITV value such as 101, a job can be inactive for at most the QINACTITV time plus 19 minutes.

Consider an example where QINACTITV is set to 90 minutes. With the old code, the subsystem would take the action defined by the QINACTMSGQ system value if a job was inactive anywhere from 90 minutes to 180 minutes. With the new code, the subsystem will take action if a job has been inactive anywhere from 90 minutes to 100 minutes.

Consider another example. In this case QINACTITV is set to 45 minutes. With the old code, the subsystem would take action if a job was inactive anywhere from 45 minutes to 90 minutes. With the new code, the subsystem will take action if a job has been inactive anywhere from 50 minutes to 60 minutes.

A job will be inactive for at least the QINACTITV value before action is taken. But sometimes you might be interested in "at most" rather than "at least". If you have a requirement that jobs must not be left inactive for longer than some specified value, you need to set the QINACTITV value to something less than the limit that you want to enforce. With the old code, the QINACTITV value had to be half of the limit that you wanted to enforce. With the new code, the QINACTITV value can be much closer to the limit that you want to enforce.

If system value QINACTMSGQ is set to the name of a message queue and the job identified by the CPI1126 message remains inactive and is not ended or disconnected, another CPI1126 is sent the next time the subsystem checks for inactivity. This behavior is unchanged, but the subsystem may be checking for inactivity much more often. This could affect the logic of whatever program is handling messages sent to that message queue. The program will have a lot less time before it gets another CPI1126 message for a given job.

It can still be useful to set the QINACTMSGQ (Inactive job message queue) system value to the name of a message queue so that you can control which jobs get ended or disconnected and how many jobs get processed at a time. Activity can be uneven and can be influenced by external factors such as lunchtime. A significant number of jobs can become inactive within the same 10-minute interval. This is less of a problem with a 10-minute interval than it is with an interval measured in hours. Also, due to other changes in the code, this is less of a problem when the interactive work is spread across multiple subsystems.

Now on to the performance improvements provided by this PTF.

Subsystems that are started at the same time will no longer check for inactivity at the same time. When the subsystems start or when the QINACTITV system value is changed, the subsystems now use a small but variable delay when setting up the checking for inactivity. This helps reduce the number of jobs that are disconnected or ended at a single time. With the old code, you could introduce delays between starting subsystems and these delays would translate more or less directly into delays between the times the different subsystems would check for inactivity. With the new code, every subsystem has to check every 10 minutes, but the system takes care of spreading out the work. Now, the only reason to have delays between starting subsystems would be to manage the amount of work done when starting the subsystems.

When a subsystem sets up its checking for inactivity, it has no idea how many other subsystems will need to do this same type of work. In addition to checking for inactive interactive jobs, subsystems also check the number of prestart jobs that are not being used. PTF SI46398 spreads out the work for ending of inactive interactive jobs and for ending of unused prestart jobs.

Because it is still the subsystem that checks for inactivity, much remains the same. But the system now reacts to inactivity sooner and spreads out the work better. This makes the enforcement of QINACTITV more predictable and reduces the impact on overall system performance.

 

I'd like to thank Dan Tarara for this blog; Dan is a member of the IBM i work management team. Thanks, Dan!

 

Lesser-Known Features of Work With System Activity

Hopefully most of you now know that in the 6.1 release, IBM moved the Work with System Activity (WRKSYSACT) command from the Performance Tools licensed program product into the base operating system. WRKSYSACT has been around for a long time, but it has a few features that you may not be aware of.

Work with System Activity is a great tool for getting a basic understanding of what is happening on your partition in terms of performance. WRKSYSACT will show you jobs, threads, and tasks that are using system resources – CPU utilization, synchronous and asynchronous I/O, storage allocation information, etc. WRKSYSACT is the only green-screen command interface in the operating system where you will find information about tasks. Other command interfaces – such as WRKACTJOB – only show information about jobs and threads.

It would take far too much space to write about all of the things you can see through WRKSYSACT; rather, there are a few features that I want to discuss in more detail.

 

WRKSYSACT in Batch Mode

First of all, did you know that you can run WRKSYSACT in a batch monitoring mode and write the performance data to a file? You enable the batch mode with the OUTPUT parameter; by default this is * and the information is presented on your display. But there are two additional options, *FILE and *BOTH, which will write the performance data to a database file (QAITMON in QPFRDATA by default). You can specify the sample intervals with the INTERVAL parameter (which also controls the auto-refresh intervals as well) and how many intervals are collected (NBRITV parameter). When you take the option to write the data to a file, a job named WRKSYSACT is submitted to QBATCH to collect the performance data.

The Print Activity Report (PRTACTRPT) command can then be used to print out a report from the sample data collected by WRKSYSACT. The PRTACTRPT command can generate a summary report as well as a detailed report and has options to allow you to specify a timeframe of interest, specific jobs you may want in the report, as well as what criteria the report should use in ordering.

 

Wait Information

Wait information will tell you what a particular job, thread or task is waiting on when it is waiting to do more work. I wrote about wait accounting in “i Can Tell You Why You're Waiting” some time ago. You can also see wait information using WRKSYSACT; you simply need to determine which job/thread/task you want to look at, and use option 6, Wait detail, for that job or task. Option 6 will bring up a display that will list the wait categories for that job/thread/task. Be aware that this is a high-level summary of the kinds of wait for that job or task. However, it make give you some clues about what job or task you need more detailed information for; you can then use the Performance Data Investigator with Collection Services or Job Watcher data to view detailed wait information.

 

Storage Allocated/Deallocated

WRKSYSACT will also show the storage allocated or deallocated by a job/thread/task by pressing the F11 key to get to View 4. I wrote about “Understanding Disk Space Usage” and in that blog reviewed WRKSYSACT and the storage allocated/deallocated information.

 

Scaled CPU

Scaled CPU time is a metric that can be used to determine if energy management features, such as power capping or power savings, are in effect on your IBM i partition. These energy management capabilities can be turned on through Active Energy Manager, which is a plug-in to IBM Systems Director. When you use these energy management features, the system's processor may run at lower or higher frequencies, which can affect your partition's performance. I wrote about this topic at length in the blog “i Can Understand Scaled CPU Time.”

On the WRKSYSACT display, the value “Average CPU rate” will be displayed if you are using energy management capabilities. This rate is an easy way to determine if your processor has been slowed down due to power capping or power savings.

One final note – one thing that has not changed with WRKSYSACT is the limit of one user at any given time. WRKSYSACT collects a lot of information from critical interfaces in the operating system and the limit of one user ensures limited contention on those critical interfaces.

 

 

 

 

Interrupt Job Exit - Invoking a Program in Another Job

In last week's blog on QTEMP, I mentioned at the very end that I'd write about the interrupt job exit program this week.

Introduced in the 5.4 release, the interrupt job capability allows a program to be run within a specified job. A user wanting to run a program in a different job prior to 5.4 was essentially limited to using the submit job command to start a new job to run the program; however, sometimes there was a need to run a program in a job that had already been started. The interrupt job support enabled this capability.

There are two major components to this function:

• The ability to control whether the function is allowed
• The programmatic interfaces to request and execute the program in the target job

The Allow jobs to be interrupted (QALWJOBITP) system value is the systemwide control for whether you can interrupt another job to call a program. This system value has three options:

• 0 – jobs cannot be interrupted (the default)
• 1 – jobs can be interrupted but new jobs default to be uninterruptible
• 2 – jobs can be interrupted and new jobs default to be interruptible

This system value is the “big switch” -- setting it to 0 disables this capability systemwide. The system value must be set to either 1 or 2 in order to call a program in another job. This system value is further controlled since it is one of the system values that can be “locked” -- that is, set so it cannot be changed during normal system processing. The lock function of security system values page in the Information Center discusses this topic. If you need to change a system value that is locked in this manner, you need to change the “Allow security-related system value changes” parameter in system service tools (SST).

If this system value is set to 1 or 2, controlling this function then goes down to the job level. The default interruptible attribute for new jobs is taken from the system value when the job is started; the Change Job Interrupt Status (QWCCJITP) API can be used to change the interruptible status of a given job. The change job interrupt status API retrieves and optionally modifies the interrupt status of the job in which it is run. If the job is modified to be interruptible, programs called by the interrupt job API will be able to run in this job.

Calling a program in another job has two parts -- the API called in a job to specify the target job and the program to be run, and the exit program in the target job to actually run the program.

The requesting job uses the Call Job Interrupt Program (QWCJBITP) API to specify the program and the job in which the program is to be invoked. The target job must be active and must not be a system job, a subsystem job, or a spool reader or writer job. This API will return with an error if the “Allow jobs to be interrupted” system value is set to 0. If the system value is set to 1 or 2, the actual interruptibility status of the target job determines whether the function is allowed.

The target job, if interruptible, would then run the specified program through the Call Job Interrupt Exit Program (exit point QIBM_QWC_JOBITPPGM) exit program. This exit point must be defined in the system registration information. The program runs in the initial thread of the target job and if that job has multiple threads, secondary threads will continue unaffected. In addition, the exit program will run under the user profile of the requesting job; that is, the user profile in the target job is swapped to the same user profile of the job that invoked the call job interrupt program API.

If any errors occur during the running of the program in the target job, it is that job which needs to be looked at for those errors; for example, any messages logged in the job log.

As you can see, there are many elements to running a program in a target job and the capability to do so is not enabled by default. This feature is probably not something you need very often, but it can be useful when developing tools that help with specific systems management challenges, particularly when one job needs programmatic interfaces to manipulate or manage other jobs.

 

 

 

 

Instrument Your Applications for IBM i Job Trace

Last week I wrote about the IBM i Job Trace capabilities.In that article, I briefly mentioned that you can instrument your own application so you can collect your own trace data as part of a job trace.

The secret to this support is something called the “QTRC APIs,” which were introduced with the 6.1 release. These APIs are documented under the “Problem Management” category of APIs in the IBM i Information Center. The QTRC API documentation section has much more information than just the programming interfaces; it discusses how you use these APIs and covers the related information you need to know to fully utilize these programming interfaces.

An important concept in how the QTRC APIs work is something called the “component.” The component is a parameter you specify on each of the QTRC APIs to identify the trace collection, and the component is the parameter on the STRTRC command that will turn trace on or off for your component. Below is the documentation taken from the Information Center on a QTRC Component:

Component

A 10-byte character string which names the application component. Each Trace Point has an associated component that is specified on the QTRC Trace APIs. The component name is also shown in the trace output for each Trace Record that is written. A component is defined to the QTRC Trace environment when specified on the Start Trace (STRTRC) CL command. Component names should only contain characters in the invariant character set. Avoid using lowercase a through z. Application component names should contain an abbreviated company name along with the application name. Component names starting with an asterisk (*) are reserved for the operating system. Component names starting with Q are reserved for IBM licensed programs.

There are three basic steps to collect diagnostic data from your application as part of a job trace:

1. Instrument your application to collect the desired diagnostic information.

This is the step that will require the most thought. What kind of data will you need for diagnostic purposes? What are the performance implications of collecting additional data? Do you have different types of data you would need for different kinds of problems?

You use the QTRC APIs to collect the data you need.

• Use the Check Trace Level API to determine the trace level that has been set for your component trace. This check will determine whether trace has been turned on for your component; if the required trace is not active, you can easily bypass the data collection logic in your application and avoid adding overhead to your application when trace has not been turned on.
• Based upon the trace level, you then write trace records; you can collect text data, hexadecimal data, and the call stack. The QTRC API documentation in the IBM i Information Center describes how to use these APIs.

2. Collect your trace data using the STRTRC command.

The magic to using the STRTRC command to collect your own trace data is on the TRCTYPE parameter. While IBM has defined special values to start trace collection for components within the operating system, the TRCTYPE keyword is not restricted to just these special values; you can specify your own component name on the TRCTYPE parameter.

The operating system components that you can trace with the TRCTYPE parameter are using the QTRC APIs as well – the QTRC APIs allow you to instrument your applications in the exact same way that IBM developers have instrumented the operating system.

3. Interpret the collected data by printing out the trace with the PRTTRC command.

There will be trace information included automatically by the system in addition to the component-specific data you add by using the APIs.

Traces can be large so you want to ensure that you clean up after yourself. Once you have completed diagnostics with the trace data, be sure to delete the trace data with the DLTTRC command. You should also delete any spooled trace files when you are done with them as well.

Adding diagnostic information to an application can be time-consuming, but the improved diagnostic capabilities provided by having trace data can ultimately save you a lot of time when problems arise. IBM i has significant tracing capabilities built into the operating system and you can be sure that every developer thinks about how they are going to debug problems that may arise. And with the QTRC APIs, you can do the same thing with your application.

 

 

 

IBM i Job Trace

When you have a problem you need to resolve, how you approach doing problem determination depends upon the actual problem. IBM i has a very rich set of diagnostic tools for problem determination purposes.

In this week’s blog, I’ll talk about what is called the “job trace facility.”

Those of you familiar with i probably know that the Trace Job command (TRCJOB) was part of the original release of OS/400. In the V5R1 release, IBM released a new job trace facility called Start Trace (STRTRC), along with the related commands called End Trace (ENDTRC), Print Trace (PRTTRC) and Delete Trace (DLTTRC). In the V5R4 release, IBM changed the TRCJOB command to actually invoke the various xxxTRC commands under the covers. The data-collection capabilities of the trace started with STRTRC are far superior in terms of performance and functionality than those of the original TRCJOB capabilities.

What you might not know is some of the finer points of this trace facility:

• The ability to start a trace on a job that has not yet started.
  The use of a generic name in the JOB parameter (job name or job user) allows you to trace a group of jobs. Jobs that start after a trace is started that match the generic job name criteria will automatically be included in the trace.
• The ability to trace jobs within a specified subsystem.
  The subsystem element of the JOB parameter allows you to specify the name of the subsystem to be included (and this can also be a generic name). Specifying a subsystem name allows the job whose name matches the JOB name specification that runs in the specified subsystem to be traced. Jobs whose name matches the JOB name specification but do not run in the specified subsystem will not be included in the trace.
• Integrated watch support to automatically end the trace upon some condition.
  The “Watch” parameters allow you to specify watch criteria when you start a trace. If the watch criteria are met, the trace will be automatically ended. This is the same watch capability that I wrote about in the “Automate Monitoring with Watches” blog, but it is integrated with the Start Trace command.
• Component trace support to specify additional data for components within the operating system. The Trace Type (TRCTYPE) parameters control whether *FLOW trace (the program/procedure call flow) and *DATA trace (operating system data) is collected as part of the trace. In addition, you can also specify *TRCTYPE (Trace Type); *TRCTYPE allows for additional component trace types to be optionally included as part of the trace. This is useful if you are working with the IBM Support Center to collect diagnostic data for operating system problems, as many operating system components have added trace instrumentation to collect debug data via STRTRC. A quick review of the values allowed on the *TRCTYPE parameter will show that many different components have provided job trace instrumentation.

But did you know you can also instrument your own application and have your application trace data collection also controlled with the *TRCTYPE parameter? Next week I’ll write more about this.

• Join Trace support to automatically have additional jobs dynamically be added to the trace.
  The Join Trace (JOINTRC) parameter specifies whether jobs will set their trace characteristics in a related job or thread. This parameter is used when work may be passed off to a prestart job or a batch immediate job.
• You can have a job trace automatically started when the system is IPLed.
  With the “Restart after next IPL” (RESTRIPL) parameter, you can set up a trace that will be started when the system is IPLed. This is useful if you have jobs that run when the system is IPLed and you need to collect job trace information for them before you can actually sign on to the system.

For more information about IBM i diagnostic tools, refer to the IBM Redbooks publication “i5/OS Diagnostic Tools for System Administrators.” 

 

Work With Active Jobs

This week's blog was written by Mike Russell, who is a member of the IBM i work management team. Everyone knows about the Work with Active Jobs command, but Mike shares some of the lesser known features as well. Thanks, Mike!

This article describes the Work with Active Jobs command. Work with Active Jobs helps you see and manage what is running on your IBM i system. It shows a list of all the active jobs with their current status and basic performance information.

Basic Features

The job type, current user, priority, memory pool, status and number of threads can all be shown by pressing F11. Basic performance information shows the job’s processor usage, paging and interactive response time. Options can be entered to control the job or find more detailed information about a single job. F23 can be used to see all the options available and F24 to see all the function keys available.

For information that can be different for each thread in a job, the initial thread’s information is shown. The status shows whether the job is currently running or not. The status values ending in W show the job is waiting. The same value ending in A means it is waiting in the activity level.

Elapsed interval

Work with Active Jobs gathers current information for each active job when the command is run and each time the screen is refreshed. The current performance information is compared to information collected at an earlier start time. The difference between the current time and the start time is called the elapsed interval. The elapsed interval is used to calculate the CPU percent, number of interactions, average response time and auxiliary I/O operations for each job. The elapsed interval is shown at the top of the screen along with the percentage of CPU used by the system and number of active jobs. When the WRKACTJOB command is used for the first time in a job, the elapsed interval is zero and no performance data is shown. This is the start time of the elapsed interval. The F5=Refresh key causes the end time of the elapsed interval to be set to the current time, the start time remains the same. The F10=Restart key causes the end time of the elapsed interval to be set to the current time, with the start time set to the previous end time. In other words, F10 shows you what has happened since the last time you looked. F5 extends the elapsed interval. F13=Reset statistics can be used to set the elapsed time to zero and reset the display.

If you are looking for performance problems, you want to use F10 occasionally. With a long elapsed time the performance information becomes watered down, making it difficult to see what's currently happening.

Other Features

Sorting and limiting

It can be difficult to find what you are looking for in a long list of jobs. There are features in WRKACTJOB to help you find, sort and limit the number of jobs shown. F7=Find displays a pop-up to allow you to enter a search string and what column to search. F8=Repeat find will find the next instance. F18=Bottom and F17=Top will get you to the bottom or top of the list without scrolling.

By default the jobs are grouped under the subsystem they are running in. You can sort the jobs by the values in any column on the display by moving the cursor into the column and pressing F16=Resequence. The SEQ (sequence) parameter on the command can also be used to sort the jobs. And the F20=Subset pop-up also lets you change how the jobs are ordered. Sometimes sorting the jobs into a different order can make it easier to see which jobs are using the processor, work with all jobs in MSGW status, or monitor the paging of jobs running in a memory pool.

By default all active jobs are shown except those that are normally expected to be idle. Idle jobs include the suspended system request and group jobs, disconnected jobs, and prestart jobs that are waiting for a request. You can show the idle jobs by pressing F14=Include. There are several parameters on the command to let you work with a smaller list of jobs. The CPUPCTLMT and RSPLMT parameters let you only see jobs that are using the processor or having long response times. The SBS parameter lets you see only jobs in subsystems you are interested in. The JOB parameter lets you see only jobs matching a generic job name or system jobs or subsystems. The F20=Subset pop up lets you change which jobs are included on the display without exiting the display and entering the command again.

Option 9=Exclude can be used to remove jobs from the display that you're not interested in. This allows you to get the jobs you are interested in on the same screen so you don’t have to roll back and forth to compare their performance information. The jobs remain excluded until you use F13 to reset the display. F21=Nondisplay instructions/keys can be used to get more jobs shown at one time. The option and function key descriptions are removed. F21 again will change back to display the descriptions.

Automatic refresh

WRKACTJOB can be left running to monitor the system activity by pressing F19=Start automatic refresh. The screen will be refreshed every 5 minutes. You can change how often the screen is refreshed with the INTERVAL parameter. In release 7.1, the automatic refresh function was changed to only show activity for the latest interval. In previous releases, the elapsed time increased each interval to show activity for the entire period since automatic refresh was started, making the information less and less useful over time.

Setting the Function field

The function field usually shows a high-level command, menu or program used by the job. You can use the Change Current Job QWCCCJOB API to customize the function field in a long-running program to something more meaningful. USR- will precede the specified value on the display.

 

Subsystem Configuration for SQL Server Mode Jobs

SQL server mode jobs are the jobs named QSQSRVR that run in the QSYSWRK subsystems. One of the more common concerns that has been raised with QSQSRVR jobs is the difficulty to manage them, and in particular, the inability to configure the subsystem in which they run.

By default, the QSQSRVR jobs will run in QSYSWRK. IBM has made available PTFs on the 6.1 and 5.4 releases to allow you to control the subsystem in which these jobs run. You will need to create an environment variable to turn on the capability to run these jobs in other subsystems. The PTF solution is rather simple; the environment variable indicates that the QSQSRVR server mode jobs should run in the same subsystem as the requesting job.

It is also recommended that you add prestart job entries to the subsystem where the QSQSRVR jobs will be run. Prestart jobs allow the job to be available when the request is received, as well as allowing the job to be reused and are important for best performance. If the prestart job entry does not exist, a batch immediate job will be used, which has more overhead.

See the PTF cover letters for the detailed instructions on creating the environment variable and setting up the prestart job entries. 

• 6.1 – SI33949 
• 5.4 – SI33298 
  
  

In the 7.1 release, this capability is now part of the JDBC connection properties and the CLI connection attributes. You specify the server mode subsystem where the jobs should run. By default, the jobs will run in QSYSWRK, but you can specify a specific subsystem name, or you can specify *SAME, which indicates the QSQSRVR jobs will run in the same subsystem as the job making the request.

Like described above for the PTF solutions on 6.1 and 5.4, on 7.1 you should also ensure that a prestart job entry exists for the QSQSRVR job in the desired subsystem description. If the prestart job entry does not exist, the QSQSRVR job will run as a batch immediate job.

 

PTFs for Additional Work-Management Controls

The work-management team recently made available some changes with PTFs to help control the behavior of job creation for cases when a program may get in a loop submitting jobs. The change made with the PTF is to slow down the creation of new jobs when the system is approaching the limit for the maximum number of jobs (QMAXJOB system value). 

QMAXJOB defines how many jobs can be on the system and includes all jobs: jobs submitted to a job queue, active jobs and ended jobs with spooled output. When the system reaches the limit specified by QMAXJOB, no more jobs can be started and the system can become inoperable. 

There’s a warning message--CPI1468 “System job tables nearing capacity”—that’s sent to QSYSOPR when the system is nearing the limit of the maximum number of jobs. Support was added to customize when these messages are sent with the following PTFs:
•    6.1 - SI30171
•    5.4 - SI29585
•    This same function exists in 7.1 without additional PTFs

By default, the CPI1468 is sent when the job tables are 90-percent full. In some situations, that’s too late and you don’t have time to recover from the condition. Create a data area named QMAXJOBPCT in QSYS with the desired percentage value at which you want the message to be set. The PTF cover letter has more detail.

The new PTFs that slow down the creation of new jobs when the system is approaching the QMAXJOB limit are:
•    7.1 – SI40568 
•    6.1 – SI40569
•    5.4 – SI40571

With these PTFs, the system adds delays to starting new jobs when the number of jobs is at 90 percent of the QMAXJOB setting. Override this percentage by creating a data named QWTSBSDLY in QSYS; for example, you could have the threshold be set to 80 percent. There are no delays if the number of jobs in the system remains below that percentage.

The delays start at a half second, but are increased as the system approaches the QMAXJOB limit. A single delay isn’t noticeable, but if you have a job that’s looping starting jobs, these delays can add up and offer some protection to keep the system operational as you approach the QMAXJOB limit. You can still run out of job-table entries, but it will take longer and you’ll have a little more time to react to the situation.

In 2008, we wrote a Redpaper, “Best Practices for Managing IBM I Jobs and Output (and a few other special tips)." This Redpaper covers general recommendations for improved management of jobs and output. Provided in that Redpaper is a sample CL program source for a Submit Job (SMBJOB) exit program. The submit job exit program is the best way to control the runaway submission of many jobs.

Another set of PTFs were provided that change the default behavior of the system to briefly display the end-of-job display when an interactive job ends. Previously, when an interactive job was ended, the end-of-job display would remain until the user pressed enter or 30 seconds had passed.  With these PTFs, this delay is removed and the interactive jobs are ended immediately without delay due to the end-of-job message; the message is still sent, but is only seen briefly. This change is helpful for ENDSBS, for example. Prior to the PTFs, ending an interactive subsystem would generally not end the jobs right away because the end-of-job display would be displayed for 30 seconds for those jobs where the user didn't press enter. With these PTFs, the jobs are ended right away.

These  PTFs are:
•    7.1 – SI40867
•    6.1 – SI40866 

The PTF cover letters containing documentation on how to create a data area named QWTEOJDSP in QSYS to get the old behavior back.

i Can … Manage Workloads Better with IBM i 7.1

In today's announcement, there was a one-sentence statement that I thought would be good to expand upon:

IBM i 7.1 workload management enhancement enables clients to limit to the number of cores on which a selected workload can run concurrently.

So what is this, why would you want to use it and how do you get it?

IBM i's work management capabilities have long been a strength of the operating system. The capability to configure subsystems and memory pools to isolate workloads has been an important architectural design that allowed i to support multiple, different workloads concurrently on the same partition.

However, as the hardware has become more powerful, and partitions have become larger with more cores, additional controls for managing work within a partition have became more important. One thing the IBM i development team has been looking at for a long time was how to prevent one workload from consuming all of the CPU resources available. Although you could define subsystems and memory pools, it wasn't been very easy to limit the amount of CPU a workload used. The CPU time parameter is a class parameter, so all jobs that used the same routing entry or prestart job entry had the same configuration for the amount of CPU time they could use. Not many environments used the CPU time control simply because jobs generally need to run to completion, regardless of the amount of CPU time required; thus this really was not effective in controlling the utilization of processor resources.

This newly announced workload management enhancement provides a way to control the number of processor cores that can be concurrently used by a workload so you can now limit the processor resources consumed by a workload.

This enhancement allows you to define what are called “workload capping groups,” and for each workload capping group you can assign the number of cores that you want to allow that group to use. Then you associate a workload capping group with a subsystem so the jobs in that subsystem are limited to the number of cores defined by the workload capping group. There’s also the capability to associate a workload capping group with an individual job, if you want more granular control.

It might be good to provide an example of how workload capping can work for you...

One thing I often hear about is how ad-hoc queries can be disruptive to a production environment. You want your users to get at the data they require, but you don’t want that work to interfere with your mission-critical workloads. By using subsystem configuration for database server jobs, you could have these ad-hoc queries originated by users from their PCs run in their own subsystem and have this subsystem associated with a workload capping group. By doing this, these ad-hoc queries can be put into a ‘processing container’ to ensure they’re kept to a limited amount of system capacity. The subsystem that has been assigned to a workload capping group will have all jobs and their associated threads limited to the number of processing cores specified in the capping group. This same concept also applies to individual jobs; you can associate a workload capping group with a job. If the capping group has a processor core limit of one, then that job and all its threads will only be allowed to run on a single processing core. If this job is running on a multiple threaded core, multiple threads may be running for that designated jobs, but only a single core will be used at a time.

This feature provides the capability to allow mission-critical business applications the resources they need, while restricting other applications to fewer processor resources, and further strengthens IBM i's work management capabilities.

This support will be available with PTF SI39795 in September. There are actually several PTFs you need for the support, but this primary PTF will have the information on all the PTFs you need to install, as well as the technical documentation on how to use this powerful new function.