Sending a Spooled File via TCP When Working With Spooled Files

You can work with spooled files through a variety of command interfaces: Work with Spooled Files (WRKSPLF), Work with Output Queue (WRKOUTQ), and Work with Job (WKRJOB) option 4 (aka WRKJOB OPTION(*SPLF)). These interfaces have option 1=Send, which can be used to send spooled files to another system; this send option invokes Send Network Spooled File (SNDNETSPLF).

This was a very convenient way to send a spooled file to another system using SNADS when SNA was the networking protocol of choice. SNA, as well as SNADS, is not as widely used today. A few clients still use SNA and SNADS, but most have moved away from SNA to exclusively use TCP/IP and use the Send TCP Spooled File (SNDTCPSPLF) command to send spooled files via TCP/IP.

Last fall, a PTF for 7.1, SI50975, was delivered to provide the capability to modify the send option when working with spooled files. Users can now use SNDTCPSPLF rather than SNDNETSPLF. This function is not available on 6.1.

Note: The 7.1 Print PTF Group (SF99366) does not yet include this PTF, so even if you have installed the latest print group, you’ll still need to download and install SI50975.

To change the behavior to use SNDTCPSPLF rather than SNDNETSPLF, you need to add an environment variable. From the SI50975 PTF cover letter:

To change the "1=Send" option on the WRKSPLF, WRKOUTQ, and WRKJOB
OPTION(*SPLF) panels to use SNDTCPSPLF instead of SNDNETSPLF, add a new environment variable QIBM_SNDSPLF_OPT with a value of 'TCP'. The environment variable name and its value must be uppercase, and may be set at the *SYS level or *JOB, or both. Example:
> ADDENVVAR ENVVAR(QIBM_SNDSPLF_OPT) VALUE(TCP) LEVEL(*SYS)

After you have added the environment variable, when you select option 1 on the WRKSPLF command, you will now see the prompt for SNDTCPSPLF.

If you are using the GUI (either the Navigator for i Web console or the System i Navigator application), this PTF does not matter since the GUI already provides the ability to send spooled files by both TCP/IP and SNA:

IBM provided this PTF as an ease-of-use enhancement for working with spooled files via the command line interface. You can also extend WRKSPLF yourself by using the UIM Spooled File User-defined List Action (QIBM_QSP_SPLF_LSTACT) exit point, which allows you to define additional list actions on the following green-screen interfaces:

• • Work with Printer Output (WRKSPLF ASTLVL(*BASIC))
• • Work with Spooled Files (WRKSPLF ASTLVL(*INTERMED))
• • Work with Job Spooled Files (WRKJOB OPTION(*SPLF))
• • Work with Output Queue (WRKOUTQ output queue name)
• • Work with Spooled File Status (D P or WRKSPLF DSPFMT(*S36FMT))

There is an example program in QUSRTOOL/QATTCL.TSPUIML that uses QIBM_QSP_SPLF_LSTACT to add a 'C' option to the WRKSPLF, WRKOUTQ and WRKJOB panels to do a CPYSPLF, and a 'T' option to do SNDTCPSPLF. The biggest disadvantage of using this exit program support is the additional list actions do not show up in the "1=Send 2=Change 3=Hold 4=Delete..." options list at the top of the screen.

IBM Support also has a useful article, Adding Options to WRKSPLF, WRKOUTQ and WRKJOB to Invoke the CPYSPLF and SNDTCPSPLF Commands, on using the exit point, which is what you will need to do if you want this capability on the 6.1 release.

 

 

Schedule Verifications With Application Runtime Expert

IBM has made another set of announcements and among them is IBM i 7.1 Technology Refresh 7. As part of the TR7 enhancements, the ability to schedule verifications has been added to the Application Runtime Expert.

From the IBM i TR7 RFA:

“Scheduling flexibility enhancements have been made to IBM Application Runtime Expert.”

The Application Runtime Expert product (5733-ARE) has been enhanced to include the ability to schedule verifications based on a timeframe that fits your needs. From the Application Runtime Expert GUI interface, it is possible to schedule a verification to run at a specific time against a list of chosen systems. This provides additional flexibility and usability to help extend the usefulness of the product. This new support will be delivered in the latest HTTP PTF Group and will require that the 5733-ARE product is already installed. These new enhancements are available for both IBM i 6.1 and IBM i 7.1.

The Application Runtime Expert is easy to use and the capability to schedule verifications makes it even easier; you no longer have to remember to manually run verifications. You can set up groups of systems with the attributes you want to have validated and specify the scheduling frequency; once it is configured, the verifications will be done automatically for you. All you have to do is review the resulting reports from the verification.

When you go to the console to verify the systems you have selected, there is now a scheduling icon.  Click on the Schedule link to open the scheduling window where you can select when you want the verification to automatically run. You can schedule verifications on a daily, weekly or monthly schedule.

The scheduling support is part of the Application Runtime Expert HTTP server, so the IBMARE server must be running in order for the scheduled verification to occur.

As a reminder, the HTTP group PTFs are:

• 7.1 - SF99368

• 6.1 - SF99115

The Application Runtime Expert for i website contains a lot of useful information.

I’ve also written about the application runtime expert before in the following blogs:

Application Runtime Expert – Network Health Checker

The Story of the Application Runtime Expert

Finally, be sure to read the following blogs to learn about other content in this latest announcement

• You and i from Steve Will
• iSight from Alison Butterill
• Modern-i-zation from Tim Rowe
• DB2 for i from Mike Cain
• RPG Café Blog where Barbara Morris contributes about RPG topics.

 

More Changes for the Handling of QINACTITV

A little over a year ago, IBM changed the handling of the inactive job time-out (QINACTITV) system value to improve accuracy and performance. That change is described in the blog article IBM Improves the Handling of the Inactivity Timeout.

IBM has made some additional improvements to the handling of QINACTITV. This time, the changes address two questions:

1. What does it mean for an interactive job to be inactive? 
With the change, a job can use CPU time and still be considered inactive.

2. Which jobs can be disconnected and which jobs must be ended? 
With the change, a job can be disconnected even if a device name was not supplied when the session was created. What this means is that jobs for devices whose names start with QPADEV can be disconnected. This affects the Disconnect Job (DSCJOB) command processing and the inactivity timer processing, but it does not affect the handling of device errors.

The change is available only for the 7.1 release with PTF SI50502. This is a delayed PTF, so it requires an IPL to put it on or to take it off.

First, let's look at what it means for a job to be inactive.

• Is there I/O occurring?
  We first look at the session – the connection between the user and the system. A session is inactive if there is no input or output occurring. A session is active if the system continues to send data to the user even if the user never sends input to the system.
• Who is waiting?
  A session will be inactive while the system is working on a long-running command, because there is no I/O occurring. In order for the job to be considered inactive, the system must be waiting for input from the user. The Work with Active Jobs (WRKACTJOB) command shows a status of DSPW for a job that is waiting for input from a workstation.
• Is the job using CPU?
  Looking at whether the job uses CPU time can be both helpful and harmful. Let’s look at some examples and how the new PTF affects these examples.
  
  The job might do work that comes from sources other than the workstation. For example, the job may have a message queue allocated and a break message handling program defined. In this case, using CPU indicates that the job is doing work and is in some sense active. With the new PTF, running a break message handling program will no longer be a reason to consider the job active. The new code gives less weight to unpredictable, background activities.
  
  The job might do work that is just a part of being there and interacting with other jobs. The job uses CPU time, but there is no reason to consider the job to be active. For example, when some other job looks at the invocation stack of an interactive job, the interactive job does some of that work and is charged with the CPU time it uses. This might occur when you have the inactive job message queue (QINACTMSGQ) system value set to the name of a message queue. The program that handles the CPI1126 message might want to look at an invocation stack to decide if a job should be considered inactive. With the new PTF, looking at an invocation stack will no longer be a reason to consider the target job to be active.
  
  More and more things are being done in the background, often in secondary threads. Using JAVA code in an interactive job is one way to see CPU time used just for being there and interacting with the environment.

There is no perfect way to decide whether a job should be considered active. A program handling CPI1126 messages can decide what to do with a job that the system considers inactive, but it is not notified about a job that the system considers active. The PTF cover letter describes a way to use an environment variable to change how the system treats CPU time for determining inactivity, but it cannot solve all the potential problems. The session will look active when someone sends a message to the workstation message queue, even if the user for that workstation isn't there.

Now on to the question of which jobs can be disconnected.

In order for a Disconnect Job (DSCJOB) to make sense, there has to be the capability to reconnect to the job. In order to reconnect to the job, the same user must sign back on to the same device description.

For the case where DSCJOB is being done in response to a device error, DSCJOB is not allowed for sessions where no specific virtual device is requested and the system selects which device to use. The session is gone and it is very unlikely for that user to get the same device when the user next connects to the system or when the device is next used.

For inactivity, the session is still intact and connected to the same user's workstation. The DSCJOB command is used by programs that handle the CPI1126 message when the QINACTMSGQ system value names a message queue. That means the command should act the same way the inactivity timer code acts. Also, if a user wants to DSCJOB rather than SIGNOFF, this is probably a good thing to allow.

This PTF allows more jobs to be disconnected. If your system is set up with QINACTITV(*NONE) or QINACTMSGQ(*ENDJOB), this PTF will probably have very little effect on the workload. So, for discussion, we'll assume that jobs are being disconnected for inactivity.

There is a cost to start a new job, to end a job, to disconnect a job, or to reconnect to a job. Different applications will have different costs. While the overall amount of work is important, many systems are more strongly affected by how that work is distributed over time. Disconnected jobs continue to hold locks and use system resources. This is also true of inactive jobs.

1. When a user signs on and is reconnected to a disconnected job, the system avoids the cost of ending the old job and creating a new job. When a large number of users reconnect, the work is generally well spread out over time.
2. When a user does not reconnect before the time limit defined by the time interval before disconnected job ends (QDSCJOBITV) system value, the disconnected job is ended. By comparison to QINACTMSGQ(*ENDJOB), the system sees the extra work of a disconnect, but there is no work avoided. All jobs see the same QDSCJOBITV value, so the work is spread out the same way the work for QINACTITV is spread out.
3. When a user does not reconnect and the subsystem is ended before the QDSCJOBITV time limit, the work of ending the disconnected jobs gets done all at once during the ending of the subsystem. This can significantly increase the stress on the system. Jobs that are inactive and not disconnected are also ended during the ending of the subsystem.

The QDSCJOBITV system value should be set high enough to allow users enough time to reconnect to their disconnected jobs, but low enough that the disconnected jobs are likely to be ended before the subsystem ends. If users are not going to reconnect, the QINACTMSGQ should be set to *ENDJOB rather than *DSCJOB.

One of the common problems with QINACTITV occurs when a user returns to a session at the same time the system is checking for inactivity. The system does not know that the user is there until the system sees input on that session and the system only sees input when the user presses enter or a function key.

If QINACTMSGQ is set to *DSCJOB and the user is using a virtual device selected by the system (a QPADEVxxxx device), the job will now be disconnected rather than ended. The user can sign on and continue. The PTF should be very helpful, even though the handling of QINACTITV will never be perfect.

 

I’d like to thank Dan Tarara for writing this blog. Dan is a member of the IBM i Work Management development team. Thanks, Dan!

Job Level SQL Metrics in Collection Services

Late last year, IBM enabled new support to collect SQL metrics in Collection Services. I’ll be covering this new capability in two blog articles:

1. This blog will review the new metrics that are now collected and describe what you need to do to enable the collection of these additional metrics.
2. A subsequent blog will review the support added to the Performance Data Investigator to allow you to view and analyze these metrics.

Collection Services collects a wealth of information, but metrics related to SQL operations were limited.  The tools used for DB2 and SQL performance analysis are separate from the system performance tools; interfaces such as the DB2 Health Center, SQL Plan Cache and Index Advisor, among others, are what you need to use to understand database performance issues.

The most recent PTFs augment the tools already available for SQL performance analysis and there are now SQL metrics collected at the job level. This allows you to begin your performance analysis with Collection Services and determine if SQL operations are a contributing factor. You will probably still use the DB2-specific tools for digging into the details, but you now have insight into the SQL operations from Collection Services data.

This enhancement is only available on IBM i 7.1 (i.e., it is not available on 6.1) and requires that you have installed the latest DB2 Group PTF SF99701, as well as SI47594, which provides the Collection Services enablement. It’s also a good idea to ensure that you have the latest Performance Tools Group PTF, SF99145, installed as well.

Just installing the required PTFs is not sufficient. There are activation instructions you must take to enable the collection of these metrics documented in the SI47594 PTF cover letter. The instructions are rather lengthy but not difficult to follow. The Job Level SQL Metrics article on developerWorks also documents these activation instructions.

The reason for these special instructions has to do with the fact that the file, QAPMJOBMI, needs to be updated to a new level to support these additional metrics.  Specifically, an updated template file for QAPMJOBMI is shipped in QSYS with the additional fields. This template file will only be used when creating a new QAPMJOBMI file in libraries where it doesn't already exist. Existing collection libraries (for example, QPFRDATA) that contain a prior version of QAPMJOBMI will not be affected by the PTF.

Once you have the required PTFs installed and completed the necessary activation instructions, you will have SQL metrics in your job-level Collection Services data. The Performance Data Investigator can then be used to view and analyze these metrics. As I mentioned earlier, I’ll write more about the PDI support in a future blog. To be prepared to use this new capability, review the blog Navigator for i Enhancements and install the PTFs as documented in that blog.

The following are the new metrics now available for collection:

New Collection Services metrics for SQL:

The following SQL performance metrics are collected as part of the *JOBMI category.

1. SQL clock time (total time in SQ and below) per thread (microseconds)
   The amount of clock time (in microseconds) this thread has spent performing work done on behalf of an SQL operation.
2. SQL unscaled CPU per thread (microseconds)
   Thread unscaled SQL CPU time used. The amount of unscaled processor time (in microseconds) this thread has used performing work done on behalf of an SQL operation.
3. SQL scaled CPU per thread (microseconds)
   Thread scaled SQL CPU time used. The amount of scaled processor time (in microseconds) this thread has used performing work done on behalf of an SQL operation.
4. SQL synchronous database reads per thread
   Total number of physical synchronous database read operations done on behalf of an SQL operation.
5. SQL synchronous non-database reads per thread
   Total number of physical synchronous nondatabase read operations done on behalf of an SQL operation.
6. SQL synchronous database writes per thread
   Total number of physical synchronous database write operations done on behalf of an SQL operation.
7. SQL synchronous non-database writes per thread
   Total number of physical synchronous nondatabase write operations done on behalf of an SQL operation.
8. SQL asynchronous database reads per thread
   Total number of physical asynchronous database read operations done on behalf of an SQL operation.
9. SQL asynchronous non-database reads per thread
   Total number of physical asynchronous nondatabase read operations done on behalf of an SQL operation.
10. SQL asynchronous database writes per thread
    Total number of physical asynchronous database write operations done on behalf of an SQL operation.
11. SQL asynchronous non-database writes per thread
    Total number of physical asynchronous nondatabase write operations done on behalf of an SQL operation.
12. Number of high Level SQL Statement count per thread
    The number of high level SQL statements that executed during the Collection Services time interval. This count includes only initial invocation, independent SQL statements. It does not include dependent SQL statements invoked from within another SQL statement. This count also includes initial invocation, independent SQL statements which failed to execute successfully.

 

 

 

 

Integrate Web Services With DB2 for i 7.1 Part Two

This week's blog article was written by Nick Lawrence. Nick works for IBM in Rochester, Minn., on DB2 for IBM i. His responsibilities include full-text search for DB2 and SQL/XML. Thanks, Nick!

In part one of this two-part blog entry, the idea was introduced that DB2 for i could be integrated with a Web service, using the user-defined functions in the SYSTOOLS schema and the built-in SQL/XML support. Part one includes a number of links to information about the user-defined functions, and also to information on the built-in SQL/XML support. One important resource was a link to a whitepaper that provides the complete details for these functions.

The second part of this blog entry examines a specific scenario where the data from a Web service is converted into relational rows and columns.

Suppose that a delivery service stores its vehicles' fuel and maintenance costs in a DB2 for i database. In addition to the actual expenses incurred, the application's processing involves the weekly national average cost (per gallon) of fuel. In other words, an SQL query might reference the national average, or the average might be stored in the database for auditing purposes.

This problem has two parts to it.

1. The database must retrieve an XML document that contains the current week's average fuel prices from a Web service (this example uses a Web service located at http://www.fueleconomy.gov).
2. After retrieval, the XML document needs to be transformed into rows and columns.

Figure 1 is a diagram of this setup.

Figure 1 - Web service interaction with DB2 for i

Using the functions that have been provided in SYSTOOLS, we can use SQL to get the average fuel price for the current week as an XML document. This is accomplished using the HTTPGETBLOB function and is shown in Figure 2.

Figure 2 - Retrieve current fuel prices

VALUES
 XMLPARSE(DOCUMENT
   SYSTOOLS.HTTPGETBLOB(
     --- URL ---
     'http://www.fueleconomy.gov/ws/rest/fuelprices',

     --- HTTP request header ---
     '<httpHeader>
       <header name="Accept" value="application/xml"/>
      </httpHeader>'
   )
 )

In Figure 2, the URL identifies the resource that is to be obtained, and the HTTP request header indicates that the data should be returned in an XML format. The XML document that is returned from the Web service is shown in Figure 3.

Figure 3 - Returned XML document

<?xml version="1.0" encoding="UTF-8"?>
<fuelPrices>
  <cng>2.12</cng>
  <diesel>4.16</diesel>
  <e85>3.47</e85>
  <electric>0.12</electric>
  <lpg>2.56</lpg>
  <midgrade>3.89</midgrade>
  <premium>4.04</premium>
  <regular>3.75</regular>
</fuelPrices>

The XML document shown in Figure 3 is not useful in SQL unless SQL has a way of understanding the relationships and data defined by the document. Fortunately, the XML support in DB2 for i 7.1 includes the built-in XMLTABLE table function. Figure 4 shows a query that uses the XMLTABLE function to convert the XML document in Figure 3 into a result set (rows and columns).

Figure 4 - Current fuel prices as rows and column

SELECT fuel_prices.*
FROM
 XMLTABLE('fuelPrices' PASSING
  XMLPARSE(DOCUMENT
    SYSTOOLS.HTTPGETBLOB(
      --- URL ---
      'http://www.fueleconomy.gov/ws/rest/fuelprices',

      --- HTTP Request Header ---
      '<httpHeader>
        <header name="Accept" value="application/xml"/>
       </httpHeader>'
    )
   )

 COLUMNS
   "regular"  DOUBLE,
   "midgrade" DOUBLE,
   "premium"  DOUBLE,
   "diesel"   DOUBLE) fuel_prices;

Figure 5 shows the result set that was created by the query in Figure 4.

Figure 5 - Result set

 

The query in Figure 4 is an SQL query, meaning that it can be easily incorporated into other SQL statements or queries that need to reference the data in Figure 5. An example of how to join data from a Web service with relational data is included in the whitepaper.

Although the HTTP GET request might be the most commonly utilized HTTP method, the SQL UDFs can also invoke the POST, PUT and DELETE HTTP methods. The additional HTTP methods can be invaluable for causing a Web service to take some action in response to an event that happens in a DB2 for i database. Some examples of these HTTP methods are included in the whitepaper.

The HTTP functions really make it easier to access Web services, allowing a software architect more time to figure out the most effective way to use the service. The built-in XML support allows developers and architects to spend their time utilizing the information in XML documents, rather than writing code to compose and decompose XML.

 

 

 

New System SSL support

This blog was written by Tim Mullenbach.  Tim is an Advisory Software Engineer at IBM Rochester. He has been a part of the IBM i sockets/SSL development team since 1995 and leading the team for the past eight years. He has worked on designing, implementing, and testing various socket and System SSL enhancements for the V4R2 through V7R1 releases.

Hopefully you noticed the recent messaging for IBM i 7.1 TR6 which included the statement “Latest industry standards of Transport Layer Security version 1.2 (TLSv1.2) and Transport Layer Security version 1.1 (TLSv1.1) protocols implemented.”  

In my conversations with customers I find that many do not realize that TLS is the “new” name for SSL.  And by new, I mean for the past 14 years.  SSL stands for Secure Sockets Layer and was created by Netscape a long time ago.  TLS stands for Transport Layer Security and was created/designed/defined by the Internet Engineering Task Force (IETF) in 1999.  IETF standardized and improved the SSLv3 specification making TLSv1.0. While technically called TLSv1.0, many times the .0 was left off leaving TLSv1.  Implementers assumed the next version would be TLSv2.0 since SSLv2 preceded SSLv3.  No such luck – the next version was called TLSv1.1 with the one after that called TLSv1.2. 

Unfortunately the vast majority of people still call the entire concept SSL.  It can make for confusing conversations as people like me try to interpret what SSL means in any given context.  In all likelihood it is both SSLv3 and TLSv1.0 being described by the use of SSL.  A level of uncertainly always remains however due to a handful of applications that still only support the SSL protocol version known as SSLv3.  And for the few of us that call it all TLS instead of SSL, we lump SSLv3 under the term TLS.

At this point you might be asking, “Thanks for the name game and history lesson, but what does new support mean to me and my business?” 

IBM i and its precursors have supported TLSv1.0 (SSLv3 and SSLv2 too) at least back to V4R5 days.  As of Feb 2013 with the release of IBM i 7.1 TR6, we have support for TLSv1.1, defined by RFC 4346 and TLSv1.2, defined by RFC 5246.  Each successive version of the SSL/TLS protocol corrects issues found with the previous version and improves upon the state of the art.  The latest version available should be used to obtain the highest level of security available for TLS/SSL.

There is a catch though, nothing can be simple.  From 1999 to 2005 almost every application, work station, and intelligent device in the world had SSLv3/TLSv1.0 support built into it.  The vast majority of those devices still work, perform business critical transactions, and have zero ability to be enhanced with newer TLS/SSL protocols.  This significantly impacted and is still impacting the roll out of TLSv1.1.  With TLSv1.2 defined so close to TLSv1.1, TLSv1.1 became a lost protocol version.  TLSv1.2 adoption initially suffered for the same reasons.  The difference is that TLSv1.2 is more of a game changer than TLSv1.1.  Sure TLSv1.1 protects from additional attack vectors than the previous protocols did, but other workarounds existed to limp along without it. 

So what is the big deal about TLSv1.2?  Besides all of the incremental improvements, its big selling point is that it uses the SHA-2 hash algorithm instead of SHA-1 and/or MD5.  This is important because of NIST 800-131a.  You won't see SSL or TLS mentioned by name, yet when pieced together the recommendations lead to TLSv1.2.  NIST is playing the prediction game – how long will things secure today remain secure?  While you might have longer than the end of 2013 as predicted in that document, a wise security administrator would start transitioning to the new protocols and algorithms sooner rather than later.

The IBM i 7.1 TR6 enhancements make it easy enough to enable TLSv1.2 for your applications.  That is only half the story, the easy half at that.  Your communication partner also must support TLSv1.2 for it to be negotiated.  This ability varies greatly by application and the peer devices themselves.

For years client applications had no incentive to add TLSv1.2 support since the servers didn't support the protocol.  To make matters worse, a very small number of broken SSL implementations in the world fail when a client tries to negotiate TLSv1.2. 

It would be a good idea to enable TLSv1.2 for all of your IBM i 7.1 servers.  Then start work to enable TLSv1.2 in all of the client applications you have control over that connect to those servers.  It could take years, but once all of your clients support TLSv1.2, that should become the only protocol supported by the server.

The IBM i 7.1 client applications you use should also have TLSv1.2 enabled after verifying the server at least tolerates TLSv1.2 being requested.  Like with the server, once you know the server supports TLSv1.2, the older protocols should be disabled.

The unfortunate reality is that many applications must continue to support TLSv1.0 for years.  Some peer devices have no upgrade path to get new SSL support so you wait for the device to be replaced. 

Please read “How to use TLSv1.2 with System SSL on IBM i 7.1” on developerWorks for details on enabling TLSv1.2.

While overshadowed in the developerWorks article by TLSv1.2, the new application level cipher suite control is also very interesting.  For the first time ever, you can control the ciphers supported for the various IBM applications on an individual basis.  This means FTP, Telnet, and SMTP (client or server) can all use a different list of ciphers at the same time.  Gone are the days of taking a sledgehammer to the Secure Sockets Layer cipher specification list (QSSLCSL) system value in order to get an individual application to pass a security scan. 

This configuration option is available from the enhanced Digital Certificate Manager (DCM) application definition support.  To locate this support, select ‘Manage Applications’ and then ‘Update application definition’ from the left hand panel in DCM.  Once the application to update has been selected, the ‘Update Application Definition’ panel will display the new fields that can be updated.  Use these fields to control both the ciphers and protocols used by the application.  Refer to the panel help text for details on these configuration options as well as other additional System SSL properties now located on that panel.

Performance Data Investigator

A few weeks ago I wrote about the Performance Data Collectors in IBM i; you can collect many different kinds of performance data, but once you have the data, what is the best way to view and analyze that data? Use the Performance Data Investigator. If you are on 6.1 or 7.1 you already have it.

The Performance Data Investigator was introduced with the 6.1 release and is found in the Navigator for i web console; it is not available in System i Navigator (and it will never be in System i Navigator).

Within the Navigator browser console, you will find the Performance tasks in the left-hand navigation pane. When you select the Performance tasks, you will see the following options:

The Investigate Data option is how you access the Performance Data Investigator. When you take the Investigate Data option, you will see something like the following image:

The initial list of perspectives shows you the set of content packages that are available; a content package is simply a grouping of perspectives for the same type of performance data. The Collection Services content package contains all the graphs and tables for Collection Services data. Which content packages you see depends upon what options of the Performance Tools licensed program product are installed.

You will always see Collection Services and Health Indicators; the capability to view and analyze Collection Services data is part of the base operating system and no other product is necessary.

However, if you want to analyze Disk Watcher or Performance Explorer data, you will need to have the Performance Tools product, Manager feature installed. To analyze Job Watcher data, you need to have the Performance Tools product, Job Watcher feature installed. Below is a screen capture of the Performance Tools product with all the options installed:

On the main Investigate Data panel, open the type of performance data you want to analyze – for example, you can expand the Collection Services folder to show the following navigation:

You can see that the list of perspectives (which is simply a fancy name for a graph or table) is organized by the type of information you want to see. You can further expand the folders to find the perspective you want to view. Once you have identified the view you want to see, you can display it.

Note at the bottom of the screen you have the capability to select which performance data collection you want to display, so you can have various performance data collections in different libraries.

Once you have identified the graph or table you want to display along with the performance collection you want to use, you display the data. An example of what a graph looks like is below:

The Performance Data Investigator has a wealth of capabilities; for a more complete article on the Performance Data Investigator, please refer to the IBM i Performance Data Investigator article on the developerWorks site.

Finally, here is a list of prior blogs with information related to the Performance Data Investigator:

Investigate Performance Data

Display CPU Utilization for all Partitions

Tell You Why You Are Waiting

Understand Scaled CPU Time

Measure Disk Response Times

Customizing a Perspective in PDI

Find All My Performance Collections in One Place

Job Performance Information

Performance Data Collectors in IBM i

 

 

 

Multi-Threaded Job Table Cleanup

I'd like to thank Mike Russell for writing most of this blog article. Mike works on the IBM i Work Management team and was responsible for the changes described in this blog. Thanks, Mike!

 

Some time ago, I wrote about the job table and permanent job structures in IBM i Job Structures (January 11, 2011). That article reviewed the job table and also discussed compressing the job table. 

In the IBM i 7.1 release, IBM restructured the processing of the job table cleanup that at happens during an abnormal IPL as well as the processing that is done when job tables are compressed. The job table cleanup IPL step is also known as Work Control Block Table cleanup (C900 2C40) IPL step.

Prior to 7.1, the job table was processed by one thread during the IPL step; when the job table became large, it could take this one thread quite some time to process all the entries in the job table. In the 7.1 release, this single-threaded processing was changed to be multi-threaded. This change decreases the time the system is unavailable after an unexpected termination as well as when the job tables are compressed during an IPL.

This new news is that this function is now available for the 6.1 release with PTF SI48088.

The performance improvement is seen on systems with a large number of entries in the job table. You can use the Display Job Table (DSPJOBTBL) command to view the number of entries on your system. The Change IPL Attributes (CHGIPLA) command allows you to influence how much cleanup is done at IPL with the Check job tables (CHKJOBTBL) and Compress job tables (CPRJOBTBL) parameters. How the system powered down is the other influence on how much cleanup is done at IPL. You can check the history on your system to see if you have had any recent abnormal IPLs with the Display Log command: DSPLOG PERIOD((*AVAIL *BEGIN)) MSGID(CPI0990 CPI091D)

There is also a 7.1 release PTF (SI48454) that includes a minor performance improvement for job table cleanup and a correction for the IPL status estimated time remaining:

 

You should not have to run job table cleanup or compression very often, but it is one of those things that when it is necessary, time is most likely important as well. This was an important fix for reducing IPL time in the 7.1 release and it's now available on 6.1.

Work With Job and Select a Thread …. to Print Thread Stacks

In the 7.1 release, the Work with Job (WRKJOB) command was enhanced; you can determine this by going to the CL Command Finder and take the option to display “All changed commands.” The Information Center will show you a list of changed commands and you can read the documentation on the command. Of course, there's a slight problem in that commands are not change-flagged, so you cannot actually see what changed. You need to compare the command documentation with that of the prior release to figure out exactly what has changed.

By comparing the 6.1 version of WRKJOB with the 7.1 version, you will discover that IBM added a new parameter, Select Thread (SLTTHD) to the command. Upon first glance, it might appear that this allows you to simply work with a specific thread for that job; maybe assuming it's a “fast path” way to do a WRKJOB followed by Option 20 (work with threads).

But that is not the case.

A blog I wrote nearly three years ago, Display Call Stacks, discussed the various ways you can access call stacks, as well as some enhancements that were made to the call stack itself in the 6.1 release.

What do these two things have in common?

The Select Thread parameter provides an interface to print the thread stacks for the initial thread of a process (the default), selected threads in a process, or all threads in a process. Prior to this enhancement, the only way to access all thread stacks within a job was to do it interactively; if you specified OUTPUT(*PRINT), it would just print the call stack of the initial thread. A better way was needed to capture integrated thread stacks (which include the IBM Technology for Java stacks and PASE stacks) for diagnostic purposes, which the addition of the SLTTHD parameter on the WRKJOB command provided.

Here's how you would print the call stacks for all threads within a specified job:

WRKJOB JOB(######/user/jobname) OUTPUT(*PRINT) OPTION(*PGMSTK) SLTTHD(*ALL)

And here's how you could select one or more threads from a list; if you select multiple threads from the list presented, each thread's call stack will be printed in the single spooled file that is generated:

WRKJOB JOB(######/user/jobname) OUTPUT(*PRINT) OPTION(*PGMSTK) SLTTHD(*SELECT)

This enhancement is sneaky is because the new parameter, SLTTHD, was hidden from the WRKJOB command prompt – you need to press F9 in order to see it, and because it requires the specific combination of OUTPUT(*PRINT) and OPTION(*PGMSTK) for the option to be valid.

If you have a need to generate spooled output of thread stacks for a job, this handy command invocation will do it for you.

Delete That Object

Did you know that in the IBM i 7.1 release, a new command was added to delete an object? I'm sure you can easily figure out what the name of the command is. Did you also know that in the 6.1 release, the API to delete an object was added?

The Delete Object (DLTOBJ) command (or the Delete Object API) is similar to the other object commands, for example Dump Object or Move Object, where you specify the name of the object along with the object type. Not all object types are supported by the generic Delete Object command or API, but many of them are. Some of the DLTxxx commands have additional parameters; if you use the generalized DLTOBJ interface to delete these objects, the system will use the default values that could have been specified on the more specific DLTxxx command (these default values are documented in the Delete Object API documentation). The Delete Object API also documents the object types that are supported by these interfaces.

There are a few advantages that you have when using the DLTOBJ command:

• You can delete multiple libraries (object type of *LIB) with one DLTOBJ command by specifying a generic name. Be careful!
• You can specify *ALL for the object name and have an easy way to delete all of the objects of a specific type from a library. Be careful!
• You can delete an object from a specific ASP or a group of ASPs by using the ASPDEV parameter.
  
  

And here's a little bit of trivia for you … the request for providing a generic delete command originated back in 1990; it may take IBM a little while, but good ideas are not forgotten, they sometimes just take some time to work their way to the top of the priority list.