Working With Your Spooled Output

In recent blog articles I wrote about Job Log Pending and the Job Log Output attribute that can be used to prevent the generation of spooled job logs. In addition, in a prior blog, I also wrote how to Detach Spooled Files from a job.

These all can be very helpful features, but one common concern I hear when I discuss these configuration options with clients is how to find their spooled files and job logs. Taking advantage of these features means you will likely need to change your approach to finding spooled files, but the capability to do so has been built into the operating system.

Everyone who has been using IBM i's character-based user interface (the “green screen”) is familiar with the Work with Spooled Files (WRKSPLF) command. This command has been part of the operating system “forever.” However, when a function is part of the base operating system, we don't always realize enhancements have been added because IBM always does it in a way that is upwardly compatible. That is, you can keep on using WRKSPLF just as you always have and not even realize that new features are available. And IBM does not change-flag enhancements to CL commands in the Information Center, which makes it all the harder to know what has changed. And the IBMers that talk about the latest and greatest additions to the operating system generally don't get too excited about incremental enhancements to that “old stuff”.... So while WRKSPLF command has been around “forever” and although the major enhancements were made to it in 5.4 (a “long” time ago), I've discovered that many people are still not aware of these changes.

As a quick exercise, let's look at the command parameters between V5R2 and 7.1.

Here is the Work with Spooled Files Command syntax diagram from the V5R2 Information Center (remember those?!)

>>-WRKSPLF------------------------------------------------------>

>--+-----------------------------------------------------------------+-->

   |          .-*CURRENT--.  .-*ALL--------.  .-*ALL------.  .-*ALL------.  .-*ALL-------.    |

   '-SELECT(--+-*ALL------+--+-*OUTQ-------+--+-*STD------+--+-user-data-+--+-ASP-number-+--)-'

              '-user-name-'  '-device-name-'  '-form-type-'

 

                               (1)

>--+-------------------------+---------------------------------->

   |          .-*INLENV-.    |

   '-DSPFMT(--+-*S36FMT-+--)-'

              '-*NORMAL-'

 

>--+---------------------------+--+------------------------+---><

   |          .-*PRV------.    |  |          .-*------.    |

   '-ASTLVL(--+-*USRPRF---+--)-'  '-OUTPUT(--+-*PRINT-+--)-'

              +-*BASIC----+

              '-*INTERMED-'

Below are the Work with Spooled Files Command parameters from the 7.1 Information Center, but I've changed the Notes column to identify changes to the WRKSPLF command. You'll observe that major changes were done in the 5.4 release.

 

 

Keyword	Description	Choices	Notes
SELECT	Select files for	Element list
	Element 1: User	Generic name, name, *CURRENT, *ALL	Generic name support added in 5.4
	Element 2: Print device	Name, *ALL, *OUTQ
	Element 3: Form type	Character value, *ALL, *STD
	Element 4: User data	Character value, *ALL	Generic support added in 5.4
	Element 5: ASP	1-32, *ALL, *ASPDEV	*ASPDEV added, extended to 32 in V5R3 for IASP support
	Element 6: Spooled file	Generic name, name, *ALL	Added in 5.4
JOB	Job name	Single values: *ALL Other values: Qualified job name	Added in 5.4
	Qualifier 1: Job name	Generic name, name
	Qualifier 2: User	Generic name, name, *ALL
	Qualifier 3: Number	000000-999999, *ALL
PERIOD	Time period	Element list	Added in 5.4
	Element 1: Start time and date	Element list
	Element 1: Beginning time	Time, *AVAIL
	Element 2: Beginning date	Date, *BEGIN
	Element 2: End time and date	Element list
	Element 1: Ending time	Time, *AVAIL
	Element 2: Ending date	Date, *END
ASPDEV	ASP device	Name, *, *SYSBAS, *CURASPGRP	Added in V5R3 for IASP Support
OUTPUT	Output	*, *PRINT
DSPFMT	Display format	*INLENV, *S36FMT, *NORMAL
ASTLVL	Assistance level	*PRV, *USRPRF, *BASIC, *INTERMED

How can these additional parameters help you?

You can use the WRKSPLF command to list spooled files by their file type – e.g., on the command prompt if you type QPJOB* for the Spooled File element on the SELECT parameter, you will display all the spooled files that begin with QPJOB* for your user profile (i.e., all your spooled job logs).

You can use the WRKSPLF command with the JOB parameter to work with the spooled files for a specific job or a generic job specification. For example, if you are detaching spooled files from their jobs, you can use the JOB parameter on the WRKSPLF command to locate the spooled files, even through the job is no longer in the system. Note the JOB parameter requires the USER element of the SELECT parameter to be *ALL.

You can use the WRKSPLF command with the PERIOD parameter to find spooled files from a specific timeframe. You can combine the JOB and PERIOD parameters to further refine the search criteria on which spooled files you are looking for.

And remember, too, that the Work with Job Logs (WRKJOBLOG) command was also added in the 5.4 release to make finding and working with spooled and pending job logs easier.

 

 

 

Job Log Output

Last week I wrote about Job Log Pending. With job log pending, you do not have to write job logs to spooled files.

At the end of that blog, I mentioned how you can use the Log Output (QLOGOUTPUT) system value or job attribute to take advantage of job log pending. This week, I'll explain the values available for “Log output” and give some general recommendations.

There are three options for the log output system value: 

• *JOBEND – the job log is written to a spooled file when the job is ended. This is the default value and is compatible with prior releases and how job logs were generated.
  
  
• *JOBLOGSVR – when the job ends, the job log remains in a pending status until the job log is written to a spooled file by the job log server.
  
  
• *PND – the job log is not written to a spooled file, but remains in a pending status.

While the shipped value for log output is *JOBEND, IBM recommends that you use the job log server. Using the job log server has a major advantage - the generation of the spooled job logs is handled in a more controlled manner. Remember the scenarios I described in last week's blog where you could potentially have hundreds or thousands of jobs writing job logs at the same time? That is a significant drain on system resources. By having the job log server generate the spooled job logs, the system controls the amount of work that is done at one point in time; if there are a lot of job logs to be written, the job logs will remain in a pending state until the job log server can write the job log to the spooled file.

But do you really need all those job logs written to spooled files? If you don't, you should think about using the pending attribute. Job logs that are in pending status will never have their job log written to a spooled file unless you take explicit action to do so. The system's automatic cleanup function will removing pending job logs in the same manner as it removes spooled job logs. If you need to display or print a pending job log, you can use one of the following methods: 

• Use the Display Job Log (DSPJOBLOG) command and direct the output as required.
• Use the WRKJOB or WRKUSRJOB command on ended jobs with pending job logs, similar to how you would work with an active job.
• Use the Work with Job Logs (WRKJOBLOG) command.
• Use the Change Job (CHGJOB) command to change the Log Output value from *PND to *JOBEND. This will write the job log to a spooled file and move the job to output queue status.

 

Job Log Pending

Job log pending has long been a possible state for a job log. When a job has a pending job log, the job has ended but the job log has not yet been written to an spooled file. In older releases of IBM i, this could occur when you powered down the system – the job logs would be saved in a pending state and the job logs would be written to spool files when the system was powered back up.

Every job on the system has a set of job structures, as I described in the IBM i Job Structures blog earlier this year. Job log messages are stored in an internal structure that is called the “job message queue.” The “traditional” behavior is for job log messages to be written a spooled file when the job ends.

However, think about certain scenarios: You end a subsystem that ends many jobs. You have a network failure that causes all Telnet sessions to be terminated. You have a runaway job in a loop that creates an unexpected number of jobs that do a small amount of work and then end. Or other situations where a large number of jobs are ended at about the same time.

In each of these scenarios, you can potentially have hundreds or even thousands of jobs that will all end at about the same time. If all of these jobs that are ending are also writing their job logs to spooled files, you can imagine the additional workload that would put on the system - writing job logs is a fairly expensive operation. Do you need to have all those job logs written? Do you have a need to save those job logs at all?

In the 5.4 release, IBM made job log pending a much more valuable and available option to use for job logs.

Rather than writing job logs to spooled files, job log pending allows you to keep the job logs in the internal form that is used when a job is active. No additional system resources are required to keep a job log in pending form (other than the job structure used to store information about the ended job).

Job log pending is controlled by the Job log output system value (QLOGOUTPUT) or the Job log output attribute on job descriptions.

There is much more to be written about job log pending, which I will do in subsequent blogs.

 

 

 

Clone an Integrated Windows Server

In a Windows server environment, it's normal to have a need to quickly create another Windows server to be used for a new application or even as a backup system to test the latest service pack without interfering with your production server. The problem though is creating multiple servers from scratch gets tedious when needing to install additional device drivers, service packs and common applications after the operating system is installed. But with a recent new-function PTF, support for cloning integrated Windows servers is possible.

An easy-to-use GUI Wizard duplicates an existing iSCSI based Windows 2003 or Windows 2008 server by copying the network server description (NWSD), network server storage (NWSSTG), virtual Ethernet line descriptions, and then makes them ready for that first activation. This support was added to IBM i 7.1 in December 2010, and IBM i 6.1 in May 2011.

Although it's possible to duplicate a single server for test purposes, the primary reason cloning support was added was for rapid server deployment where multiple blades or System x servers are designated for Windows server installations. The first task in creating servers based on an existing server is to ensure your base server is ready to be cloned. This means your server is fully installed with a supported Windows server operating system, latest service packs and any applications that are needed on the clone servers. Once the base server is ready to be cloned, a Windows tool named “SYSPREP” is used to remove server specific information such as log files and license key information and prepares it for a first-time run welcome experience. Information on how to access SYSPREP on Windows 2003 or Windows 2008 is available from the following link: http://publib.boulder.ibm.com/infocenter/toolsctr/v1r0/index.jsp?topic=%2Fdpsccm%2Fdpsccm_t_running_sysprep.html

The SYSPREP command suggested for the integrated server environment should be run as follows to prepare the cloned server to run the mini out of box experience when first started.

• Windows 2003:  SYSPREP -reseal -mini
• Windows 2008:  SYSPREP /generalize /oobe

After SYSPREP has been run, shut down the server and start making as many copies as needed. To start the server cloning wizard, use the Web GUI, select Create Server task and select the Base on an existing object option and specify the NWSD name of the server to be cloned as well as the name of the new server. Continue through the wizard to copy the storage, specify a compatible remote system, and then click Finish on the Summary page. The progress bar will show that status of the creation process, which will take some time duplicating large storage spaces.

The clone server is created with the same properties as the base server with one exception. The virtual Ethernet point-to-point connection for the clone server will have a different TCP/IP interface on IBM i and will need to be manually set on Windows once the server is activated to be in the same subnet.

For detailed instructions on the entire cloning process and using the integrated server cloning wizard, see the section titled “Server cloning road map and checklist” in the IBM i iSCSI Solutions Guide. 

The required PTFs for this support are as follows.

• IBM I 6.1 PTF SI43599 and WebNav GUI service pack PTFs (WSP8)
  or IBM I integration with BladeCenter and System x Group PTF SF99357 Level 20.
• IBM I 7.1 PTFs SI41049, SI41817 and WebNav GUI service pack PTFs (WSP3)
  or IBM I integration with BladeCenter and System x Group PTF SF99369 Level 4.

I'd like to thank Thom Haze from the Integrated Server Development team for writing this blog article.

 

 

 

The Secret History Log Enhancements

This week, I thought I’d write about some enhancements that have been made to the history log in recent releases. While these changes have been available for some time, they are well hidden because they were generally small changes on existing interfaces. The history log, as we all know, can be very valuable for understanding what has happened on the system, but it can also be a little overwhelming to deal with because of the potential volume of messages that get logged.

 

History Log Size - *DAILY

In V5R4, we made a little enhancement to the History Log Size (QHSTLOGSIZ) system value. A special value of *DAILY was added. This option allows for a new version of the history log to be created each day, rather than based upon size. If you have a busy system and reach the maximum number of records in a day, you will get a new log version when that occurs, so on these busy systems, you may have more than one log version a day even if you specify *DAILY for the system value.

I'll make an observation about the values allowed for the history log size. IBM i allows you to specify a size of 1. I have no idea why this is, but it’s silly! In fact, if you do specify 1 for the QHSTLOGSIZ system value, the system actually uses 10 – which is still ridiculously small. The default size is 5,000, and that is also too small for today's environments! I like the capability to have history logs created on a daily basis and that approach eliminates the need to play around with log size values.

 

Open List of History Log Messages API

In the 6.1 release, we added the Open List of History Log Messages (QMHOLHST) API. The API provides a programmatic interface to access history log messages. Prior to the 6.1 development of this API, your choices for accessing history log messages were either through the Display Log (DSPLOG) interface or directly by reading the history log files.

There are several advantages to using the API:

• It’s a formal programming interface; prior to the introduction of the API, you had to read from the files directly to programmatically access the history log messages.
• You can retrieve many messages with one invocation of the API, as opposed to reading the files record by record
• Using the API, you can filter on message severity or you can filter on the message type; in addition you can specify with the filters to either include or omit the filtered data
• You can also identify specific message IDs that should be retrieved or omitted
• This API can be used to get messages from the QHST message queue into the history log files. Sometimes you need to do a DSPLOG twice in order to see the latest messages, but that is not the case when using the API.

 

Watches

I wrote a blog on watches some time ago. Using watches you can watch for messages that are sent to the history log. But be careful! Do not watch for common messages (such as job started/job ended messages) since those will be matched frequently. If you do need to watch for a commonly sent message, qualify it with a comparison data so the match criteria are more selective.

If you have a need for real-time notification of messages sent to the history log, using watches is more effective than using the API or the DSPLOG command. Note that while Management Central has message monitors, you cannot monitor the history log messages with these message monitors; rather, you use file monitors to monitor the history log through Management Central. However, file monitors are more resource intensive than watches.

 

Print Second Level Message Text

Way back in V5R3, support was added to the Display Log (DSPLOG) command to allow you to print the second level text of messages from the history log. This was done by extending the keywords on the Output (OUTPUT) parameter to support *PRTSECLVL.

 

 

 

 

Avoid Locking Libraries in the Library List

There is a system value, Library Locking Level (QLIBLCKLVL), that you can use to control whether libraries that are in a job's library search list are locked. This system value is not new – it has been around since the V5R1 release – but it seems that it’s not very well known as I've talked with several people who’ve never heard of it.

The Library Locking Level system value has the default value of '1', which indicates that the libraries in the library search list are to be locked in each job. Each library in a job's library search list is locked when the job is started and the locks are released when the job ends.

As the number of active jobs on a system grows, the number of locks just for libraries can get very large. Consider an example where there are 1,500 user jobs and the library search list has 10 libraries – that's 15,000 locks held on the system just to protect those 10 libraries. This is not an efficient use of the system's locking table. This affects the performance of job initiation and termination when the locks are allocated and released; it also has an effect on overall system performance as there is a single lock table and additional overhead as this table grows very large.

The “Best Practices for Managing IBM i Jobs and Output” Redpaper documents IBM's recommendation of setting this system value to 0 to not lock the libraries in the library list.

While IBM recommends setting this system value to 0, to not lock the libraries, it could not make that value the default value because, without additional changes, setting this system value to 0 would allow a library to be deleted or renamed even when the library is in the library search list.

If you change the QLIBLCKLVL system value to 0, you need to take an additional step to prevent the libraries from being deleted or renamed—you just need one job to allocate the libraries you want to protect. This job can be set up as an autostart job that runs in the controlling subsystem; it would simply do an Allocate Object (ALCOBJ) on each library that needs to be protected, then loop on a Delay Job (DLYJOB) command. This job would remain active until the controlling subsystem is ended or the job is explicitly ended.

By using this strategy, you can protect the libraries from being deleted or renamed and you can greatly reduce the number of locks on the system.

One final point - the QSYSARB system job locks the libraries that are defined in the system library list (QSYSLIBL) and the user library list (QUSRLIBL) system values. IBM has a support article on this. 

 

 

 

Automated Problem Reporting With Service Monitor

IBM i (and its predecessors) has had the capability to automatically identify and report software problems to IBM for many releases. This was first introduced as the “Software error logging” (QSFWERRLOG) system value.

The capability to automatically identify problems when they occur is something we call First Failure Data Collection (FFDC). The intent is the first time a problem occurs, the data necessary for problem determination is automatically collected. That data, for operating system problems, can then be sent to IBM for diagnostic purposes. The lofty goal is to identify and resolve problems the first time they happen without ever having to recreate the problem.

This is an admirable goal, but in reality is very difficult to achieve. The challenge is that defects occur in unknown and unexpected places –– simply because defects are unexpected failures and programmers don't intend for unexpected failures to occur. As such, the capability to programmatically identify what data should be collected for a potential software bug is inherently flawed as it requires the ability to predict knowledge about what the problem may be in order to program the data collection routines.

In the 5.4 release, we decided to take a new approach to our FFDC support; key to that new approach is the recognition that one can never predict where a problem may occur. As such, there needed to be a dynamic way to modify the parameters around the identification of potential problems and the data that is collected for those problems –– this new support for software problem reporting was called Service Monitor. With the introduction of Service Monitor in the 5.4 release, we moved to a design that is dependent upon a “policy” that identifies potential problems along with the data that should be collected for those problems; this policy can be updated dynamically. Service Monitor is automatically started and supported with the *LOG option of the QSFWERRLOG system value.

While the Service Monitor has existed for a few releases now, it seems that it is a relatively unknown feature. The biggest reason is probably due to the fact that there’s no high-level overview of the Service Monitor function within the Information Center. Bits and pieces of information can be found by reviewing APIs, but there’s no summarization of the function and its capabilities.

Service Monitor, then, is a policy-based software function that is used to automatically identify problems that occur and to take the defined actions defined by the policy when the problem occurs. It’s primarily used for problems within the operating system and licensed internal code.

In the 5.4 release, customers primarily noticed the presence of the Service Monitor function by the many QSRVMONxxxx jobs that run in the QUSRWRK subsystem; a job started for each potential problem that could be reported to IBM. A common question I've heard is “What are all those jobs for?” In the 6.1 release, the design was changed to use prestart jobs rather than individual jobs, so the number of jobs has been reduced, but the Service Monitor function remains.

The policy file that is used by Service Monitor is maintained by the IBM Support Center in Rochester. As experience is gained with known problems, or when new problems are encountered, the policy file can be updated. The latest version of the policy file is downloaded to your system when you connect to IBM using Electronic Service Agent.

The policy file identifies the symptom of the problem, which can be a message, a licensed internal code (LIC) log (also known as a Vlog), or a product activity log (PAL) entry. The policy also identifies the action that is taken when that symptom occurs –– that action could be to collect diagnostic data to send to IBM, download a PTF that corrects the issue or some other action.

Several months ago I wrote a blog about Watches. Service Monitor uses watches as the underlying mechanism to implement the automated notification mechanism for problems that can be detected. Service Monitor policies identify the conditions to watch for –– which can be messages, LIC logs and problem activity logs (PAL entries) –– and thus Service Monitor sets up a watch for each item in the policy file. When the watch condition is matched, Service Monitor has an exit program that gets invoked and uses the policy definition as the way to identify the actions that should be taken when the watch occurs.

If you use the WRKWCH command to look at the *SRVMON watches, you will find many active watches (unless you have changed the QSFWERRLOG system value to *NOLOG). By displaying individual watch entries, you can see the kinds of things that service monitor watches for –– messages, LIC logs or PALs. For example, on a 6.1 system, if you look at the SRVMON0003 watch, you can see that it’s monitoring for CPF1101 sent to the QSYSOPR message queue. CPF1101 is “Subsystem &1 had a function check.” You can imagine that IBM probably wants to know if a subsystem takes a function check and has some basic diagnostic information that should be reviewed if this situation occurs. When this message occurs, the policy indicates that the job log should be collected and the problem reported to IBM.

Most of the policies used by Service Monitor are for LIC logs. Since LIC logs are commonly used to log diagnostic information for problems detected by the Licensed Internal Code, LIC logs are something easy to monitor for and then to automatically send the log data to IBM for review.

 

How to Use the Integrated Web Application Server

This week’s blog was written by Mark Schroeder. Mark is the team leader of the Web Integration team and architect of the Application Runtime Expert product. He has worked in Java and Web technologies for IBM i for the past 10 years.

Are you looking for a simple way to get started with Web applications? Are you overwhelmed by the complexity of creating an application server, configuring it, and finally trying to deploy your application into the server? Do you need a simple “on ramp” to the Web? If your answer to any of those questions is yes, then the IBM integrated Web application server (IAS) may be just what you need.

IAS is a lightweight, Java software-based application server that’s integrated into the IBM i operating system. IAS provides an Open Services Gateway initiative (OSGi)-based Web container where JSP and servlet-based Web applications run. Web applications running in IAS can access IBM i DB2 databases using JDBC, with support for both the native and IBM Toolbox for Java JDBC drivers.  IAS runs on the IBM Technology for Java JVM, which--combined with the modular OSGi Web container--means the server starts very fast and has a small memory footprint. The IAS server is a reliable place to run both your development and production Web environments and, since it shares some common technology with IBM’s WebSphere Application Server, once your Web application gets to the point where you need additional function and high-end performance, you can easily move it to a WebSphere environment.

But what really sets IAS apart from other application servers is how simple it is to use: creating a server and deploying Web applications is very simple. And because IAS is integrated into the IBM i operating system, there are no separate install or setup steps to get it – you already have it. Maintaining IAS with the latest fixes and updates is also simple; all updates are made via PTFs. There are no additional scripts or installers required; just load and apply the PTFs and all IAS servers are updated to the latest fix levels.

So how can you start using IAS? The easiest way to get started is by using the IBM Web Administration for i. This GUI interface makes creating and managing IAS servers easy. It provides intuitive wizards for performing tasks such as creating a new IAS server or deploying a Web application into an existing server.  Additionally, the IBM Web Administration for i provides the capability to manage many aspects of an IAS server, all from a single, easy-to-use GUI interface.

The IBM Web Administration for i is part of the Admin server that runs on every IBM i system. It can be accessed via the following URL: http://hostname:2001/HTTPAdmin.The "hostname" part of the URL is meant to be substituted with the actual host name of a customer's IBM i system.

To give you an idea of just how simple it is to start using IAS, let’s walk through an example of how to create a new IAS server. Once you have accessed the IBM Web Administration for i GUI, click on the Create Application Server link in the Common Tasks and Wizards list in the GUI’s navigation bar:

This will launch the Create Application Server wizard. The first step provides an introduction to the wizard. After clicking the Next button, you’re asked to select an application server type. To create an IAS server, select the IBM integrated Web application server for i server type:

Depending on the IBM i release, the IAS server version choice will be either V8.1 or V7.1. At this point, all that’s left between you and a fully operational IAS server are six simple steps.

Since the wizard provides very detailed information during each step, we won’t cover the entire create server wizard in detail. But here’s a brief summary of what information is required at each step in the wizard:
    Step 1: Provide a server name and description.
    Step 2: Specify the port range for the server.
    Step 3: Provide HTTP server information (name, description and IP address to listen on). The HTTP server will be the front-end Web server for the IAS application server.
    Step 4: Specify the user ID that the IAS server will run under.
    Step 5: Informational step that describes the Eclipse Help application, which is installed in every IAS server.
    Step 6: Summary of the information provided in the previous five steps.

And that’s it. Upon completion of the wizard, the IBM Web Administration for i creates and configures the IAS server and, once completed, automatically starts it. Typically, it takes a couple of minutes to create, configure and start the new server. Once the IAS server has been created, it can be managed using the IBM Web Administration for i GUI. 

Deploying applications into an IAS server is just as easy as creating a server; a wizard guides you through a few steps and, once completed, the application is deployed and automatically started for you. To deploy your application, all you need is to have it packaged into a Web Archive (WAR) file. The GUI interface will take the WAR file and convert it into the format required by IAS; this is done automatically by the GUI using a built-in conversion utility. Once the conversion is complete, the application is deployed and started. Once started, the application is ready for use.

As previously discussed, IAS is updated via PTFs. The best way to ensure you have the latest features and fixes for IAS is to apply the latest HTTP PTF group. The HTTP PTF group number for each IBM i release is:
•    IBM i 5.4: SF99114
•    IBM i 6.1: SF99115
•    IBM i 7.1: SF99368

IAS is a fast, reliable and simple way to deploy and run Web applications. The simplicity of IAS makes it an excellent on ramp to the Web by hiding much of the complexity often associated with creating, configuring and maintaining a Web application server. For more information about IAS, including a complete tutorial on creating, managing and deploying applications into an IAS server, visit the IAS product website.

i Can … Display the Status of Your IOA Cache Batteries

I assume most of you are familiar with the I/O adapters that have cache batteries on them and the importance of monitoring the status of those batteries to ensure they are replaced before they fail. In case you’re not: the IOA has a cache to improve performance; the application doesn’t have to wait for the data to be written to the disk. The cache is powered by a battery in the IOA to ensure the cache data is preserved even if the IOA is powered off or reset. However, if the cache battery is failing, then the data is written directly to disk and the cache isn’t used, which can result in significant performance degradation. As such, you should know when the cache batteries are failing to get them replaced before it affects your application's performance. 

Various ways to display the status of your cache batteries exist, including one very new method that we’ve recently made available via PTFs. I'm only going to briefly summarize the existing methods of checking the battery status, since I really want to share the new information.

The system is aware when the cache batteries start to fail, and sends message CPPEA13 to the QSYSOPR message queue.  The first-level message text for this message is somewhat vague – “Contact your hardware service provider.” However, in the recovery section of the message it says you should have the cache battery replaced. When this message is sent to QSYSOPR, you’ll also have a problem created in the problem log, and you can use WRKPRB to display the problem details.

You can use the Work with Disk Status (WRKDSKSTS) command to determine if you have cache batteries requiring attention. Press F11 to get to the second panel; if the status for the disk unit is DEGRADED, it’s possible the issue is due to failing cache batteries. If your drives are mirrored, however, they’ll always show ACTIVE status and you must use one of the alternative ways to check the cache battery status.

There are a two different ways you can view your cache battery status using System Service Tools (STRSST): 

1. You can use the Hardware Service Manager to work with resources containing cache battery packs, which will show you the resources that have cache batteries, and you can then display the battery information.

2. You can use the Display/Alter/Dump service function to invoke the batteryinfo Advanced Analysis feature to display the status of your cache batteries.


And now for the news: some recent PTFs (one for each supported release) now provide a simple program that you can call that to display the cache battery status.  They are:

7.1 - SI40406
6.1 - SI40404
5.4 - SI40403

To display the cache battery information, simply run the following:

CALL QSYS/QSMBTTCC

This will create a spool file containing the cache battery information for the IBM i partition where the function is run. To use this function you must have *SERVICE special authority, or be authorized to the Service Trace function. The Change Function Usage (CHGFCNUSG) command, with a function ID of QIBM_SERVICE_TRACE, can also be used to change the list of users that are allowed to perform this operation.

i Can …. Manage My Passwords with IBM PowerHA SystemMirror for i

This blog article was written by Lilo Bucknell. Lilo is a member of the IBM i Power HA design and development team.Thanks, Lilo!

Today I’m going to show you how to keep your passwords synchronized across multiple systems using some simple commands provided with IBM PowerHA SystemMirror for i. The IBM PowerHA SystemMirror for i LP was introduced in release 6.1 as the IBM i disk-based high availability and disaster-recovery solution. Part of that solution includes the application known as cluster administrative domain. Even if you aren’t ready for a full-blown HA/DR configuration, you can use cluster administrative domain as a simplified management tool for managing resources across multiple systems, including your user profiles and passwords. 

PoweHA Cluster Administrative Domain
The PowerHA cluster administrative domain application provides the capability to maintain a consistent operational environment across multiple systems. A consistent operational environment, along with data and application resilience, is the cornerstone of a highly available application. The operational environment is defined to be the set of resources and attributes that are used by an active application – resources such as user profiles or system values. On IBM i, the cluster administrative domain maintains resources consistently across the nodes in an HA cluster by synchronizing changes to the resources. These resources are called monitored resources within the cluster administrative domain.

Setting Up a Cluster Administrative Domain in 6 Easy Steps
I’m going to show you how to manage your user profiles across multiple systems using the CL commands provided with the PowerHA SystemMirror for i licensed product. In addition to the individual commands used in my example, you can find these steps as options within the WRKCLU command, or you can use one of the PowerHA Web applications that are integrated into IBM Systems Director Navigator for i.

1. Set the QRETSVRSEC System Value

It’s important that the QRETSVRSEC system value be set to 1 before adding any user profiles to your cluster administrative domain. In addition, you should change the password for any user profile you want managed by the administrative domain after changing the system value. This allows the cluster administrative domain to synchronize the password across the nodes in the domain. To change the system value, use this command:

CHGSYSVAL SYSVAL(QRETSVRSEC) VALUE('1')

2. Create the Cluster

Now we’re going to create a one-node cluster and start the node. The cluster name is a 10-character name of your choosing. It’s recommended that the eight-character node name be the same as your system name, for easier management. Using this command, the cluster MYCLUSTER will get created on your first system, represented by the IP address on that system, with a node name of NODEA and the node will be started:

CRTCLU CLUSTER(MYCLUSTER) NODE((NODEA ('9.5.114.142')))

3. Add a Second Node to the Cluster

Next we’re going to add a second node to the MYCLUSTER. This command will result in a cluster node called NODEB, represented by its IP address, being created and started on your second system.

ADDCLUNODE CLUSTER(MYCLUSTER) NODE(NODEB ('9.5.111.151'))

You can repeat this command to add more nodes to your cluster.

4. Create the Cluster Administrative Domain

Now we’re going to create a cluster administrative domain on the two nodes in our cluster using this command:

CRTCAD CLUSTER(MYCLUSTER) ADMDMN(MYDOMAIN) DMNNODL(NODEA NODEB)

5. Start the Cluster Administrative Domain

Next we’ll start the cluster administrative domain, which starts the synchronization process.  Since we don’t have any monitored resources in our administrative domain yet, there’s nothing to synchronize yet. By starting the domain first, any resources we add as monitored-resource entries will get synchronized as they get added. We’ll use the following command to start the domain MYDOMAIN in cluster MYCLUSTER.

STRCAD CLUSTER(MYCLUSTER) ADMDMN(MYDOMAIN)

6. Add a User Profile to the Administrative Domain

Finally, we need to add our user profile to the cluster administrative domain as a monitored resource. The user profile must exist on the system where we’re running the ADDCADMRE command. It may or may not exist on the other node. If it doesn’t exist on the other node, the cluster administrative domain will create it, based on the user profile from the originating system.
   For our example, we only want the administrative domain to synchronize our password, so we will specify PASSWORD as the attribute to monitor. 

ADDCADMRE CLUSTER(MYCLUSTER) ADMDMN(MYDOMAIN)
                        RESOURCE(MYPROFILE)
                        RSCTYPE(*USRPRF) ATTRIBUTE(PASSWORD)       


I now have an active cluster with an active cluster administrative domain monitoring for password changes to MYPROFILE. This means that any time the password is changed on any node, it will be changed on all nodes in the administrative domain. If I change some other attribute of the user profile, such as its job description, the new job description won’t be propagated and changed on the other nodes. I’m not replicating the complete *USRPRF object, only those attributes that I want synchronized; in this case, just the password. I could specify *ALL for the ATTRIBUTE parameter to synchronize all of the supported attributes for user profiles.

You’re now ready to test drive your cluster administrative domain setup. Change the password for your user profile on either one of the nodes in your domain. Now go to second system and sign on with new password. There you go! You have a synchronized password in six simple steps!

There are a couple of neat features with this application that may not be obvious. First of all, you don’t need a central site from which to manage your passwords. Once you have the cluster administrative domain set up, you can make changes from any node, and they will be propagated to the other nodes. Second, and probably even more important, you don’t have to have all of your nodes active in order to maintain consistency. If one or more of the nodes in your domain aren’t active when you make a change to the monitored resource, the change will be applied on the inactive node when it rejoins the cluster.

If you’re interested in reading more about IBM PowerHA SystemMirror for i and the cluster administrative domain application, here are some useful links:

• PowerHA

• Implementing PowerHA for i

• Information Center