Integrate Web Services With DB2 for i 7.1 Part Two

This week's blog article was written by Nick Lawrence. Nick works for IBM in Rochester, Minn., on DB2 for IBM i. His responsibilities include full-text search for DB2 and SQL/XML. Thanks, Nick!

In part one of this two-part blog entry, the idea was introduced that DB2 for i could be integrated with a Web service, using the user-defined functions in the SYSTOOLS schema and the built-in SQL/XML support. Part one includes a number of links to information about the user-defined functions, and also to information on the built-in SQL/XML support. One important resource was a link to a whitepaper that provides the complete details for these functions.

The second part of this blog entry examines a specific scenario where the data from a Web service is converted into relational rows and columns.

Suppose that a delivery service stores its vehicles' fuel and maintenance costs in a DB2 for i database. In addition to the actual expenses incurred, the application's processing involves the weekly national average cost (per gallon) of fuel. In other words, an SQL query might reference the national average, or the average might be stored in the database for auditing purposes.

This problem has two parts to it.

1. The database must retrieve an XML document that contains the current week's average fuel prices from a Web service (this example uses a Web service located at http://www.fueleconomy.gov).
2. After retrieval, the XML document needs to be transformed into rows and columns.

Figure 1 is a diagram of this setup.

Figure 1 - Web service interaction with DB2 for i

Using the functions that have been provided in SYSTOOLS, we can use SQL to get the average fuel price for the current week as an XML document. This is accomplished using the HTTPGETBLOB function and is shown in Figure 2.

Figure 2 - Retrieve current fuel prices

VALUES
 XMLPARSE(DOCUMENT
   SYSTOOLS.HTTPGETBLOB(
     --- URL ---
     'http://www.fueleconomy.gov/ws/rest/fuelprices',

     --- HTTP request header ---
     '<httpHeader>
       <header name="Accept" value="application/xml"/>
      </httpHeader>'
   )
 )

In Figure 2, the URL identifies the resource that is to be obtained, and the HTTP request header indicates that the data should be returned in an XML format. The XML document that is returned from the Web service is shown in Figure 3.

Figure 3 - Returned XML document

<?xml version="1.0" encoding="UTF-8"?>
<fuelPrices>
  <cng>2.12</cng>
  <diesel>4.16</diesel>
  <e85>3.47</e85>
  <electric>0.12</electric>
  <lpg>2.56</lpg>
  <midgrade>3.89</midgrade>
  <premium>4.04</premium>
  <regular>3.75</regular>
</fuelPrices>

The XML document shown in Figure 3 is not useful in SQL unless SQL has a way of understanding the relationships and data defined by the document. Fortunately, the XML support in DB2 for i 7.1 includes the built-in XMLTABLE table function. Figure 4 shows a query that uses the XMLTABLE function to convert the XML document in Figure 3 into a result set (rows and columns).

Figure 4 - Current fuel prices as rows and column

SELECT fuel_prices.*
FROM
 XMLTABLE('fuelPrices' PASSING
  XMLPARSE(DOCUMENT
    SYSTOOLS.HTTPGETBLOB(
      --- URL ---
      'http://www.fueleconomy.gov/ws/rest/fuelprices',

      --- HTTP Request Header ---
      '<httpHeader>
        <header name="Accept" value="application/xml"/>
       </httpHeader>'
    )
   )

 COLUMNS
   "regular"  DOUBLE,
   "midgrade" DOUBLE,
   "premium"  DOUBLE,
   "diesel"   DOUBLE) fuel_prices;

Figure 5 shows the result set that was created by the query in Figure 4.

Figure 5 - Result set

 

The query in Figure 4 is an SQL query, meaning that it can be easily incorporated into other SQL statements or queries that need to reference the data in Figure 5. An example of how to join data from a Web service with relational data is included in the whitepaper.

Although the HTTP GET request might be the most commonly utilized HTTP method, the SQL UDFs can also invoke the POST, PUT and DELETE HTTP methods. The additional HTTP methods can be invaluable for causing a Web service to take some action in response to an event that happens in a DB2 for i database. Some examples of these HTTP methods are included in the whitepaper.

The HTTP functions really make it easier to access Web services, allowing a software architect more time to figure out the most effective way to use the service. The built-in XML support allows developers and architects to spend their time utilizing the information in XML documents, rather than writing code to compose and decompose XML.

 

 

 

New System SSL support

This blog was written by Tim Mullenbach.  Tim is an Advisory Software Engineer at IBM Rochester. He has been a part of the IBM i sockets/SSL development team since 1995 and leading the team for the past eight years. He has worked on designing, implementing, and testing various socket and System SSL enhancements for the V4R2 through V7R1 releases.

Hopefully you noticed the recent messaging for IBM i 7.1 TR6 which included the statement “Latest industry standards of Transport Layer Security version 1.2 (TLSv1.2) and Transport Layer Security version 1.1 (TLSv1.1) protocols implemented.”  

In my conversations with customers I find that many do not realize that TLS is the “new” name for SSL.  And by new, I mean for the past 14 years.  SSL stands for Secure Sockets Layer and was created by Netscape a long time ago.  TLS stands for Transport Layer Security and was created/designed/defined by the Internet Engineering Task Force (IETF) in 1999.  IETF standardized and improved the SSLv3 specification making TLSv1.0. While technically called TLSv1.0, many times the .0 was left off leaving TLSv1.  Implementers assumed the next version would be TLSv2.0 since SSLv2 preceded SSLv3.  No such luck – the next version was called TLSv1.1 with the one after that called TLSv1.2. 

Unfortunately the vast majority of people still call the entire concept SSL.  It can make for confusing conversations as people like me try to interpret what SSL means in any given context.  In all likelihood it is both SSLv3 and TLSv1.0 being described by the use of SSL.  A level of uncertainly always remains however due to a handful of applications that still only support the SSL protocol version known as SSLv3.  And for the few of us that call it all TLS instead of SSL, we lump SSLv3 under the term TLS.

At this point you might be asking, “Thanks for the name game and history lesson, but what does new support mean to me and my business?” 

IBM i and its precursors have supported TLSv1.0 (SSLv3 and SSLv2 too) at least back to V4R5 days.  As of Feb 2013 with the release of IBM i 7.1 TR6, we have support for TLSv1.1, defined by RFC 4346 and TLSv1.2, defined by RFC 5246.  Each successive version of the SSL/TLS protocol corrects issues found with the previous version and improves upon the state of the art.  The latest version available should be used to obtain the highest level of security available for TLS/SSL.

There is a catch though, nothing can be simple.  From 1999 to 2005 almost every application, work station, and intelligent device in the world had SSLv3/TLSv1.0 support built into it.  The vast majority of those devices still work, perform business critical transactions, and have zero ability to be enhanced with newer TLS/SSL protocols.  This significantly impacted and is still impacting the roll out of TLSv1.1.  With TLSv1.2 defined so close to TLSv1.1, TLSv1.1 became a lost protocol version.  TLSv1.2 adoption initially suffered for the same reasons.  The difference is that TLSv1.2 is more of a game changer than TLSv1.1.  Sure TLSv1.1 protects from additional attack vectors than the previous protocols did, but other workarounds existed to limp along without it. 

So what is the big deal about TLSv1.2?  Besides all of the incremental improvements, its big selling point is that it uses the SHA-2 hash algorithm instead of SHA-1 and/or MD5.  This is important because of NIST 800-131a.  You won't see SSL or TLS mentioned by name, yet when pieced together the recommendations lead to TLSv1.2.  NIST is playing the prediction game – how long will things secure today remain secure?  While you might have longer than the end of 2013 as predicted in that document, a wise security administrator would start transitioning to the new protocols and algorithms sooner rather than later.

The IBM i 7.1 TR6 enhancements make it easy enough to enable TLSv1.2 for your applications.  That is only half the story, the easy half at that.  Your communication partner also must support TLSv1.2 for it to be negotiated.  This ability varies greatly by application and the peer devices themselves.

For years client applications had no incentive to add TLSv1.2 support since the servers didn't support the protocol.  To make matters worse, a very small number of broken SSL implementations in the world fail when a client tries to negotiate TLSv1.2. 

It would be a good idea to enable TLSv1.2 for all of your IBM i 7.1 servers.  Then start work to enable TLSv1.2 in all of the client applications you have control over that connect to those servers.  It could take years, but once all of your clients support TLSv1.2, that should become the only protocol supported by the server.

The IBM i 7.1 client applications you use should also have TLSv1.2 enabled after verifying the server at least tolerates TLSv1.2 being requested.  Like with the server, once you know the server supports TLSv1.2, the older protocols should be disabled.

The unfortunate reality is that many applications must continue to support TLSv1.0 for years.  Some peer devices have no upgrade path to get new SSL support so you wait for the device to be replaced. 

Please read “How to use TLSv1.2 with System SSL on IBM i 7.1” on developerWorks for details on enabling TLSv1.2.

While overshadowed in the developerWorks article by TLSv1.2, the new application level cipher suite control is also very interesting.  For the first time ever, you can control the ciphers supported for the various IBM applications on an individual basis.  This means FTP, Telnet, and SMTP (client or server) can all use a different list of ciphers at the same time.  Gone are the days of taking a sledgehammer to the Secure Sockets Layer cipher specification list (QSSLCSL) system value in order to get an individual application to pass a security scan. 

This configuration option is available from the enhanced Digital Certificate Manager (DCM) application definition support.  To locate this support, select ‘Manage Applications’ and then ‘Update application definition’ from the left hand panel in DCM.  Once the application to update has been selected, the ‘Update Application Definition’ panel will display the new fields that can be updated.  Use these fields to control both the ciphers and protocols used by the application.  Refer to the panel help text for details on these configuration options as well as other additional System SSL properties now located on that panel.

Performance Data Investigator

A few weeks ago I wrote about the Performance Data Collectors in IBM i; you can collect many different kinds of performance data, but once you have the data, what is the best way to view and analyze that data? Use the Performance Data Investigator. If you are on 6.1 or 7.1 you already have it.

The Performance Data Investigator was introduced with the 6.1 release and is found in the Navigator for i web console; it is not available in System i Navigator (and it will never be in System i Navigator).

Within the Navigator browser console, you will find the Performance tasks in the left-hand navigation pane. When you select the Performance tasks, you will see the following options:

The Investigate Data option is how you access the Performance Data Investigator. When you take the Investigate Data option, you will see something like the following image:

The initial list of perspectives shows you the set of content packages that are available; a content package is simply a grouping of perspectives for the same type of performance data. The Collection Services content package contains all the graphs and tables for Collection Services data. Which content packages you see depends upon what options of the Performance Tools licensed program product are installed.

You will always see Collection Services and Health Indicators; the capability to view and analyze Collection Services data is part of the base operating system and no other product is necessary.

However, if you want to analyze Disk Watcher or Performance Explorer data, you will need to have the Performance Tools product, Manager feature installed. To analyze Job Watcher data, you need to have the Performance Tools product, Job Watcher feature installed. Below is a screen capture of the Performance Tools product with all the options installed:

On the main Investigate Data panel, open the type of performance data you want to analyze – for example, you can expand the Collection Services folder to show the following navigation:

You can see that the list of perspectives (which is simply a fancy name for a graph or table) is organized by the type of information you want to see. You can further expand the folders to find the perspective you want to view. Once you have identified the view you want to see, you can display it.

Note at the bottom of the screen you have the capability to select which performance data collection you want to display, so you can have various performance data collections in different libraries.

Once you have identified the graph or table you want to display along with the performance collection you want to use, you display the data. An example of what a graph looks like is below:

The Performance Data Investigator has a wealth of capabilities; for a more complete article on the Performance Data Investigator, please refer to the IBM i Performance Data Investigator article on the developerWorks site.

Finally, here is a list of prior blogs with information related to the Performance Data Investigator:

Investigate Performance Data

Display CPU Utilization for all Partitions

Tell You Why You Are Waiting

Understand Scaled CPU Time

Measure Disk Response Times

Customizing a Perspective in PDI

Find All My Performance Collections in One Place

Job Performance Information

Performance Data Collectors in IBM i

 

 

 

Multi-Threaded Job Table Cleanup

I'd like to thank Mike Russell for writing most of this blog article. Mike works on the IBM i Work Management team and was responsible for the changes described in this blog. Thanks, Mike!

 

Some time ago, I wrote about the job table and permanent job structures in IBM i Job Structures (January 11, 2011). That article reviewed the job table and also discussed compressing the job table. 

In the IBM i 7.1 release, IBM restructured the processing of the job table cleanup that at happens during an abnormal IPL as well as the processing that is done when job tables are compressed. The job table cleanup IPL step is also known as Work Control Block Table cleanup (C900 2C40) IPL step.

Prior to 7.1, the job table was processed by one thread during the IPL step; when the job table became large, it could take this one thread quite some time to process all the entries in the job table. In the 7.1 release, this single-threaded processing was changed to be multi-threaded. This change decreases the time the system is unavailable after an unexpected termination as well as when the job tables are compressed during an IPL.

This new news is that this function is now available for the 6.1 release with PTF SI48088.

The performance improvement is seen on systems with a large number of entries in the job table. You can use the Display Job Table (DSPJOBTBL) command to view the number of entries on your system. The Change IPL Attributes (CHGIPLA) command allows you to influence how much cleanup is done at IPL with the Check job tables (CHKJOBTBL) and Compress job tables (CPRJOBTBL) parameters. How the system powered down is the other influence on how much cleanup is done at IPL. You can check the history on your system to see if you have had any recent abnormal IPLs with the Display Log command: DSPLOG PERIOD((*AVAIL *BEGIN)) MSGID(CPI0990 CPI091D)

There is also a 7.1 release PTF (SI48454) that includes a minor performance improvement for job table cleanup and a correction for the IPL status estimated time remaining:

 

You should not have to run job table cleanup or compression very often, but it is one of those things that when it is necessary, time is most likely important as well. This was an important fix for reducing IPL time in the 7.1 release and it's now available on 6.1.

Work With Job and Select a Thread …. to Print Thread Stacks

In the 7.1 release, the Work with Job (WRKJOB) command was enhanced; you can determine this by going to the CL Command Finder and take the option to display “All changed commands.” The Information Center will show you a list of changed commands and you can read the documentation on the command. Of course, there's a slight problem in that commands are not change-flagged, so you cannot actually see what changed. You need to compare the command documentation with that of the prior release to figure out exactly what has changed.

By comparing the 6.1 version of WRKJOB with the 7.1 version, you will discover that IBM added a new parameter, Select Thread (SLTTHD) to the command. Upon first glance, it might appear that this allows you to simply work with a specific thread for that job; maybe assuming it's a “fast path” way to do a WRKJOB followed by Option 20 (work with threads).

But that is not the case.

A blog I wrote nearly three years ago, Display Call Stacks, discussed the various ways you can access call stacks, as well as some enhancements that were made to the call stack itself in the 6.1 release.

What do these two things have in common?

The Select Thread parameter provides an interface to print the thread stacks for the initial thread of a process (the default), selected threads in a process, or all threads in a process. Prior to this enhancement, the only way to access all thread stacks within a job was to do it interactively; if you specified OUTPUT(*PRINT), it would just print the call stack of the initial thread. A better way was needed to capture integrated thread stacks (which include the IBM Technology for Java stacks and PASE stacks) for diagnostic purposes, which the addition of the SLTTHD parameter on the WRKJOB command provided.

Here's how you would print the call stacks for all threads within a specified job:

WRKJOB JOB(######/user/jobname) OUTPUT(*PRINT) OPTION(*PGMSTK) SLTTHD(*ALL)

And here's how you could select one or more threads from a list; if you select multiple threads from the list presented, each thread's call stack will be printed in the single spooled file that is generated:

WRKJOB JOB(######/user/jobname) OUTPUT(*PRINT) OPTION(*PGMSTK) SLTTHD(*SELECT)

This enhancement is sneaky is because the new parameter, SLTTHD, was hidden from the WRKJOB command prompt – you need to press F9 in order to see it, and because it requires the specific combination of OUTPUT(*PRINT) and OPTION(*PGMSTK) for the option to be valid.

If you have a need to generate spooled output of thread stacks for a job, this handy command invocation will do it for you.

Delete That Object

Did you know that in the IBM i 7.1 release, a new command was added to delete an object? I'm sure you can easily figure out what the name of the command is. Did you also know that in the 6.1 release, the API to delete an object was added?

The Delete Object (DLTOBJ) command (or the Delete Object API) is similar to the other object commands, for example Dump Object or Move Object, where you specify the name of the object along with the object type. Not all object types are supported by the generic Delete Object command or API, but many of them are. Some of the DLTxxx commands have additional parameters; if you use the generalized DLTOBJ interface to delete these objects, the system will use the default values that could have been specified on the more specific DLTxxx command (these default values are documented in the Delete Object API documentation). The Delete Object API also documents the object types that are supported by these interfaces.

There are a few advantages that you have when using the DLTOBJ command:

• You can delete multiple libraries (object type of *LIB) with one DLTOBJ command by specifying a generic name. Be careful!
• You can specify *ALL for the object name and have an easy way to delete all of the objects of a specific type from a library. Be careful!
• You can delete an object from a specific ASP or a group of ASPs by using the ASPDEV parameter.
  
  

And here's a little bit of trivia for you … the request for providing a generic delete command originated back in 1990; it may take IBM a little while, but good ideas are not forgotten, they sometimes just take some time to work their way to the top of the priority list.

 

IBM i Security Never Sleeps

This week's blog was written by Scott Forstie, senior software engineer at IBM. He is the DB2 for i Business Architect, SQL development leader and IBM i developerWorks content manager. Thanks Scott!

Security never sleeps. On IBM i, we understand that we must continue to invest in delivering improved security auditing, security deployment and granularity of security controls.

To that end, this post provides a brief recap of security enhancements that have been delivered to existing IBM i releases over that last 12 months. Each enhancement has its own "fact page" on the IBM i Technology Updates wiki.

Command Language Enhancements

CL retrieve exit programs can run after command completion
The IBM i operating system supports two exit points for control language (CL) commands:

QIBM_QCA_CHG_COMMAND and QIBM_QCA_RTV_COMMAND. For each regular CL command, and proxy CL commands in the QSYS library, one exit program can be registered for the CHG exit point and up to ten exit programs can be registered for the RTV exit point.

Prior to this enhancement, QIBM_QCA_RTV_COMMAND exit programs were always called just before control was transferred to the command processing program (CPP) associated with the command being run. The enhancement allows you to be able to register an exit program for the QIBM_QCA_RTV_COMMAND exit point and indicate that you want the exit program to be called after control returns from the CPP.

Delivered on: V5R4 (SI45987), IBM i 6.1 (SI45986) and IBM i 7.1 (SI45985)

Extend CD (Command String) journal entries
The IBM i operating system allows customers to track the CL commands that are being run by a user. Auditing needs to be active before command-level auditing can be done. The Change Security Auditing (CHGSECAUD) command allows you to change the current settings for the system values that control what is being audited on the system. To turn on command-level auditing for a user profile, run the Change User Audit (CHGUSRAUD) command specifying the user profile name for the USRPRF parameter and *CMD for the AUDLVL parameter. This will cause CD (Command String) audit records to be generated for each CL command run by the specified user profile. The model file QASYCDJ5 describes the fields in CD audit records. One of these fields, CDCLP, has been redefined to convey more information concerning how the audited CL command was run.

The new values for the CDCLP field map to the values for the ALLOW (where allowed to run) parameter on the Create Command (CRTCMD) command as follows:

'Y' maps to *IPGM, *BPGM, *IMOD, *BMOD
'R' maps to *IREXX, *BREXX
'E' maps to *EXEC
'B' maps to *BATCH
'N' maps to *INTERACT

Delivered on: V5R4 (SI44854), IBM i 6.1 (SI44864) and IBM i 7.1 (SI44865)

DB2 for IBM i (database) Enhancements

Masking support in FIELDPROCs
The FIELDPROC support has been extended to allow masking to occur to that same column data (typically based on what user is accessing the data). For example, only users that have a need to see the actual credit card number will see the value while other users may just see masked data. For example, XXXX XXXX XXXX 1234.

Delivered on: IBM i 7.1 (DB2 PTF Group SF99701 Level 14)

STRDBMON pre-filtering of QUERY-400 command use
QUERY/400 commands Run Query (RUNQRY), Work With Queries (WRKQRY) and Start Query (STRQRY) already generate database monitor records. This enhancement delivers a method to easily identify Query/400 users and the queries executed via the Filter by Client Program database monitor pre-filter.

Delivered on: IBM i 6.1 (DB2 PTF Group SF99601 Level 25) and IBM i 7.1 (DB2 PTF Group SF99701 Level 14)

Simplified DDM and DRDA authentication entry management using group profiles
This enhancement allows DDM and DRDA to take advantage of a common userid and password defined in a server authentication entry under a group profile name or supplemental group profile name. The group profile name or supplemental group profile name is specified on the USRPRF parameter of the ADDSVRAUTE command.

Delivered on: IBM i 6.1 (DB2 PTF Group SF99601 Level 25) and IBM i 7.1 (DB2 PTF Group SF99701 Level 14)

Add QIBM_DB_ZDA and QIBM_DB_DDMDRDA function usage IDs
These function usage IDs block database server inbound connections and are not based on the communication protocol. The function usage IDs ship with default authority = *ALLOWED. The security officer can easily deny access to specific users or groups.

Alternative to a User Exit Program approach. No coding required, easy to change and auditable.

1. QIBM_DB_ZDA (restrict ODBC and JDBC Toolbox from the server side, including Run SQL Scripts, System i Navigator and DB2 specific portions of Systems Director Navigator for i)
2. QIBM_DB_DDMDRDA (ability to lock down DDM and DRDA application server access)

Delivered on: IBM i 6.1 (DB2 PTF Group SF99601 Level 25) and IBM i 7.1 (DB2 PTF Group SF99701 Level 13)

QIBM_DB_OPEN exit program object auditing control
The QIBM_DB_OPEN exit program has been enhanced to allow users to control which files will cause the exit program to be called, which can significantly reduce the overhead of using the exit. The open exit will be called for ALL files that are specified in a query or open if ANY of the files have specified object auditing.

Delivered on: V5R4 (DB2 PTF Group SF99504 Level 33), IBM i 6.1 (DB2 PTF Group SF99601 Level 23) and IBM i 7.1 (DB2 PTF Group SF99701 Level 12)

Add QDDMDRDASERVER server authentication entry special value
A special value QDDMDRDASERVER has been added to the ADDSVRAUTE command SERVER parameter for DDM & DRDA connections. This special value allows an administrator to configure a user to work with all possible DDM or DRDA connections to any system in the TCP/IP network via a common userid and password. Once configured for a specific user, no additional changes need to be made for that user as systems are added to the Relational Database Directory.

Delivered on: V5R4 (DB2 PTF Group SF99504 Level 32), IBM i 6.1 (DB2 PTF Group SF99601 Level 21) and IBM i 7.1 (DB2 PTF Group SF99701 Level 11)

Query Manager profile auditing
Changes made to a Query Manager profile can now be audited if auditing is enabled for AUDLVL(*SECURITY).

• A new journal entry type of X2 will contain the old and new Query Manager profile values.
• An outfile is not provided for this journal entry. Instead view QSYS2.SQLQMProfilesAudit can be queried.

Delivered on: IBM i 6.1 (DB2 PTF Group SF99601 Level 21) and IBM i 7.1 (DB2 PTF Group SF99701 Level 10)

Move My i

Today is a major announcement day for IBM i with Technology Refresh 4 (TR4). Steve's “You and i” blog reviews the content of this announcement.

Last year, IBM provided support to suspend an i partition, which I previously wrote about in “Suspend my i”. 

The most significant function in IBM i TR4 is Live Partition Mobility. With Live Partition Mobility, you can take an active, running IBM i partition and move it from one frame to another frame, without disruption to your applications. I'm not going to discuss more about what mobility is and why you might want to use it, as Steve covered that in his blog. What I do want to write about is a little more detail on just what you need to accomplish this.

There are both hardware and software prerequisites:

• POWER7 tower/rack system for both the source and destination systems
  • Firmware service pack 730_xx, 740_xx, or later
  • IBM Hardware Management Console V7R7.5.0M0, or later
• All I/O must be virtual and supported by VIOS; it can be Virtual SCSI (VSCSI), Virtual Fibre Channel (NPIV) or Virtual Ethernet
• The configuration must use only external storage and that storage must be accessible to both the source and destination systems
• Both source and destination systems must be on the same Ethernet network
• IBM i 7.1 TR4 PTF group – SF99707 level 4
• PowerVM Enterprise Edition on both the source and target system
• VIOS 2.2.1.5 (FP25 SP3) on both the source and target system [[CORRECTION: It's VIOS 2.2.1.4]]

There are a few restrictions that are too detailed to review in this blog; the Live Partition Mobility documentation in the IBM i developerWorks site reviews the complete set of prerequisites, requirements and restrictions.

The actual move of a partition is initiated from the HMC. IBM Systems Director VMControl has relocation, which you can imagine will ultimately support IBM i as well.

In general, applications and the operating system are unaware that the partition is moved from one system to another. There are some exceptions to this, such as Collection Services; when the partition is starting to run on the target system, the Collection Services collector job will cycle the collection so correct hardware information is recorded on the target system.

There are two new work management exit points that are part of the IBM i suspend and mobility support. These exit points allow to you run a program before and after a partition is suspended or moved. (These exit points are the same, regardless of whether you are suspending a partition or moving a partition). 

            Work with Registration Information

 

 Type options, press Enter.

  5=Display exit point  8=Work with exit programs

 

              Exit

   Exit         Point

 Opt Point         Format  Registered Text

 _  QIBM_QWC_RESUME    RSMS0100   *YES   Resume system

 _  QIBM_QWC_SUSPEND   SSPS0100   *YES   Suspend system  

The suspend exit is called before the partition is suspended or moved and is called two twice. The first time it is called to check to see if the operation is allowed. The second call is to prepare for the move or suspend operation.

The resume exit is called after the move is complete or after the system has been restarted from a suspended state. Applications can use these exit points to monitor and accept or refuse move or suspend requests. Running an exit program after a move or suspend also allows an application to refresh any copies of any system-specific information, log information about mobility requests for licensing or other purposes, adjust heart beat rates or whatever actions an application may require.

For more information on these exit points, refer to the “What's New”for APIs in the IBM i 7.1 Information Center.

The following messages may be logged to the QSYSOPR message queue as part of the suspend or move request:

• CPI09A5 - Partition suspend request in progress.
• CPI09A6 - Partition not ready for suspend request.
• CPI09A7 - Partition suspend request canceled.
• CPI09A8 - Partition resumed after migration.
• CPI09A9 - Partition resumed from hibernation.

The system serial number will change when a partition has been moved. The model and processor feature may also change.

The documentation on enhancements added to IBM i via Technology Refreshes can be found on the IBM developerWorks web site.

Other sources for documentation on Live Partition Mobility for IBM i include:

• Partition mobility overview for HMC
  
• IBM PowerVM Live Partition Mobility
  
• API Guide (For registering system exits)
• Licensing information
• Software Knowledge Base

          Search for “Live Partition Mobility” to find any usage notes.

 

Remote Journal Filtering Reduces Network Traffic

This week's blog was written by Bruce Hansel, who is a member of the IBM i journal team.

Remote Journaling has been available on IBM i for many releases. It provides an efficient way to replicate journal entries from one system to another and is often part of a high availability or disaster recovery solution. As HA and DR have become a high priority in recent years, remote journaling has become more heavily used and multiple enhancements have been made in recent releases.

Many of the recent enhancements have been to make Remote Journaling more efficient and to provide information on the status of the Remote Journal environment. Common questions and concerns that arise around the use of Remote Journaling are often related to the communication environment. Network bandwidth, delays and retransmissions are among the top concerns.

Several options can be used to reduce Remote Journal traffic, one of the most significant is Remote Journal filtering which was introduced in 7.1 and is available with the HA Journal Performance (option 42) feature. Remote Journal filtering allows the user to eliminate sending journal entries that are not needed on the target system, reducing the required network bandwidth for Remote Journaling. Journal entries can be filtered in several ways: before image filtering, filtering by object, and filtering by program. Combinations of these can be specified as well. In each case, the number of journal entries and the amount of data sent over the network may be reduced. How much it is reduced is a function of the user's environment.

The simplest scenario is before image filtering. Typically, before images of a record in a database file are journaled because commitment control is being used. If a transaction is rolled back a before image is needed to return the record to its original state. This is appropriate but it is likely that the before image is not needed on the target system. This may be true in an HA/DR environment or in an environment where Remote Journaling is used just to send journal receivers to a target system where they are saved for backup. Not only is the amount of data sent over the network reduced but the size of the journal receiver on the target is also reduced, thereby reducing backup storage requirements.

There are some limitations on the use of journal receivers that have had entries filtered. Receivers that have had before images filtered cannot be used for RMVJRNCHG operations but they can still be used for APYJRNCHG operations. Other filtering criteria may inhibit the use of APYJRNCHG. Each of these limitations should be considered before using Remote Journal filtering and when considering the type of filtering to specify.

The additional flavors of Remote Journal filtering and the advantages and limitations of each are discussed in a technote on the IBM Redbooks website Journaling: Remote Journal Filtering in IBM System i 7.1.

More information on the use of Remote Journal filtering can be found in IBM i InfoCenter.

 

 

 

 

 

Achieve Faster IFS Save Times Using SAV With ASYNCBRING

An enhancement has been made available in IBM i 7.1 and 6.1 releases that may help with integrated file system SAV performance. A new parameter, Asynchronous Bring (ASYNCBRING) has been added.

The ASYNCBRING support will enable objects to be asynchronously brought into memory early so they will not need to be paged in when first accessed by the SAV processing. While results will vary, testing in the lab and by a few customers have shown some dramatic improvements - up to 60% faster save times in some cases. The performance gain seen is dependent on the directory structure, number and size of objects saved, amount of memory available, and the system configuration. This enhancement is available on both the SAV command and QsrSave() API.

To use this new function, you will need to install necessary the PTFs:

7.1 PTF information:

SI44688 Save/Restore PTF with new function added.

SI45792 Supersedes SI44688 and contains additional SAV performance improvements

SI44710 BRMS PTF that supports ASYNCBRING.

 

6.1 PTF information:

SI44739 Save/Restore PTF with new function added.

SI45791 Supersedes SI44739 and contains additional SAV performance improvements

SI44709 BRMS PTF that supports ASYNCBRING.

 

This new function has been written about in the IBM i developerWorks article "Achieve faster IFS save times using SAV with ASYNCBRING."

The BRMS product website also has additional information on this support under the topic Does BRMS support the ASYNCBRING parameter when saving the IFS? and describes how you can use this new support with BRMS.

One final note - the best performance improvement may be seen with a well-balanced directory tree in which all objects qualify for the save. In situations where a large number of objects reside in a single directory, few objects qualify for the save, or the system is memory constrained, performance may degrade with ASYNCBRING(*YES) specified.

 

I'd like to thank Jerry Simon from the Save/Restore team and David Bhaskaran from the BRMS team for their assistance in creating this blog article.