August 26, 2014

Who Was That Masked Man? Or That Masked Value?

In a recent post, we voiced our concerns about the potential for application error when taking advantage of the masking support related to RCAC in V7.2 of DB2 for i.

One of our concerns was about the potential loss of data when updating a row (aka record) that contained a masked column (aka field) value. For example, a credit card number might be masked like XXXX-XXX-XXX-1057 when retrieved in an RPG program. When that row is updated by the program, it’s quite possible that the actual value of the credit card number might be lost (i.e., overwritten by the masked value as seen by the RPG program).

Our friend, Kent Milligan, commented on the blog that there was support added to DB2 for i to allow the database itself to prevent data loss in that situation. Since we don’t yet have a 7.2 system to “play with” ourselves, we haven’t been able to pursue that and/or other potential solutions for that particular issue. We did, however, recently see an article Kent wrote on the subject.

The basic idea is that DB2 for i has implemented an enhanced capability for check constraints. Typically, if an “ordinary” check constraint were in place, an update to the row would be prevented if the check constraint were violated. That basic support could be used to prevent accidental loss of data by preventing updates if, for example, the first 12 characters of the credit card number were ‘XXXX-XXX-XXX’ (i.e., the mask value). But the problem is that we wouldn’t be able to update any other values in the row either if we had a masked credit card value in our program.

So the new support has added syntax on the check constraint “On Update Violation Preserve <column-name>.” This enhanced support means that if DB2 detects the specified value (i.e., the mask value), then updates will be allowed for other columns in the row but the original unmasked value for the credit card number will be retained. Whew!

Note that it will be necessary to set up this new check constraint option in addition to setting up the RCAC support to mask the column value. Also it doesn’t actually detect that it’s a mask put in by DB2; it only checks for a specific value. So potentially there could be other uses for this support beyond dealing with the RCAC-masked columns—although we can’t think of any.

We encourage you to learn more about this from Kent’s article. This new support doesn’t do away with all our concerns about masked values but at least, if implemented correctly, it would appear to avoid the worst of the potential nightmare scenarios! It may not be the solution we’d have preferred but it should help to prevent many of the mistakes we feared.



August 19, 2014

Boost Your RDi productivity With iSphere

Back in March 2013 we wrote an article for the IBM i Extra enewsletter in which we covered the basics of the new iSphere plug-in for RDi. Since that time, iSphere has received several enhancements, so when we saw the announcement this week of an updated version, we decided we had to take another look. Boy are we glad we did! The range of features they have added to this excellent tool is impressive.

We intend to do a complete report, which we will be publishing in an upcoming edition of the IBM i Extra enewsletter, but we wanted to give our fellow RDi enthusiasts a heads-up in advance because some features are so good you should be using them today.

The latest version includes the following features:

  • Message File Searching and Editing
  • Binding Directory Editor
  • Compare/Merge Editor
  • Fast search feature for source files.
  • Ability to open spooled files in Text, HTML or PDF format.
  • Task Tags
  • Decorators for objects in the RSE tree view

The search capabilities alone are worth the price of admission. They are blazingly fast. Quite how they have managed to so significantly outperform IBM's own source search capabilities we don't know, but they have--by an order of magnitude. The developers claim up to 60 times faster than IBM's version and based on our limited experience we would not argue with that claim.

Task tags are a nice extension to the standard task support. Simply add a simple comment to your source file to remind you to perform a specific task (e.g., "// TODO must add prototype definitions") and when you save the source a new task is added to the task list. Later, when you find the time to catch up on some of your outstanding tasks, simply clicking on a specific item opens the source file and positions to the task marker. Later, when you have completed your work, simply delete the comment (or just remove the characters "TODO" from the text), save the file and the outstanding task list is automagically updated.

Last, but not least, the new Decorator support. When enabled (via Preferences) the file/member/object description is displayed alongside the object's name. Makes it so much easier to see which source member (for example) it is that you need to open.

You can download the latest version of the plug-in from iSphere's SourceForge page. Documentation is a little thin on the ground right now although some has been incorporated in RDi's help system. The only "official" install instructions we can find are in the project's wiki. The installation is covered here. Don't skip the final step, it installs the server components which are essential to the plug-in function.

We really think you'll enjoy using this new tool. Our congratulations to the development team; it really appears to be a great productivity booster.   

August 05, 2014

Scratching the Programming Itch

In a blog post back in March we mentioned that we had participated in a KickStarter campaign to help fund the development of Scratch Jr., an evolution of the original MIT Scratch project. Last week the iPad version of Scratch Jr. was released to the world via the App Store and Jon has been having fun ever since.

Scratch Jr. is aimed at 5- to 7-year-olds (which perhaps explains its appeal to Jon) and is a lot simpler and easier to use than the original Scratch, which targeted ages 8 to 16. That's not to say that more "mature" folks can't have a lot of fun with it!

We loaded it on our iPad as soon as it was available and have been having a ball with it ever since. We also told our daughter about it and she and our granddaughter and grandson leapt straight into it and are also greatly enjoying the experience.

Rather than try to describe how the programming is done (and make no mistake--the kids are writing programs without even realizing it) we recommend that you visit the website and look at some of the videos and samples--what these tiny "programmers" can achieve is nothing short of amazing.

While very few of the kids who use these tools will grow up to be programmers in the sense that we understand the word, we desperately need more people who can think logically. Tools like Scratch and Scratch Jr. can certainly help achieve this.

Right now Scratch Jr. is only available for the iPad with an Android version in the works. But if you have young children or grandchildren it is worth buying an iPad for them just to see what they can do.

It also occurs to us that many RPGers who have yet to explore programming for Web 2.0 style applications could benefit a lot from some time spent with these tools--and since they are free, there's zero excuse.

July 30, 2014

RPG Speed Dating

Remember Y2K? It is somewhat mind-blowing to realize just how many years have elapsed since then. We were both working for IBM at the time and were heavily involved in IBM's efforts to provide tools and education to remediate the problem. In fact teaching IBMers and ISVs about IBM's Bypass2000 product took both of us around the world including our first (and to date only) visit to China. It was an interesting time.

One thing that surprised us as we discussed remediation plans with various IBM users was how few intended to take advantage of what we came to know as "real date fields" as part of their strategy. By "real" we mean DDS type L (SQL type DATE) as well as type D fields in RPG. When we questioned the rationale, two major reasons were given:

1. It took up more disk space than the four bytes used to hold a fake date in packed format. In fact this is not true. On disk a real date is held as a 4 byte integer. We just don't get to see it in that format in our programs.

2. A performance penalty was incurred every time a record was read to convert the date from the stored integer to the 8 or 10 character representation that we see in our programs. This of course is true, although much of the time bad RPG programming practice caused the performance hit to be far worse than it needed to be. Our feeling then was that for most shops the advantages far outweighed the performance difference in most cases. We still feel that way.

In recent years we have seen more and more people begin to take advantage of real dates. Many factors are at play in this increased uptake. The increase in the use of SQL and DDL-defined tables is part of the picture. The massive performance improvements in processor and disk speed is another.

But things can still get better, and the good folks on the RPG compiler team decided to add their own performance boost to date- and time-handling. This support actually came about some time ago, but since it was made available via PTF vs. a new release boundary, many people missed hearing about this when it occurred.

This support is available via PTF for both V7.1 (PTF SI43157) and V6.1 (PTF SI43156). It's a new H spec (or Ctl-Opt operation) keyword. The VALIDATE(*NODATETIME) option can speed up programs that do significant amounts of date- and time-handling. It achieves the performance boost by bypassing something you probably didn't even know it was doing - validating _every_ date field _every_ time it is used. Yup, _every_ time it is used.

With this option in play, under some circumstances the compiler will just treat the date as a character field and assume that it contains a valid date and bypass the validation. What circumstances? The following are some examples of cases where validation will not be done. Other circumstances may be added to the list later.

  • When one date field is moved to another with the same format and separator. This includes the moves that occur when the field is read from the database, assuming the same format externally and internally, which is not always the case, depending on how the I/O operation is done and/or what the default date format is in the program.
  • When a comparison, search or sort operation is being made between *ISO or *JIS formatted dates (i.e., YYYY MM DD) and between times in any format other than *USA. Using this option can have significant performance benefit for programs that handle a lot of dates in these categories. However, remember that some risk is still involved since the compiler will not be validating all date and time values. These are the main points to remember to take the biggest advantage of this new support:
  • Make sure your code handles dates and times appropriately and that you are confident of all date values--especially any that may be definted in data structures.
  • Strive for date formats and separators that are consistent both in your external database definitions and internally in RPG. Use *ISO (or *JIS) as the format of your date fields wherever feasible to gain the most benefit for comparisons, searching and sorting. Avoid *USA for any time fields.
  • Watch out for a DATFMT keyword on H specs (or Ctl-Opt operations). This will change the default format for the program and may inadvertently result in non-ISO formats. If you don't specify DATFMT, the default will be *ISO, which is one of the preferred formats.
  • When date data will be moved/assigned in the program, use the same date and time formats and separators as much as feasible. Don't forget that this applies not only to Eval (or heaven forbid, MOVE operations) but also to parameters defined using VALUE or CONST or function return values.
  • If your external date fields are not *ISO format, then ensure that those fields are defined in Data Structures so that they will retain their external formats. This way the movement to/from I and O specs by the compiler can use the faster "no validation" option.  

You can get many more details about this support from a notice on the RPG Cafe in DeveloperWorks. On that page, you'll see a PDF attachment with many details and helpful advice on how to make the most of this feature.

The compiler team intends to increase the number of cases where validation is bypassed in subsequent releases, so if you use this option make sure to watch for future announcements in the RPG Cafe that may affect your code.

July 22, 2014


Last week we spoke at the OCEAN User Group's annual conference in Costa Mesa in (normally) sunny California. It was great to see that like several other IBM i events lately, their numbers were up from previous years at the main event. They also had excellent attendance at the dinner and keynote session, featuring Alison Butterill talking about modernization and at a pre-conference workshop on Ruby for IBM i.

This year the group also tried out the idea of running in-depth workshops including hands-on Labs on the Saturday after the main event. We taught an introductory RDi class, IBM's Dan Cruikshank taught SQL topics including IBM Data Studio and Larry Bolhuis taught systems management. We're told that there were approximately 100 attendees at the workshops, which means that 100 people gave up part of their weekend to learn--and that is really encouraging.

OCEAN is a great group and if you live in reach of the Orange County, California-area you should definitely be a member.

OCEAN works with a local technical university to host the main conference and the workshops (the keynote and dinner were held at a hotel to accommodate larger numbers in a single session). We've mentioned this idea of using colleges for conferences before. It really can be a practical solution for local user groups that have considered doing events but found hosting them in hotels expensive and inflexible. In fact it does seem to be a bit of a trend with TUG, our local Toronto user group, holding its TEC event at a local college. We have heard that the OMNI group is working on a similar idea for an upcoming event in the Chicago area.

Other Items

In last week's blog we mentioned punch cards in what reader Bob L. considered a derogatory fashion:

"Don't belittle the punched card. Its legacy lives on: after all, that is why you still have to remind those "young young" programmers that the code must start on or after column 8 and end at column 80!!"

We certainly didn't mean to belittle the punch card (except for maybe those silly little 96 column things) they were really useful for writing notes. Jon remembers that in his early days of presenting, notes on punch cards were a godsend because their very size forced you to keep to bullet points so you were never tempted to just read your presentation. These days he couldn't do that even if he could find a supply as he so rarely wears a jacket to keep them in.

Before we go wanted to thank those readers of our recent article on tools to help you trace program execution, "Tracing Our Steps," who pointed out alternative tools. We haven't had a chance to check them all out yet but we will do so and report back in the article comments and also here in the blog.

We actually get to spend the next two weeks at home, which will be a rare treat this summer. Who knows? Maybe one or two chores around the house might even get done!