RPG Speed Dating

Remember Y2K? It is somewhat mind-blowing to realize just how many years have elapsed since then. We were both working for IBM at the time and were heavily involved in IBM's efforts to provide tools and education to remediate the problem. In fact teaching IBMers and ISVs about IBM's Bypass2000 product took both of us around the world including our first (and to date only) visit to China. It was an interesting time.

One thing that surprised us as we discussed remediation plans with various IBM users was how few intended to take advantage of what we came to know as "real date fields" as part of their strategy. By "real" we mean DDS type L (SQL type DATE) as well as type D fields in RPG. When we questioned the rationale, two major reasons were given:

1. It took up more disk space than the four bytes used to hold a fake date in packed format. In fact this is not true. On disk a real date is held as a 4 byte integer. We just don't get to see it in that format in our programs.

2. A performance penalty was incurred every time a record was read to convert the date from the stored integer to the 8 or 10 character representation that we see in our programs. This of course is true, although much of the time bad RPG programming practice caused the performance hit to be far worse than it needed to be. Our feeling then was that for most shops the advantages far outweighed the performance difference in most cases. We still feel that way.

In recent years we have seen more and more people begin to take advantage of real dates. Many factors are at play in this increased uptake. The increase in the use of SQL and DDL-defined tables is part of the picture. The massive performance improvements in processor and disk speed is another.

But things can still get better, and the good folks on the RPG compiler team decided to add their own performance boost to date- and time-handling. This support actually came about some time ago, but since it was made available via PTF vs. a new release boundary, many people missed hearing about this when it occurred.

This support is available via PTF for both V7.1 (PTF SI43157) and V6.1 (PTF SI43156). It's a new H spec (or Ctl-Opt operation) keyword. The VALIDATE(*NODATETIME) option can speed up programs that do significant amounts of date- and time-handling. It achieves the performance boost by bypassing something you probably didn't even know it was doing - validating _every_ date field _every_ time it is used. Yup, _every_ time it is used.

With this option in play, under some circumstances the compiler will just treat the date as a character field and assume that it contains a valid date and bypass the validation. What circumstances? The following are some examples of cases where validation will not be done. Other circumstances may be added to the list later.

• When one date field is moved to another with the same format and separator. This includes the moves that occur when the field is read from the database, assuming the same format externally and internally, which is not always the case, depending on how the I/O operation is done and/or what the default date format is in the program.
• When a comparison, search or sort operation is being made between *ISO or *JIS formatted dates (i.e., YYYY MM DD) and between times in any format other than *USA. Using this option can have significant performance benefit for programs that handle a lot of dates in these categories. However, remember that some risk is still involved since the compiler will not be validating all date and time values. These are the main points to remember to take the biggest advantage of this new support:
• Make sure your code handles dates and times appropriately and that you are confident of all date values--especially any that may be definted in data structures.
• Strive for date formats and separators that are consistent both in your external database definitions and internally in RPG. Use *ISO (or *JIS) as the format of your date fields wherever feasible to gain the most benefit for comparisons, searching and sorting. Avoid *USA for any time fields.
• Watch out for a DATFMT keyword on H specs (or Ctl-Opt operations). This will change the default format for the program and may inadvertently result in non-ISO formats. If you don't specify DATFMT, the default will be *ISO, which is one of the preferred formats.
• When date data will be moved/assigned in the program, use the same date and time formats and separators as much as feasible. Don't forget that this applies not only to Eval (or heaven forbid, MOVE operations) but also to parameters defined using VALUE or CONST or function return values.
• If your external date fields are not *ISO format, then ensure that those fields are defined in Data Structures so that they will retain their external formats. This way the movement to/from I and O specs by the compiler can use the faster "no validation" option.  

You can get many more details about this support from a notice on the RPG Cafe in DeveloperWorks. On that page, you'll see a PDF attachment with many details and helpful advice on how to make the most of this feature.

The compiler team intends to increase the number of cases where validation is bypassed in subsequent releases, so if you use this option make sure to watch for future announcements in the RPG Cafe that may affect your code.

Comments

[Birgitta Hauser]

Yet an other reason why to work with real date and time values:

SQL provides a lot of scalar functions for date and time caluclation that can be easily used in RPG.

Example:
/Free
Exec SQL Set :MyWeekIso = Week_ISO(:MyDate);
Dsply MyWeekIso;

Exec SQL Set :MyMonthEnd = Last_Day(:MyDate);

Exec SQL Set :NextMonday = Next_Day(:MyDate, 'Monday');

Exec SQL Set :IsWeekEnd = Case When DayOfWeek_ISO >= 6 Then '1' Else '0' End;
/End-Free

Those functions only work with real date/time fields. SQL does not play very nicely with numeric date/time fields.

So why reinventing the wheel.

Birgitta

[John Erps]

I'm always a bit surprised / irritated when the argument is something like this, that it takes more processing time or disk space. In my view in 99% it's just a scapegoat to not follow/implement decent rules (maintainability/correct) because too much time or too difficult etc. In this day of age, especially on IBM i and Power 8 these days, disk space is abundant and cheap (compared to labor), and - especially on IBM i - there is plenty of excess processor time. So i can't really imagine that turning off something like date validation really has a noticeable impact. For me it's an obscure feature which should be only used in very specific circumstances (and really well documented) where you for example want to process a billion records with 20 date fields, in a tight loop or so. And even then. Just 3 years ago i was "advised" by an old geezer - when making an RPG/400 program - to sort the subroutines such that the one most called is at the front. Because it improves performance.... really. And other nice bits such as "use as less work fields as possible". It just means that that person really has no idea, whatsoever, what's happening in the OS (or the compiler in this specific case). I mean, putting the most called subroutine in front only has a noticeable impact if the computer still uses drum memory. The last drum memory that IBM produced was in 1962. :)