April 01, 2014

DB2 and Hadoop an Unlikely Combo

I recently came across a great SHARE presentation that examines the relationship between DB2 for z/OS. "Big Data Strategies with DB2 for z/OS" provides a nice overview of Big Data with Hadoop before going into some technical benefits of using built-in DB2 user-defined functions (UDF) delivered with Infosphere BigInsights to process Hadoop data from DB2 for z/OS applications.

Here's a sample of the SQL to read Hadoop data:

            SELECT TX.*

            FROM TABLE ( hdfs_Read( ‗ ‘)

                         AS TX (Seller varchar(20)
                                   , Year INT
                                   , Total decimal(15,2));


The capability to enrich operational data with big data sources such as Twitter and Facebook provides powerful benefits to existing applications running DB2 for z/OS, and it will be interesting to see where IBM takes this relationship over the next year or two. I can see some customers bringing the data from Hadoop into DB2 for z/OS and accessing it from DB2 accelerators, while others will choose to store DB2 for z/OS data in Hadoop and conduct local analysis on existing big data. Providing a facility to manage the movement of data and controls around the governance of data being accessed on these different systems will, I believe, be key in the ongoing development of Hadoop support on DB2 for z/OS.