April 23, 2013

IBM Hosts a Big Day for Big Data

I was honored to be invited to IBM Almaden research center in San Jose, California, for a briefing on IBM’s Big Data launch earlier this month. Hundreds of DB2 community luminaries -- including Information Champions, analyst and DB2 Gold Consultants -- were brought together to absorb all this information and then share it with the IT industry through their blogs, articles and tweets.

I learned a lot about what is going on in the distributed world from the discussion of tools like Hadoop as well as the latest DB2 for LUW release. DB2 for LUW 10.5 features a technology called BLU, which supports data stored in columnar format rather than row format. That is, the data page contains data for a given column where row format is rows of data that are stored on a page of data. This column format can dramatically increase the speed of analytic queries.

Of course, with my focus on the mainframe, I wanted to learn more about big data on z/OS. The starting point here was the latest release of DB2 Analytics Accelerator (IDAA) for z/OS, IDAA V3.1.

Prior to IDAA V3.1, data had to be stored on the local host and replicated to the IDAA (Netezza). However, IDAA V3.1 features a High-Performance Storage Saver (HPSS), which allows you to store a DB2 table partition or full table of data solely on the accelerator, reducing the need for host storage. This feature removes the requirement for data to be replicated in DB2 storage and Accelerator storage. Tables can now be divided between traditional database resident partitions on DB2 for z/OS and HPSS on the IDAA.

What are your company's plans for big data? How do you see big data impacting your job as a DBA? Please share your thoughts in comments.