Getting Started with Shared Storage Pools

The December AIX Virtual User Group webinar featured Nigel Griffiths' discussion of phase 2 of shared storage pools. If you didn't tune in, download the presentation materials and listen to the replay.

The new shared storage pool functionality is enabled with the latest PowerVM 2.2 service pack, and is a feature of PowerVM Standard and Enterprise. If you already have PowerVM, simply download the VIO server fixpack to obtain these new features. (Note: Because this TL is based on AIX 6.1 TL7, your NIM server must be at AIX 6.1 TL7 or AIX 7.1 TL1 to use your NIM server with your VIO server.)

 One thing to note, as Nigel points out in the presentation, is that the most common VIOS storage options have been around for some time:

1) Logical volumes, created from a volume group and presented to client LPARs
2) Whole local disks
3) SAN LUNs
4) File-backed storage, either from a file system on local disk or a file system on SAN disks
5) NPIV LUNs from SAN

Nigel then discusses the newest option: using SAN LUN disks that are placed into a shared storage pool. This new option, he emphasizes, doesn't eliminate any of the other options. It does not portend the death of NPIV. It's just an additional VIOS storage choice we now have.

Listen to the replay or look over the slides to gather Nigel's thoughts on the benefits of shared storage pools. He explains that fibre channel LUNs and NPIV can be complex. They require knowledge of the SAN switch and the SAN disk subsystem. If you need to make changes, it might take your SAN guys awhile to implement them. This can slow overall responsiveness. That's to say nothing of smaller organizations that don't have dedicated SAN guys. Live partition mobility can be tough work if your disks aren't pre-zoned to the different frames.

With a shared storage pool you pre allocate the disk to the VIO servers. Then it's under your control. You can more easily allocate the space to your virtual machines.

POWER6 and POWER7 servers (including blades) are needed to use shared storage pools. At minimum you should allocate a 1 GB of LUN for your repository and another 1 GB of LUN for data, but in order to be useful, in most cases you'll need much larger LUN(s) – think terabytes of disk -- if you plan to do much with it. 

Your VIOS must have the hostname set correctly to resolve the other hostnames. In Nigel's experience he couldn't use short hostnames -- they had to be changed to their fully qualified names.

He also recommends giving your VIOS a CPU and at least 4 GB of memory. "Skinny" VIOS servers aren't advisable with shared storage pools. Currently, the maximum number of nodes is four, the maximum physical disks in a pool is 256, the maximum virtual disks in a cluster is 1,024, and the number of clients is 40. A pool can have 5 GB to 4 TB of individual disks, and storage pool totals can range from 20 GB to 128 TB. Virtual disk capacity (LU) can be from 1 GB to 4 TB, with only one repository disk.

If you played around with phase one, you'll find that many of your limits have been removed. Now you can use shared storage pools for live partition mobility, perform non-disruptive cluster upgrades and use third party multi-pathing software.

You cannot have active memory sharing paging disks on shared storage pool disks.

Nigel covers the relevant terminology (clusters, pools, logical units, etc.).  He also demonstrates how to actually prepare and set up your disks. In a nutshell you must get your LUNs online and zoned to your VIO servers, and you need to set your reserve policy to no_reserve on your LUNs.

After covering the commands for managing clusters -- cluster –create, cluster –list, cluster –status, cluster –addnode, cluster –rmnode and cluster –delete -- he recommends creating the cluster on one of the VIO servers and then adding additional VIO servers to the shared storage pool cluster. From there, you can allocate space to the VM clients.

Next week I'll have more information from Nigel’s presentation, including scripts and cheat sheets. In the meantime, why not upgrade your test machine's VIO servers to the latest level so you can try this functionality?

Virtualization Webinars Add to AIX Education Offerings

For some time, I've informally collected a few go-to resources for AIX pros. For starters, there's Anthony English's AIX Down Under blog, Chris Gibson's AIX blog and the AIX Virtual User Group-USA. And for sure, get on Twitter.

Others who provide good AIX info include Andy Wojo, Nigel Griffiths, Waldemar Mark Duszyk and whoever is behind AIX Mind. (Feel free to let us know in the Comments section.)

Beyond those, have a look at these great AIX movies. And here's some links to the Quicksheet and QuickStart documents that cover AIX and PowerVM.

Finally, courtesy of Anthony English, here's word of a new webinar series on Power Systems Virtualization from IBM:

"As an IT professional, you may have heard of IBM PowerVM or Power Systems based around the IBM POWER processor. You may even have seen a presentation on it, but have you wondered:

• What is it like to actually use?
• What are the key features for POWER and AIX, Linux for Power and IBM i?
• How will it save me systems administration time and reduce weekend working?
• What do I need to run it and how do I get started?

"What are we planning to do?

• Well .... it is best to let the product talk for itself via a series of live lectures and hands-on demonstration of these features.
• The sessions aim to be about 50 minutes long and roughly once every two to three weeks.

"Who should attend?

• These webinars are aimed at a technical audience (operators, systems administrators and technical specialists) -- people using (or planning to use) IBM's Power based systems.
• Primarily customers, but also available to IBMers and IBM Business Partners."

These webinars are being held during U.K. business hours (hence the euro spelling of "virtualisation"). Currently four sessions are listed; replays are available for the first two, which have already taken place.

Session 1: Exploiting Virtualisation on IBM Power Systems with PowerVM
Session 2: VIOS -- how to get going
Session 3: Controlling processor resources in virtualised partitions
Session 4: Deeper dive into Active Memory Sharing

If you haven't figured it out by now, I'm always looking for more tips and tricks and information. So let me know: Who do you follow? And how do you keep your skills and knowledge current?

As a Term, LPAR isn't that Logical

In an AIXchange blog entry last month, when I discussed the new SDMC IBM Redbook, I noted that:

"Section 1.5 shows us how the terminology will evolve. Managed systems are now called servers, frames are power units, LPARs are virtual servers, the hscroot ID becomes the sysadmin ID, partition mobility becomes relocation, etc.”

Nigel Griffiths took this a step further, recently arguing that the time has come to call our partitions virtual servers or virtual machines rather than LPARs:

"So the observant might have noticed a sharp decline in the term LPAR in the last three or four months. Apparently ... this change is now recommended within IBM and IBM marketing, so you will see a lot more use of the new terms. This is a change in name that I whole heartedly approve- unlike, for example, RS/6000 to pSeries to System p to Power Systems (which now confuses the world’s fastest general purpose computers with mains electricity power supplies!). Of course, 'LPAR' will turn up out of habit on the Internet, in documents and articles for many years to come and be popular with IT luddites now that it is old fashioned. ...

"When I think back to it, the Logical Partition (LPAR) name never did make much sense!

"Logical: means shared or pretend or not physical.
"Partition: means a part of the whole and started life as a disk term as a group of sectors."

Like Nigel, I guess I never gave much thought to the terms "LPAR" and "logical partitions." I was used to them. In my head I always compared LPAR with a hard or dedicated partition or a standalone server.  The dedicated partition would have some sort of dedicated hardware--dedicated processors or dedicated adapters. A logical partition was fully virtualized using virtual or logical devices, disks, network, shared processors, etc. To me it made sense to call it a logical partition since it was using logical devices. Because dedicated and logical partitions could be mixed and matched on a physical frame, I appreciated that this terminology easily differentiated the kind of LPAR we were talking about.

Virtual systems and virtual machines are appropriate terms for this updated technology. But with VM's history, there is potential for confusion. There's not only VM, the old mainframe operating system, but there's VMware, the non-IBM virtualization software. PowerVM is obviously far more powerful than VMware, but again, they have that VM term in common.

I'll certainly try to call them VMs going forward, but don't be surprised if I occasionally make reference to LPARs. And I'm sure I won't be the only one--after all, plenty of customers still tell me about their AS/400 and RS/6000 systems, even though they've been on POWER7 servers for some time. Despite the new direction in terminology, I wouldn't be surprised if we continue to hear about LPAR well into the future.

All that matters really is that everyone understands what we're talking about. Perhaps going forward we'll find a way to differentiate a VM with dedicated adapters versus a fully virtualized VM. Or does that distinction even matter anymore? How quickly do you expect your vocabulary to change?

IBM's Virtualization Alternative

Did you know that when IBM publishes server benchmarks, these workloads always run on virtualized IBM Power Systems machines? The virtualization is built into the hardware and firmware; there is no concept of a non-virtualized, standalone Power machine anymore. Contrast that with offerings from other virtualization solutions running on other platforms that can degrade performance 30 percent just by using their virtualization software solutions.

The previous statements come from a recent IBM presentation. As you likely know, IBM has been at this virtualization game for a generation or so. The company developed the hypervisor that would become VM on the mainframe in 1967. In 1973, IBM was doing physical partitioning.

Here's some more material I gleaned from this training session:

• IBM Power Systems servers provide up to twice the performance of other virtualization solutions on other platforms. These numbers can be even greater depending on the level of virtualization you employ.
• IBM Power Systems servers are scalable, both in terms of being capable of accommodating workload spikes and in allowing an enterprise to grow its business.
• PowerVM technology gives you enterprise quality of service virtualization capabilities with higher performance, more scalability and enterprise security. You can have higher utilization of your machines--around 90 percent--which enables you to consolidate your workloads onto fewer physical servers. You can dynamically move from as little as 1/10th of a core to as many as 256 cores in your LPAR, using all of the resources of your server. You can make dynamic changes to resources like CPU, memory and I/O, and you can add and remove dedicated I/O adapters and storage devices, all without a reboot.
• Live Partition Mobility allows you to easily move running workloads to other frames in your server environment. You can also use LPM to move workloads between POWER6 and POWER7 machines in your environment.
• Using IBM Systems Director, VMs can be moved automatically to any physical machine in your environment, based on the criteria that you set up. If you have a busy workload on one machine, and more capacity available on another machine, Director can move that workload, without interruption and without human intervention, to the less busy machine.
• IBM Power Systems servers are secure by design. No common vulnerability exposures (CVEs) have been reported against PowerVM virtualization by US CERT or by MITRE Corp. In contrast, more than 200 VMware-related vulnerabilities are listed in the U.S. government National Vulnerability Database (NVD). VMware is a third-party software add-on, while PowerVM is integrated into the server firmware. No PowerVM vulnerabilities are currently listed in the NVD. Compare PowerVM virtualization with VMware for instance.
• POWER7 servers offer LPM, live application mobility, partition availability priority, first failure data capture, processor instruction retry, alternate processor recovery, dynamic processor deallocation, dynamic processor sparing, extended error handling and I/O adapter isolation.

The presentation featured a detailed comparison of PowerVM and VMware, making IBM's case that PowerVM virtualization runs workloads more efficiently than VMware, with far superior resource utilization, price/performance, resilience and availability. PowerVM technology outperforms VMware by up to 65 percent on Power 750, running the same Linux workloads and virtualized resources. See this comparison of PowerVM and VMware virtualization performance for more information. In addition, PowerVM on a Power 750 will scale better than VMware with linear scaling that maximizes resource utilization with 4X more virtual CPUs. And compared to a large-tier POWER7 model such as the Power 795, you can have 32X more virtual CPUs than VMware.

Assuming I have my facts right (I borrowed them from the presentation, please correct me in Comments if you disagree with the information) VMware ESX 3.5 allows for four virtual CPUs per VM, 64 GB per VM, 192 VMs on a server, 32 CPU threads on a server and 256 GB on a server. ESX 4.0 allows for eight virtual CPUs per VM, 255 GB per VM, 320 VMs on a server, 64 threads on a server, and 1024GB on a server. PowerVM allows for 256 virtual CPUs per VM, 8192 GB memory per VM, 1000 VMs on a server, 1024 threads per server, and 8192 GB on a server.

With PowerVM technology, you can utilize all CPU cores and all physical memory. Which would you prefer for your enterprise workloads?

Let’s look at flexibility once your VM is running. PowerVM virtualization allows you to make dynamic
changes to virtual CPUs, memory and I/O devices, and have integrated LPAR and WPAR support with PowerVM.

None of this is possible with ESX 3.5. With ESX 4.0, you can add but not remove virtual CPU, add but not remove memory. You can only make some dynamic I/O device changes, and limited direct access to I/O devices.

The same arguments can be made with OracleVM Server for SPARC or HP Integrity VM 4.0. Oracle/Sun allows for Sun Logical Domains on UltraSPARC T1/T2 servers only--they allow for 32 partitions on a T1 or 128 on a T2. You can add or remove CPU, but only add virtual I/O. You can perform warm migrations with constraints. There's no support for dedicated I/O. With HP you can have 8 CPUs max, 64 GB ram. To do dynamic logical partitioning you need to reboot your LPAR. There's no support for dedicated I/O. There's no dynamic CPU sharing.

I still find that some shops simply aren't aware of all that IBM Power Systems servers and PowerVM technology have to offer, and all that they can do. These customers either aren't yet virtualizing their systems, or they don't see the limitations they're under using other vendors' solutions. Hopefully comparisons like these will cause them to take a close look at IBM's alternative.

VIOS Updates

I first heard about an updated version of the virtual I/O server (VIOS) during a recent IBM conference call. Now it's official.

We already use VIOS for sharing disks and networks, active memory sharing and live partition mobility. With these just-announced enhancements, we'll be able to suspend and resume workloads, do more with virtual networks and take advantage of thin storage provisioning and storage pool sharing capabilities.

Here are some announcement highlights, starting with a new feature called suspend/resume.

As I learned in the conference call, suspend/resume is the process of "freezing" an LPAR and saving the complete system state to disk. Then you can restart the workload exactly where it left off, without data loss. The entire LPAR system state is stored in a set of files and can be resumed on either the same server or a different system after migration. After suspension, the server resources are freed up for use by other workloads.

As you can imagine, this feature can make hardware maintenance much easier, because it allows system administrators to perform system updates or CEC upgrades without the need to shut down and restart applications, and without the need to engage application teams to verify that everything is running properly after the restart.

Where live partition mobility allows us to shift resources between physical machines while applications are still running, with suspend/resume, we'll be able to move workloads to another machine (though obviously with an interruption of services). We'll also be able to temporarily suspend low-priority or long-running workloads to allow more urgent processes to access server resources.

For debugging or forensics purposes, IBM states that a workload can be temporarily suspended and a copy made for offline analysis for security or performance purposes. I can't wait to test out this intriguing feature.

More about suspend/resume from the IBM announcement letter:

"Using Suspend/Resume, clients can provide long-term suspension (greater than 5-10 seconds) of partitions, saving partition state (memory, NVRAM and VSP state) on persistent storage, freeing server resources that were in use by that partition, restoring partition state to server resources, and resuming operation of that partition and its applications either on the same server or on a different server.

"Requirements for Suspend/Resume: All resources must be virtualized prior to suspending a partition. If the partition is to be resumed on a different server, then the shared external I/O (disk and LAN) should remain identical. Suspend/Resume works with AIX and Linux workloads when managed by HMC."

Here's what the announcement letter says regarding shared storage pools, VIOS grouping and thin provisioning:

"VIOS 2.2 allows the creation of storage pools that can be accessed by VIOS partitions deployed across multiple Power Systems servers so that an assigned allocation of storage capacity can be efficiently managed and shared. ... Multiple VIOS 2.2 partitions can utilize a common shared storage pool to more efficiently utilize limited storage resources and simplify the management and integration of storage subsystems."

During the conference call the presenters mentioned that this would eliminate the need for vscsi devices or NPIV, but I'll need to do some hands-on testing to understand the functionality better.

"VIOS 2.2 supports highly efficient storage provisioning, whereby virtualized workloads in VMs can have storage resources from a shared storage pool dynamically added or released as required."

It sounds like the thin provisioning that we're used to managing on our storage subsystems can now be managed from our VIO servers. I look forward to testing it out.

"When a new VM is created, the amount of physical storage used is less than the amount defined for the virtual workload, resulting in optimal storage utilization across the shared storage pool. Additional
storage is delivered dynamically when workloads expand and released when workloads contract.  This automates optimized storage utilization, has a more cost-efficient use of storage resources and integrates multiple storage subsystems."

The last thing they touched on in the training was the enhancements to the virtual networking.

"The virtualized network switch functionality within the VIOS will include support for SNMP, networking QoS, dynamic VLAN and MAC access control lists (ACLs). There will be more sophisticated controls for monitoring and tuning network traffic between virtualized workloads. There will be control over networking QoS (quality of service) rules for specific LPARs and you can fine-tune the performance of network-sensitive workloads. There will be support for MAC based access ACLs to allow administrators to impose higher levels of protection for specific workloads."

According to the announcement letter, VIOS 2.2 is set for availability on Oct. 15.

The Lines Blur Between Prod and Test

Recently I was helping a customer implement an IBM PowerHA cluster. We were on the whiteboard going over various failover scenarios. There were going to be two physical servers in the environment, and this question came up: "Are we planning to have one frame be the 'production' frame and the other be the 'test/QA' frame?"

Not that long ago, implementing a test machine alongside a "prod" machine was a given. Hardware simply wasn't as reliable back then. So, to protect themselves from hardware failure, companies would install a hot standby backup along with their production machine -- just in case. Since that backup box typically sat idle, many companies opted to run test workloads on it. At least this way, that second machine was doing something worthwhile.

However, with the advent of Live Partition Mobility and PowerHA -- and with more Reliability, Availabilty and Serviceability (RAS) built into newer hardware -- it's more or less assumed that machines will stay up. And somewhere between then and now, the distinction between prod and test has started to blur.

Almost three years ago I saw my first Live Partition Mobility demo, and I immediately went from skeptic to true believer.

But even now, I find many customers can't quite believe what they're seeing. For instance, a few weeks back I was demonstrating how to move a busy LPAR from one frame to another. The customer had the same skepticism I had back at the beginning: Will it work? Will I drop packets? Is this smoke and mirrors and magic?

Yes, it works. No smoke, no mirrors -- and no dropped packets.

Because you can quickly and easily move workloads around your environment, you're freed from the entire concept of "this frame is production" and "that frame is test." You can concentrate on properly mixing workloads across the environment based on need and available resources. You can create uncapped partitions with proper values for the weights of your partitions. If the machine has free cycles, you can allocate them on a very granular level. If one machine becomes constrained, you can easily shift your workload to another frame that can better handle the load.

When my customer and I were discussing PowerHA and whether they wanted the capability of failing multiple LPARs, a comment was made -- and a lightbulb went on in the minds of those present. What if you set things up the "old way," your production frame dies for some reason, and you need to failover your prod workload? Should the whole environment failover at once, or would it be preferable to have half of prod failover while the other half keeps on processing? After all, in a mixed environment with production LPARs running on different physical machines, losing a frame means only failing a subset of the environment as opposed to the whole thing.

CPU micro-partitioning, PowerVM server virtualization, Live Partition Mobility and PowerHA are all game changers. When we plan for these technologies, we must also rethink the way our systems are implemented. Though it's tempting to still think in terms of standalone systems, alternatives are now possible. Rather than separate prod from test, we may find that mixing production with test on the same frame might make perfect sense.

Note: IBM is hosting a pair of webcasts on future trends relating to Power Systems. Register here and here.

Energy Efficiency Another POWER7 Benefit

More and more companies are becoming sensitive to their energy costs. Though the idea of going green inspires some jokes (if the walls in my datacenter are painted green, does that make me part of the green computing movement?), the reality is that energy costs are high and getting higher.

Rising cost isn't even the whole story. I've seen datacenters that were running at maximum capacity -- they literally couldn't run any more circuits. Obviously consolidation should be a serious consideration in these cases, but even bigger steps are being taken.

A customer recently told me over lunch that some of his company's largest expenditures are energy costs. In response, they're planning to get off of the grid completely. The goal is to use solar and wind power, and to generate energy internally by burning biomass. I've heard other companies are looking to geothermal for heating and cooling.

An interesting and, perhaps, overlooked aspect of IBM's recent POWER7 announcement is the energy savings potential of these systems.

From IBM:

"The new systems can deliver four times the performance and four times the virtualization capability for the same price, and are three to four times more energy efficient...

"IBM Power 750 Express, an Energy Star qualified business server for mid-market clients [offers] four times the processing capacity of its predecessor, the IBM Power 550 Express...

"POWER7 technology features 'Intelligent Threads' that can dynamically vary based on workload demand. With more threads, POWER7 can deliver more total capacity as more tasks are accomplished in parallel, such as monitoring the energy usage of millions of households by the minute in a smart grid. With fewer threads, those workloads that need very fast individual processing -- such as real-time analytics or database transactions -- can get the performance they need for maximum benefit. Intelligent Threads work on all POWER7 processors...

"IBM's POWER7 systems are designed to make dramatically better use of energy. Unique Intelligent Energy technology allows customers to power on and off various parts of the system or to dynamically increase or decrease processor clock speeds based on thermal conditions and system utilization, on a single server or across a pool of multiple servers. POWER7 energy management technologies are integrated from its processor, to firmware, PowerVM virtualization, operating system support, and up to IBM Active Energy Manager software, included in the new IBM Systems Director Standard and
Enterprise Editions. As a result, the system dynamically balances between energy usage and performance and systems utilization based on policy. The result is improved performance per watt -- more than two-times better than similar Intel x86-based systems, four times better than Sun SPARC servers and eight times better than similar HP Itanium-based servers."

As we dig into the details of the POWER7 announcement and get excited about the performance gains we can expect, don’t overlook the greater energy efficiency. It's one of the biggest benefits we'll see as we upgrade our hardware.

More on Migrating to Virtual Storage

The now published IBM Redbook, "PowerVM Migration from Physical to Virtual Storage," contains a wealth of valuable information. Here are a few more highlights.

* Choosing a SAN disk -- The authors of the Redbook detail the various methods for determining which SAN disk to use when virtualizing: Physical Volume Identifier (PVID), IEEE Volume Identifiers and Unique Device Identifiers (UDID).

* The new chkdev command -- Lots of material here. As the authors state:

"As of Virtual I/O Server Fix Pack 22, a new command has been introduced to assist with the identification of disks and their capabilities. The Virtual I/O Server chkdev command is capable of displaying the same values as mentioned previously (IEEE, UDID, and PVID) but provides some additional information:

#chkdev
NAME:                hdisk0
IDENTIFIER:          2F1135000C50006D9C1F30BST9146802SS08IBM-ESXSsas
PHYS2VIRT_CAPABLE:   YES
VIRT2NPIV_CAPABLE:   NA
VIRT2PHYS_CAPABLE:   NA

* What can be virtualized? Again, I'll let the authors tell you:

"PHYS2VIRT_CAPABLE: This disk may be virtualized to a logical partition. Once this is performed, this field will change to a value of NA if the mapping is successful. A value of NO indicates this volume may not be virtualized.

"VIRT2NPIV_CAPABLE: If the disk is capable of moving from a virtual SCSI environment to an N_Port ID Virtualization (NPIV) environment, this field will be set to YES, otherwise it will be set to NO. A value of NA means that this disk has already been moved and the Virtual Target Device or VTD as it is abbreviated in the command output, will indicate the mapping.

"VIRT2PHYS_CAPABLE: If the device is capable of moving from a virtual environment to a physical environment and is currently mapped to a VTD then the value here will be YES. A value of NA means the disk is not in use by a VTD, while a value of NO means the disk is not capable of such a move. For further information, please refer to the Virtual I/O Server chkdev manual page."

* It figures -- The Redbook also includes some nice charts (see the Figures section in the table of contents) that help make concepts clear. For example, Figure 2-1 shows the relationship between physical SCSI disk and the target SCSI device on the Virtual I/O Server. Figure 2-2 shows the relationship between physical SCSI disk and the virtual SCSI devices on a client partition.

* NPIV -- As we reduce our reliance on physical adapters, and with more customers adopting NPIV, it's nice to see this topic covered:

"N_Port ID virtualization (NPIV) is a technology that allows multiple logical partitions to access independent physical storage through the same physical Fibre Channel adapter. Each partition is identified by a pair of unique worldwide port names enabling you to connect each partition to independent physical storage on a SAN. Unlike virtual SCSI, only the client partitions see the disk. The
Virtual I/O Server acts only as a pass-through managing the data transfer through the POWER Hypervisor."

Finally, there's this interesting note on page 29:

"A POWER Hypervisor has a limit of 32,000 pairs of WWPNs. If you run out of WWPNs, you must obtain an activation code for an additional set of 32,000 pairs.

"Each time you configure a virtual Fibre Channel adapter, whether dynamically or by adding to a partition profile, the HMC obtains a new, non-reusable, pair of WWPNs from the POWER Hypervisor. Therefore, the correct procedure for dynamically allocating a virtual Fibre Channel adapter to an active partition that must keep the configuration across a partition shutdown is to first dynamically allocate the adapter to the partition and then to use the HMC Save Current Configuration feature to save the configuration to a new profile. This new profile then must be used to start the partition after a shutdown. This will ensure that the WWPNs that were allocated during the dynamic operation will be the same ones in the profile. If instead, you dynamically add an adapter and then add an adapter to the
partition profile, the partition will come up with a different pair of WWPNs after a partition shutdown and access to the storage will be lost."

This summary -- and this, and this -- barely scratch the surface when it comes to summarizing the valuable information in this publication. So take the time to read this Redbook.

Also, some interesting response to the POWER7 announcement:

* POWER7 facts and features

* "POWER7 has the most remarkable on-chip cache hardware of any processor on the market."

* Smarter systems for a smarter planet

* POWER7 does not validate Sun's CMT SPARC Processor Architecture

* Active Memory Expansion Performance

* Active Memory Expansion Overview and Usage

* Power your planet with Power Systems

Active Memory Sharing Takes the Guesswork out of Partitioning

Like I wrote in AIXchange a few weeks ago, I was minding my own business when someone tweeted a link to a new IBM developerWorks article by Chris Gibson, "Configuring Active Memory Sharing: From a customer's experience."

I'd seen the movies that Nigel Griffiths had done based on this topic:

http://download.boulder.ibm.com/ibmdl/pub/systems/power/community/aix/AIX_Movies/AMS_RegularPaging.wmv

http://download.boulder.ibm.com/ibmdl/pub/systems/power/community/aix/AIX_Movies/AMS_Concepts.wmv

http://download.boulder.ibm.com/ibmdl/pub/systems/power/community/aix/AIX_Movies/AMS_Setup2.wmv

http://download.boulder.ibm.com/ibmdl/pub/systems/power/community/aix/AIX_Movies/AMS_Monitor2.wmv

I'd also read the IBM Redpaper, "PowerVM Virtualization Active Memory Sharing."

Chris' article, though, is yet another great source of information about this additional functionality. PowerVM Active Memory Sharing allows you to virtualize memory on your frame. You can overcommit physical memory and allow the memory to flow between logical partitions and paging space.

Virtualized memory operations aren't as instantaneous as virtualizing your CPU. While your micro-partitioned CPU makes changes on a millisecond basis, memory sharing uses paging space, so it's obviously orders of magnitudes slower as it reads and writes to disk. So if you have several logical partitions that are all starved for real memory at the same time, this isn't for you. However, it is great if your LPARs don't simultaneously demand memory. When you create an LPAR and allocate some
amount of memory, it's possible, even likely, to guess wrong. Allocate too much memory and it goes to waste. Allocate too little and you might run into performance issues. With Active Memory Sharing however, the system determines which LPAR actually needs the memory. This makes more efficient use of your resources.

From the Redpaper:

"Active Memory Sharing allows overcommitment of memory resources. Since logical memory is mapped to physical memory depending on logical partitions' memory demand, the sum of all logical partitions' logical memory can exceed the shared memory pool's size. Each logical partition is allowed to use all assigned logical memory. When the cumulative usage of physical memory reaches the pool's size, the hypervisor can transparently steal memory from a shared memory partition and assign it to another shared memory partition. If the removed memory page contains data, it is stored on a paging device and the memory page content is cleared before it is assigned to another logical partition. If the newly assigned page contained data, it is restored from the disk device. Since paging disk activity has a cost in terms of logical memory access time, the hypervisor keeps track of memory usage to steal memory that will likely not be used in the near future. The shared memory partition's operating system cooperates with the hypervisor by providing hints about page usage and by freeing memory pages to limit hypervisor paging."

Read Chris’ article and the Redpaper, and watch the movies. Think about the different ways that this functionality can be useful in your environment. Look at the list of requirements.

Again, from the Redpaper:

"In order to use the Active Memory Sharing feature of IBM PowerVM, the following are the minimum requirements:

"An IBM Power System server based on the POWER6 processor
Enterprise PowerVM activation
Firmware level 340_075
HMC version 7.3.4 service pack 2 (V7R3.4.0M2) for HMC managed systems
Virtual I/O Server Version 2.1.0.1-FP21 for both HMC and IVM managed systems
AIX 6.1 TL 3

"Shared memory partitions can be created on a system as soon as the shared memory pool has been defined. In order to be defined as a shared memory partition, the logical partition must meet the following requirements:

• Use shared processors.
• Use virtual I/O, including: virtual Ethernet adapters, virtual SCSI adapters, virtual Fibre Channel adapters, virtual serial adapters

The operating system running in the logical partition can be either AIX, IBM i, or Linux."

Start planning now for any necessary upgrades so that you can take advantage of this new way to manage your machines.

IBM Hardware Fits in the Big Picture

I've worked for numerous companies that were IBM customers. For several years I worked for IBM in Boulder, Colo. I currently work for an IBM business partner. Through all this time, I've been happily using IBM racks with IBM servers. Sure, there were other servers running other operating systems that used different racks in the various computer rooms, but the IBM servers all lived in IBM racks.

I was recently on a raised floor at a server co-location company and watched IBM customer engineers move a POWER5 Model 570 from an IBM rack to a non-IBM rack. To install the server, they had to carefully measure its required depth. To make the flex cable on the front of the machine fit into the rack, they had to modify the side of the rack due to some protruding metal on the side doors that was interfering with the flex cable. Obviously, the whole operation took longer than a regular rack install into a standard IBM rack would have taken.

While they got the server to work in the non-IBM rack, it wasn't an ideal situation. The power distribution units in the back of the rack were rotated compared to the IBM rack, and the plugs were oriented so that when the servers were plugged in, power supplies and PCI cards couldn't be swapped without removing the entire PDU first.

So why go to this trouble? These non-IBM racks are taller and have a smaller footprint than the previously used IBM racks. When you're paying by the square foot, you want to squeeze in as much equipment as you can. By using extra space at the top of the rack, the co-locator could fit more equipment into its floor space. I can certainly understand that with several taller, narrower racks, you can wedge more racks into a row of hardware, and that can translate into more revenue for the operators of the shared environment. (This might not matter so much in a customer-owned data center, but in this shared environment, it's critical.)

One of the facility managers said that IBM just doesn't get co-location. He argues that every other vendor can make its equipment fit into these racks, why can't IBM?

My take is that IBM engineering designs IBM racks with IBM hardware in mind. They're rugged. They're built to fit without modification, and they're built so that parts can be easily hot-swapped in and out of the machines. If IBM racks don't fit nicely into your cookie cutter server room that was designed with non-IBM servers and racks in mind, if they're wider than other enclosures, then maybe it's time to rethink the design of the room. The problem as I see it is that many customers think that everything is the size of an x86 machine. However, they should plan for equipment of other sizes. It's not necessary to redesign the computer room--but maybe designating an aisle for nonstardard-sized racks would give them some flexibility.

The thing is, the greater serviceability of IBM solutions is well worth the sacrifice of some floor space. IBM engineering designed and built its racks with IBM servers in mind. Using PowerVM and IBM Power Systems servers, you can consolidate multiple large workloads onto a much smaller footprint. And isn't creating the smallest footprint the goal here? Yes, IBM racks might take up an extra square foot or two compared to non-IBM racks, but this should be easily offset by an overall reduction in server hardware.

I believe IBM keenly understands co-location. They've given me hardware that I can easily carve into
multiple LPARs per physical frame, which allows me to consolidate workloads. The technology is very forgiving and flexible--as workloads grow and change, it's very easy to use a dynamic LPAR operation and make the adjustment.

Ultimately of course, the customer is always right. So if you need to make your server fit into a smaller footprint on your raised floor, you can and should do so. But first, reconsider what you're giving up by making this change. Saving physical space doesn't always equate to saving money.