Hyper-V 3.0 high availability and redundancy

Microsoft has taken major steps toward improving Hyper-V 3.0’s high availability capabilities with the addition of predictive failure analysis and increased redundancy.

IT administrators face the critical task of ensuring the integrity and availability of network servers, the importance of which only increases with virtualization. Prior to the advent of server virtualization, a server failure typically affected only a single workload. A failed virtual host, however, may affect dozens of workloads — resulting in a major outage.

Given the importance of high availability in virtual data centers, Microsoft Hyper-V 3.0 includes new features to spot potential errors and add redundancy.

New: Hyper-V 3.0 predictive failure analysis
Predictive failure analysis is a major improvement to Hyper-V 3.0. It allows the Windows Server 8 operating system to support native handling of error correction codes (ECC), which should reduce application downtime.

With ECC support, the OS system memory manager monitors memory pages and the takes the page off-line if the error count exceeds a predefined threshold. It also adds the page to the persistent bad page list, so it won’t be used again.

With Hyper-V 3.0, when Windows identifies a bad memory page, Hyper-V momentarily suspends all virtual machines. If the operating system can isolate the error to a single virtual machine, it shuts down the VM, marks the memory page as bad and restarts the VM. If the OS cannot trace the bad memory page to a single VM,  it resumes all virtual machines. In this case, the potential remains for a fatal error to occur if the page is accessed later.

Improved: Hyper-V 3.0 redundancy technologies
Microsoft also added redundancy to almost every level of Hyper-V 3.0 architecture. The previous version of Hyper-V offers two forms of node redundancy: live migration for planned downtime and failover clustering for unplanned downtime.  This functionality now supports Hyper-V 3.0’s larger clusters.

To ensure failures do not occur as a result of storage I/O problems, Hyper-V 3.0 includes an I/O redundancy feature through network interface card (NIC) teaming.  With this OS feature, administrators can combine multiple network adapters, providing additional bandwidth, load-balancing and failover capabilities.

Previously, Hyper-V NIC teaming was only possible with proprietary  hardware. With native, OS-level NIC teaming, it’s possible to mix and match NICs from different vendors and still ensure that if a single NIC fails, communication can continue through the remaining NICs. Additionally, Hyper-V 3.0 offers multichannel server message block (SMB) and  multipath I/O, which give the server more than one channel through which to communicate with the storage.

More: Hyper-V 3.0 replication capabilities
Hyper-V 3.0 also enables virtual machine (VM) replication, synchronously through integration with storage arrays, or asynchronously through the hypervisor.

In terms of high availability, both replication capabilities create working copies of virtual machines, which you can use if there’s an outage. Though reliable, synchronous replication is susceptible to network latency and is generally considered suitable only when a high bandwidth connection is available between two data centers located in close proximity. Conversely, asynchronous replication isn’t as sensitive to network latency offers much better performance, but it does have the potential for a small amount of data loss.

Hyper-V 3.0’s asynchronous replication capabilities promise to be a boon to cost-sensitive IT shops. Today, it is possible to create host server and virtual machine replicas at the storage level, but this hardware-based approach tends to be expensive and is not application aware. On the other hand, Hyper-V 3.0 replication will create application-consistent virtual machine replicas without the need for additional expensive hardware. This capability assists VMs running applications like Exchange Server because it allows the underlying databases to remain in a consistent state.

The Hyper-V 3.0 replication process also allows you to perform the initial replication either online, over the network, or offline, if you have a slow wide area network. The offline replication process entails the copying the VMs, shipping them to the remote site, loading them on your server and then replicating any changes that have occurred since the copies were originally made. This option reduces replication time for large VMs.

In addition, the replication process supports Windows-integrated and certificate-based authentication, which allow two hosts to mutually authenticate each other’s identity. The replication data also can be compressed and encrypt, which is essential to performance and security.

Overall, Hyper-V 3.0’s predictive failure analysis and redundancy will help ensure high availability –which, in turn, should reduce virtual machine and application downtime.

Exchange Server virtualization improved in vSphere 5

There are many new features in VMware vSphere 5 that supercharge performance, storage and quality-of-service of virtualized servers. Some of the improvements are particularly beneficial for virtualized Exchange servers.

Out of the 140-plus new features in vSphere 5, here are five that offer the most value to Exchange Server virtualization:

Storage Distributed Resource Scheduler

One of the most impressive features in vSphere 5 is Storage Distributed Resource Scheduler (SDRS). A traditional VMware Distributed Resource Scheduler (DRS) automatically places virtual machines (VM) onto servers with low CPU and RAM resource utilization to support the VM requirements. DRS also automatically load balances VMs by dynamically moving them from one host to another if they aren’t receiving necessary resources.

The new SDRS tool performs both these very powerful functions, but for virtual storage. In other words, SDRS:

• Places VM disk files onto shared storage that has the necessary space and storage latency;
• Balances VM disk files across shared storage to ensure optimal storage performance; and
• Balances VM disk files across shared storage to ensure the VM has the space it needs.

If your VM’s data store runs out of space, SDRS moves the VM disk file to another data store that contains the necessary space. Additionally, if a VM’s data store isn’t performing particularly well, SDRS moves the VM disk file to the data store that offers the best performance.

In vSphere 5, VMware also introduces the concept of the “data store cluster” which is simply a group of data stores. You can use data store clusters with or without SDRS.

Exchange servers need proper storage I/O to perform optimally. SDRS automatically resolves storage I/O and storage capacity issues, preventing slowness and outages for your Exchange users.

 VMware vSphere 5 virtual machine file system and VM scalability enhancements

The latest version of vSphere also offers increased scalability for VMs and the virtual machine file system (VMFS). Here are some specific improvements:

• 512 virtual machines per host;
• Up to 160 CPU threads (logical pCPUs) per host;
• Up to 2 TB of RAM per host;
• Up to 32 vCPUs per VM;
• Up to 1 TB of RAM per VM; and
• Up to 2 TB for new data stores.

These enhancements help when your Exchange VMs grow and also when you need to create large files on the VMFS.

 The vSphere 5 Storage Appliance

VMware also introduces the concept of the vSphere Storage Appliance (VSA) in vSphere 5, which is essentially a virtual network attached storage (NAS) option. It is fully supported by VMware for all advanced vSphere features like vMotion, DRS, VMware High Availability (VMHA) and even Site Recovery Manager (SRM). The downside is that you must purchase it separately.

VSA uses local storage from either two or three VMware ESXi servers, plus vCenter. These servers must be identical and fresh installs of ESXi. These servers also can’t have any VMs running on them — not even vCenter. The local storage is presented by the VSA as a network file system NAS share.

VSA is meant for small- to medium-sized businesses that don’t already have a storage area network (SAN) or NAS and use VMware for server virtualization. The VSA is a great fit for remote office/branch office locations where it’s hard to justify the cost of a NAS.

The VSA does offer a unique benefit in that if an ESXi host is lost, VMs running across the VSA keep working without downtime (Figure 1). Thus, VSA offers better high availability and redundancy than a hardware-based NAS/SAN at a much lower price than redundant NAS/SAN.

 

So, how does VSA help virtualized Exchange infrastructures? Well, I’m not sure I’d recommend the new VSA as the single NAS/SAN in a large datacenter with hundreds of VMs – including Exchange – hitting it.

But the VSA is ideal for branch offices of a larger company that require a local Exchange infrastructure. The VSA helps you bypass dedicated NAS hardware, while still achieving high availability, making it a strong option for shared storage.

VMware vSphere replication

Before vSphere 5, you could only protect virtual infrastructures using either VMware Site Recovery Manager (SRM) with a hardware-based SAN or an application-specific recovery tool. Both options were poor value for your investment. You either had to purchase two hardware-based SANs with replication — one for each datacenter — or spend a lot to protect a single application like Exchange Server.

With vSphere 5 and SRM5, VMware announced the option for “host-based replication.” This means that an ESXi server replicates directly to another ESXi server at a backup datacenter, eliminating the need for two hardware-based SANs with replication.

Alternatively, you can replicate from a hardware-based SAN that you may have already invested in to a different SAN at a backup site. This is a huge cost savings for all types of companies because it allows disaster recovery to happen on a per-VM and per-application basis.

VMware sells the SRM5 host-based replication option for under $200 per VM with a minimum of 25 VMs; that translates out to about $5,000. That’s a much better value than the other options.

As you can see, this has the potential to tremendously reduce the cost of protecting virtualized Exchange servers with vSphere 4.1, or even physical Exchange servers.

The vSphere 5 vCenter Server Appliance (vCSA) and vSphere Web Client

My new favorites in vSphere 5 are the vCenter Server Appliance and the new vSphere Web Client.

The vCenter Server Appliance (vCSA) is a virtual appliance you can import into your infrastructure to get vCenter up and running fast. Besides saving time on the Windows install, database install and vCenter application install, vCSA saves money because you don’t have to buy a another Windows Server license.

Not only is it free with all vSphere 5 licenses, but it also enables the new vSphere Web Client by default; you don’t have to install anything. While the vSphere Web Client doesn’t do everything that the vSphere Windows client does, it does do about 80% of what you will need so it’s a nice option for typical day-to-day virtualization admin tasks.

 

VMware said that the vCSA (virtual Linux-based vCenter appliance) and the vSphere Web Client are the direction they are going in the future so we might as well start learning about these new options, now.

As you can see, it’s a good idea to use vSphere 5 for Exchange Server virtualization because it offers innovative features, better scalability, easy administration and the best disaster recovery options. You can learn more about vSphere 5 here.

SQL Server Faster recovery options that provide less or even zero loss of data

SQL Server stores some of your organization’s most important data, so recovering in the event of a devastating failure — like that of the hardware itself — is crucial. Clustering, log shipping and database mirroring offer SQL Server data protection against a server failure. But there is still the opportunity for lost data — the equivalent of a few minutes’ work or, depending on what you’ve built, much, much more. Today many organizations are looking for faster recovery options that provide less, or even zero, loss of data.

One increasingly popular SQL Server recovery technology is the lossless, disk-based backup system. Offered by a variety of vendors, it often works below SQL Server, at the Windows operating system level. Essentially, a locally installed agent monitors for changes to individual disk blocks, collects the changed blocks and sends them to a backup server. The backup server typically compresses and deduplicates the data, and then writes it to its own disk storage. By tracking which blocks change at any given point in time, the backup server can reconstruct the entire SQL Server disk image from any precise moment.

This technique provides two SQL Server recovery models: First, by bringing back the disk image you can access older data and use it to roll back changes without having to resort to slower backup tapes. Second, in the event of a total server failure, the backup server’s most recent disk image can be used to quickly construct a new server — even inside a virtual machine (VM) — a useful technique for whole-site disaster recovery. Some vendors in this particular space even offer “streaming recovery,” which gets you back up and running in just a few minutes; data is restored from the backup server to your recovery server in a stream of data over a longer period of time.

Another lossless SQL Server recovery technique takes advantage of our growing uses of virtualization. With virtualization-based recovery, SQL Server runs in two identical VMs, each running on separate virtualization hosts. Each VM is synchronized in real time, meaning that the memory, processor and possibly even the virtual disk are all in lockstep on both hosts. If a single VM or host fails, the other simply keeps right on processing with no switchover time.

Because SQL Server is such a great user of multiple processors, this recovery technique works best when it supports multiprocessor synchronization. A cleverly designed system can even support really complex failure scenarios. For example, suppose that one virtualization host loses its network connection and the other loses its disk file. Combined, the two can still function as a single server, utilizing the surviving network connection and synchronizing disk state from the machine with the healthy disk. Note that this virtualization approach doesn’t help with damaged or improperly changed data, but it does work well as a high-availability adjunct to a real-time, block-based backup system.

SQL Server’s own availability and recovery options continue to improve. In the upcoming 2012 release, SQL Server’s database mirroring grows up, becoming a true clustering system built upon Windows Cluster Services. This technology provides increasingly sophisticated availability options right in SQL Server, and can be deployed across hosts in organizations moving toward virtualization.

SQL Server’s native weak point is its inability to quickly recover a database to a specific point in time. While transaction log backups can be used to do that, it typically requires a separate copy of the database and a bit more work — and the expertise of a database administrator — than a third-party disk block-based recovery system.

Whatever your data recovery goals, the technologies are out there to meet them. Don’t be afraid to look outside of SQL Server’s native capabilities; Microsoft knows that it can’t meetevery organization’s needs right out of the box and has built a solid and extensible platform for third parties to help meet your specific SQL Server data protection requirements and goals.

 

What is Active Directory’s role in the cloud

In Windows shops, Active Directory is the authoritative user directory that governs access to email, file sharing, and often a broader set of business applications. So, it’s clear why so many IT managers are skeptical about putting AD into the cloud. Simply put, it’s too strategic an asset.

There is an answer to skeptics however, who do not want to insert AD into the heart of their cloud computing strategies. As Software-as-a-Service (SaaS) applications continue to grow, AD in the cloud can help organizations better apply these software packages to their advantage. So, instead of pushing AD to the cloud, it is replicated. The goal here isn’t to outsource the AD function, but rather to make a cloud-ready copy. The purpose of this becomes clear with the application of the technology.

Assuming a corporation uses Salesforce as its CRM tool, that company can now use a cloud-based AD system to tackle the following challenges:

1. Synchronizing user data – As users are added, removed, or moved in various OUs these changes are then automatically reflected in the SaaS application.

2. Single Sign-On – By allowing users to authenticate against their Windows-based password you remove the need to have additional sets of credentials for a SaaS application.

In fact, Microsoft has taken this a step further with something called Active Directory Federation Service 2.0 (ADFS). ADFS uses a set of secure protocols like SSL and Public Key encryption to provide Single Sign On to applications that are not hosted inside your network. This technology can then be applied to Office 365, SharePoint, Windows Azure and more.

Server cloud computing best practices tips

Cloud computing is relatively new, and the industry is still establishing best practices.  However, the idea of the cloud has been around for some time enabling IT managers to learn from some of the more mature technologies.

• Licensing:  IT shops can use cloud environments to mitigate spikes or increases in data traffic by spinning up VMs remotely. If your environment uses a cloud product, make sure it can be installed in this fashion. Certain products restrict licenses to be used from a cloud perspective. This is especially true of commercial Grid, HPC or DataGrid vendors.
• Data Transfer Costs: If you plan on using a cloud provider such as Amazon, make sure you fully understand its  cost model. Make sure the cloud model has a distinct difference between data transferred internally compared to what is transferred externally. In this example, Amazon’s pricing model states that traffic designated as internal is free while anything over an external IP is charged extra.
• Latency: If the work environment has very precise low-latency requirements, then the cloud may not be the best way to go. Remember, using the Internet to transfer data means anything can happen at any time in multiple locations. The cloud server could go down and it may not even be the fault of the provider or environment. It is absolutely essential to understand performance requirements of the environment and have a clear understanding of what is deemed business critical.
• Redundancy: IT teams should always make sure that the cloud service it approaches has some sort of redundancy. When a critical application goes down and then is recovered, many times all local changes will be wiped out and the user has to start with a clean slate. To combat this problem, many cloud service providers now offer something called persistent state storage. This means the data being used can remain linked to a specific computing instance at all times.

Cloud computing continues to advance and stabilize.  In the next few years, IT managers will begin to see definitive advantages to offloading some of their server infrastructure to the cloud. Everyone will have different reasons to adopt cloud services including smaller hardware footprint, cost savings, disaster recover, or business expansion. As virtualization and the concept of service-oriented architectures continue to evolve in the datacenter, the ability to run workloads in agile, scalable environments will eventually put every enterprise into the cloud one way or another.

How to use Diskshadow to back-up Hyper-V

In part one of this series on backing up Hyper-V virtual machines (VMs) , I discussed how to use Windows Server Backup. While effective for backing up some Hyper-V workloads, the approach has its limitations, such as a lack of support for tape drives. Other inexpensive and robust alternatives — such as the Diskshadow utility in Windows Server 2008 — can get you over these hurdles. Diskshadow also enables you to perform online backups of Hyper-V VMs and allows for interaction with various volume shadow copy service writers — and in this case, the Hyper-V Volume Shadow Copy Service (VSS). Simple Diskshadow scripts such as those below will enable you to back up your VMs to disk or to longer-term tape storage.
Diskshadow Script 1: Create a snapshot, expose as drive letter, copy to separate drive and back up using a third-party backup product.

Pros:

This commits and copies a complete online backup to a separate set of disk spindles and aids in the case of primary logical unit number (LUN) or drive disk failures.
Tape backup I/O is not being performed on the same disk spindles as where the VMs reside.
Cons:

Another large LUN/drive is necessary to hold complete copies of VMs.
Time and server resources are needed to first move files to a separate drive and then perform tape backup from that drive. This process could take hours, depending on the number of VMs and the size of their virtual hard drives.
The assumptions of the script:

The Hyper-V role is installed on the host server.
Copy the text below and create a text file named “DiskShadowRobocopyBasic.dsh.”
In this example, the virtual machines reside on D: (if VMs reside on another drive letter or multiple drives, you can adjust).
Create a command file named “backupscript_W.cmd” that copies shadow copies to alternate disk location (see below).
DiskShadowRobocopyBasic.dsh

# Assuming your VMs reside on D:, script cleans old shadows, creates shadows and
# copies files to separate LUN/drive (backupscript_W.cmd), then unexposes drive/LUN.
# Make sure the scripts are in C:vsbackup and that C:vsbackupcab exists, or make the
# appropriate modifications.
DELETE SHADOWS ALL
SET CONTEXT PERSISTENT
SET METADATA c:vsbackupcabBackup.cab
SET VERBOSE ON
BEGIN BACKUP
ADD VOLUME C: ALIAS CP0
ADD VOLUME D: ALIAS CP1
CREATE

EXPOSE CP1 W:
EXEC c:vsbackupbackupscript_W.cmd
UNEXPOSE W:

Backupscript_W.cmd (watch word wrap)

C:VSBackuprichcopy.exe W: e:%computername%W /E $RECYCLE.BIN;SYSTEM*;MP*;$*;Pagefile.sys

To execute the Diskshadow Script above, create a command file like the one below.

VSBackup.cmd
diskshadow /s c:vsbackupDiskShadowRobocopyBasic.dsh

Diskshadow Script 2: Create snapshot, expose as mount point and back up using a third-party backup product that is mount-point-aware.

Pros:

Less disk space is necessary because the process does not take a full copy of the VM.
“Stateful” backup is taken and presented as mount points in a short amount of time, allowing for tape backup within a few minutes.
Cons:

The third-party tape backup system must be able to see operating system mount points. This is sometimes a limitation of backup products.
Backup to tape will use I/O from the same disk spindles as the VMs. Tape backups should be conducted during periods of low I/O on the Hyper-V host.
The assumptions of the script:

Hyper-V role is installed on the host server.
Copy the text below, and create a text file named “DiskShadowMountpointBasic.dsh.”
For this example, the virtual machines reside on D: (if VMs are on another drive letter or multiple drives, you can adjust).
Make sure the path E:MountpointD exists.
DiskShadowMountpointBasic.dsh

*******************************************************************************************************
# Script cleans old shadows, creates shadows and copies files to separate LUN/drive
# (backupscript_W.cmd), then unexposes drive/LUN.
# Make sure the scripts are in C:vsbackup and that C:vsbackupcab exists or make the
# appropriate modifications.

DELETE SHADOWS ALL
SET CONTEXT PERSISTENT
SET METADATA c:vsbackupcabBackup.cab
SET VERBOSE ON
BEGIN BACKUP
ADD VOLUME C: ALIAS C
ADD VOLUME D: ALIAS MP1
CREATE

EXPOSE MP1 E:MountpointsD

To execute the Diskshadow Script above, create a command file like the one below.

VSBackup.cmd
diskshadow /s c:vsbackupDiskShadowMountPointBasic.dsh

The two scripts outlined above are a basic compilation of commands for online backups of virtual machines. Each has its benefits. The script we use for some Hyper-V workloads employs the DiskShadowMountPoint.dsk script because it creates only mount points of the shadow copies instead of actually copying the data to a second set of disks. This saves significant disk space but requires long-term backup systems to see mount points.

Although there are other considerations if you use LUNs/drives without drive letters to house VMs, these basic scripts can go a long way in providing reliable online backups. For detailed versions of these scripts as I have used them in production environments, go to VirtuallyAware.com.

With the increasing popularity of Hyper-V, more and more vendors have incorporated support for the Hyper-V VSS writer, which has made host-level backups of VMs directly to tape much easier. These products are valuable, but they add licensing costs. Other major vendors still do not support the Hyper-V VSS writer, making it necessary to either purchase another backup product or employ some effective, low-cost alternatives. You can try these methods out in your environment, and let me know how they work or if you have any suggestions.

How to use Windows Server Backup for Hyper-V

In my work with Hyper-V beta and with the release candidates, I have needed to recover virtual machines (VMs) that we tested. Most vendor products — even Microsoft Data Protection Manager — did not support the Hyper-V Volume Shadow Copy Service (VSS) writer that allows for online, transparent backups at the host level. So I needed an alternative. Even after some vendors began to support VSS, I continued to use alternatives for particular workloads. Why? They work and provide an excellent free- or low-cost way to provide a stable recovery point for all VMs on a host.
This two-part series focuses on two technologies: Windows Server Backup and the Diskshadow utility, both of which come standard in Windows Server 2008. They work with a non-Hyper-V VSS writer application to provide long-term tape backups. Ideally, an application that is aware of Hyper-V VSS could perform backups on VMs that run without interruption. But if your organization has invested in a backup system that is not VSS-aware or has a large environment where the backup application is handling everything from HP-UX to three generations of Windows, then it may not want to add another backup product. The offerings discussed in this series allow you to use your current backup setup and ensure reliable online backups of VMs using the Hyper-V VSS writer on the cheap.
Tip: Make sure your VMs and hosts are up to date with patches, service packs and integration agents. All three of these will affect how successful you will be with the solutions below.
At the time of this writing, the new integration agents that come with Service Pack 2 for Windows Server 2008 on the host solve almost all the problems I have had with maintaining stable backups on Windows VMs.
Virtual machines without integration agents or those that do not support online backups — such as Windows NT, Windows 200 and Windows XP — will go into a quick-saved state to perform their backups.
VMs with dynamic disks or with any file system other than NT File System (NTFS) will not be able to perform online backups.
Windows Server Backup
Windows Server Backup is a Windows Server 2008 feature that must be installed. It can incorporate a registry modification to the key below, allowing live backups of running VMs using the Hyper-V VSS writer.

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindows NTCurrentVersionWindowsServerBackupApplication Support{66841CD4-6DED-4F4B-8F17-FD23F8DDC3DE}]
“Application Identifier”=”Hyper-V”
This technology provides a good online backup of VMs but poses issues in terms of backing up files to tape with Windows Server Backup. Since Microsoft no longer supports backing up to a tape device with Windows Server Backup, your backed-up VMs must be accessible in the case of a host crash. Since Windows Server Backup does support removable drives and network locations, you could use a large USB hard drive or back up to an alternate server. This solution is not glamorous. But for a test and development environment where you need to retrieve last week’s VMs, it is adequate provided you have plenty of disk space.

Plenty of Microsoft articles and blogs explain Windows Server Backup in greater detail. This tool is a low-cost and easy way to back up Hyper-V workloads. In part two of this series, I will discuss two additional low-cost options that provide greater flexibility and recoverability via the Diskshadow utility in Windows Server 2008.

Hardware requirements for Exchange 2010 Virtualization

When virtualizing Exchange Server 2010, administrators must provision servers with adequate resources, while at the same time adhering to Microsoft’s requirements. It’s something of an art form.

Hardware requirements only scratch the surface. Virtualizing Exchange Server 2010 also requires attention to storage requirements, memory considerations and fault tolerance.

Hardware requirements for virtualizing Exchange

The hardware requirements for Exchange Server 2010 vary greatly depending on the server roles installed on the anticipated workload. As such, the minimum hardware requirements on Microsoft’s website are often completely inadequate.

Determining the hardware requirements for most Exchange Server roles is fairly straightforward. There are numerous TechNet articles that can guide you through the planning process. Hardware planning for the mailbox server role is the most difficult, especially in large deployments. The best way to accurately estimate the role’s hardware requirements is to use the Exchange 2010 Mailbox Server Role Requirements Calculator.

As you plan hardware allocation for virtualized Exchange servers, it is important to remember that Microsoft does not differentiate between physical and virtual deployments of Exchange Server 2010. The hardware requirements are the same regardless of whether you run Exchange on physical hardware or in virtual machines (VMs).

Storage requirements for virtualizing Exchange
Storage requirements are also a consideration when virtualizing Exchange Server 2010. You can configure Exchange Server 2010 to use virtual hard disks that reside locally on the server, or you can connect to a remote storage mechanism through iSCSI. Exchange Server 2010 also supports SCSI pass-through disks.

If you use virtual hard drives for Exchange Server storage, they must be a fixed size. Microsoft does not support dynamically expanding virtual hard drives with Exchange Server. (This isn’t really an issue with VMware, because VMware uses fixed-length virtual hard disks by default.)

If you plan to use iSCSI connectivity, make sure the virtual Exchange server uses a full-featured network stack. If the server does not support the use of jumbo frames, for example, then storage performance may be greatly diminished.

Memory considerations when virtualizing Exchange

xchange has always been an I/O intensive application, but one of Microsoft’s major design goals in Exchange Server 2010 was to drive down the I/O requirements. With fewer I/O requirements, it is more practical to virtualize Exchange mailbox servers. But in order to achieve decreased I/O levels, Exchange 2010 is designed to use large amounts of memory. That means efficient memory use is critical to the server’s performance.

You should avoid using memory overcommit or Dynamic Memory when virtualizing Exchange Server 2010. Memory overcommit works well for servers that occasionally need extra memory to perform an operation and release that memory when the task is complete. Exchange, however, is a memory-hungry application, and the Mailbox Server role usually consumes all the memory that it can.

No snapshots allowed

One of the most important things to know about virtualizing Exchange Server 2010 is that it doesn’t support VM snapshots. Snapshots make a point-in-time copy of the VM to be used for backup and recovery.

In Exchange Server 2010, the Mailbox Server, Hub Transport Server, and Edge Transport Server roles all use highly transactional internal databases. Using snapshots to revert a virtualized Exchange server to a previous state could therefore have disastrous consequences, especially in organizations that use multiple Exchange servers

Fault tolerance

Before the release of Service Pack 1 (SP1), Microsoft didn’t support combining Exchange Server database availability groups with hypervisor-level clusters. Exchange Server 2010 SP1 supports this configuration, but with some limitations. VMs must be configured so that their state is never saved or restored when the VM is taken offline or when a failover occurs.

With all the hardware requirements and resource-allocation considerations, virtualizing Exchange Server 2010 can be a juggling act. VMware (PDF) and Microsoft have released best practices for virtualizing Exchange Server 2010 on their hardware platforms.

 

How VMware and Microsoft server load-balancing services work

Virtual server load balancing among cluster hosts is all about the math. An automated server load-balancing service calculates resource utilization, then compares one host’s available capacity with that of other hosts to determine whether a cluster needs rebalancing.

But it’s not an exact science. Various load-balancing services use different calculation models to determine whether a cluster is balanced. VMware vSphere’s Distributed Resource Scheduler (DRS) feature, for example, uses different metrics than does Microsoft System Center Virtual Machine Manager’s Performance and Resource Optimization (PRO) feature. Ultimately, however, admins need a combination of performance monitoring and calculations before they live-migrate a virtual machine (VM) for load balancing.

Most of us leave cluster load balancing to an automated load-balancing service, but it’s important to understand the calculations that service uses. Understanding these metrics indicates when a load-balancing service should be tuned for better results. Plus, you’re better able to recognize when a vendor’s server load-balancing offering isn’t true load balancing.

 Distributed Resource Scheduler  (DRS)- VMWare Load-balancing Service

The VMware DRS load-balancing service uses two metrics to determine whether a cluster is out of balance. When a host’s current host load standard deviation number is greater than the target host load standard deviation, DRS recognizes that the host is unbalanced with the rest of the cluster. To rebalance the cluster, DRS usually uses vMotion to migrate VMs off an overloaded host.

These server load-balancing metrics reside in the VMware DRS pane inside the vSphere Client. DRS gathers its values by analyzing each host’s CPU and memory resources to determine a load level. Then, the load-balancing service determines an average load level and standard deviation from that average. As long as vSphere is operational, DRS re-evaluates its cluster load every five minutes to check for balance.

If the load-balancing service determines that rebalancing is necessary, DRS prioritizes which virtual machines need to be rebalanced across a cluster. Using the following equation, the service calculates a host’s balance compared with other hosts in the cluster.

The following  equation determines cluster load balancing.

A perfectly balanced cluster reports a zero for its current host load standard deviation. That means the host is balanced with the others in the cluster. If that number increases, it means the VMs on one server require additional resources than the average and that the total resources on the host are unbalanced from the levels on other hosts.

DRS then makes prioritized recommendations to restore balance. Priority-one recommendations should be implemented immediately, while priority-five recommendations won’t do much to fix the imbalance.

Microsoft’s Performance and Resource Optimization – Server load balancing

Microsoft’s System Center Virtual Machine Manager (SCVMM) takes a different approach to cluster load balancing. Natively, it doesn’t take into account aggregate cluster conditions when calculating resource utilization. Its load-balancing service, PRO, considers only overutilization on individual hosts.

You should also note some important conditions with SCVMM. Neither Hyper-V nor SCVMM alone can automatically relocate VMs based on performance conditions. SCVMM can relocate virtual machines only after it has been integrated with System Center Operations Manager (SCOM) and once PRO is enabled. That’s because SCVMM requires SCOM to support VM monitoring.

In SCVMM 2008 R2, if host resources are overloaded, virtual machines can be live-migrated off a cluster host. According to a Microsoft TechNet article, SCVMM recognizes that a host is overloaded when memory utilization is greater than “physical memory on the host minus the host reserve value for memory on the host.” It also recognizes when CPU utilization is greater than “100% minus the host reserve for CPU on the host.”

Neither server load-balancing calculation aggregates metrics throughout the cluster to determine resource balance. But SCVMM uses a per-host rating system that determines where to live-migrate VMs once a host is overloaded. The system uses four resources in its algorithm: CPU, memory, disk I/O capacity and network capacity. You can prioritize these resources with a slider in the SCVMM console.

There’s also an alternative solution for server load balancing: a PowerShell script that analyzes cluster conditions. Running the script balances virtual machines across a cluster by comparing the memory properties of hosts and VMs in the cluster.

Load-balancing services use numerous calculations to determine whether clustered VMs are balanced. But if you don’t understand how your service computes these metrics, server load balancing is tricky. Even if you’re not a math whiz, these metrics help prevent load-balancing problems.

The role of coordinator node in a Windows 2008 Failover cluster

I’m a big fan of Hyper-V but not its underlying clustering technology. The Windows Failover Clustering architecture was originally designed as general-purpose clustering infrastructure, but its management tool sets, such as Windows Failover Cluster coordinator nodes, are hardly a set-it-and-forget-it solution.

Hyper-V’s clustered storage requirements for Live Migration bring the complexity of a Windows Failover Cluster into high relief. For virtual machine (VM) disk file storage, Hyper-V — like its competitors — requires a shared storage area network connection between virtual hosts. Unlike its competitors, however, Hyper-V has more than one shared-storage configuration: with or without Cluster Shared Volumes (CSV).

This article explains how Windows ownership of certain tasks to minimize their time sink.

How Cluster Shared Volumes works

To understand the coordinator node’s role, you must understand CSV — and to grasp CSV, you must also comprehend how a Windows Failover Cluster creates and works with disk resources.

Prior to the release of Windows Server 2008 R2 and CSV, disks that were part of a cluster had to be created as a disk resource within the Failover Cluster Manager console. In this configuration, each disk resource behaved as an individually managed failover unit. Disks, network names, IP addresses, and VM resources were linked together to create a chain of dependencies. When a problem occurred, this linkage allowed each VM resource to simultaneously fail over to another cluster node.

Without CSV, the cluster’s failover unit was the disk resource. Setting the failover boundary as the disk resource failed over the disk data with the disk. Furthermore, any failover of the disk resources also migrated every VM on the same disk. F or most IT shops, this solution was not sufficient. To circumvent this problem, most administrators created a separate disk resource and logical unit number (LUN) for each VM.

CSV eliminates this issue by relocating the unit of failover to the files on a disk instead of to the entire disk. This process enables every VM’s disk file to be the unit of failover. By placing VMs atop a CSV-enabled disk resource, it’s possible to host multiple VM disk files on a single LUN and enjoy the benefits of individual VM failovers.

The role of the coordinator node

At this point, the Window Failover Cluster coordinator node becomes important. In a CSV-enabled configuration, individual files on a disk resource can be owned by different cluster members. But the disk resource that contains these files must also be owned by a cluster member. Microsoft calls the disk resource owner the coordinator node.

You won’t use the coordinator node often. Nearly every VM-to-disk operation occurs from the cluster node that owns the VM directly to its disk. But certain operations must go through a coordinator node, such as copying Virtual Hard Disk (VHD) files to a LUN. This action tends to be disk-intensive, and it can take a long time.

Always copy VHD files to a LUN from the coordinator node. While any cluster node can transparently complete the task, the responsibility is handed off to the coordinator node. As a result, if you initiate the action on a noncoordinator node, it takes more time to complete.

You can alter which server operates as the coordinator node by changing the ownership of the disk resource. Normally, with CSV enabled, you won’t necessarily need to do this task. If you have many file-copy operations, however, save yourself time by using the right node to start your work.

How to create a multi-site Hyper-V failover cluster

Multi Site Hyper-V Cluster Diagram

Before you build a multi-site cluster for Hyper-V failover purposes, you have to take a look at the hardware and storage requirements for highly available Hyper-V environments. After connecting servers to the proper networking and shared storage equipment, building a multi-site cluster is a relatively simple process.

First, use the Validate a Configuration wizard in Failover Cluster Manager to run a series of tests. If everything passes, you can create a Hyper-V failover cluster.

Stretching a cluster across multiple sites isn’t an exceedingly complex process. Today’s Windows Failover Cluster service in Windows Server 2008 R2 already includes most of the necessary components for creating a multi-site cluster, also known as a “stretch cluster” or “GeoCluster.” The only missing component is a mechanism to replicate shared storage between sites.

Single-site clusters limit Hyper-V failover

To truly understand the utility of a multi-site cluster, consider the types of failures against which a single-site cluster protects. In this arrangement, the Hyper-V hosts connect to a piece of shared storage. The “shared” part of this storage is important, because any connections between servers and storage are limited to their maximum cable lengths. Both Fibre Channel and iSCSI storage have a maximum effective distance that limits how far you can physically spread your servers.

While this architecture is excellent for protecting virtual machines (VMs) against the loss of a single host, it does little when an entire site goes down. An outage can occur during a catastrophic event — such as a natural disaster, for example. Or more commonly, it can happen because of a site-wide problem, such as a network or power outage.

During a site-wide failure, a single-site cluster cannot protect against the loss of VM functionality, because the hosted VM and its processing, storage and networking reside at the same location. Therefore, all these components will experience a failure during a site-wide problem.

Creating a multi-site cluster for Hyper-V failover

A multi-site cluster protects cluster functionality by extending it to one or more additional physical locations. Using Windows Failover Clustering, the shared storage contents are copied to a secondary site.

Data replication for cluster storage is typically accomplished through synchronous or asynchronous replication. With synchronous replication, each piece of data that is replicated between the two interconnected storage area networks (SANs) must be confirmed at the secondary site before the next piece of data can be processed. This acknowledgement ensures that the data is transferred between storage devices, thus guaranteeing that the two SANs are always synchronized.

Synchronous replication is excellent for data preservation, but it comes at the cost of performance. Because each piece of transferred data must be acknowledged before the next data is processed, the transfer speed can quickly bottleneck overall performance.

Asynchronous replication circumvents this performance problem by allowing the sending SAN to queue up data that requires replication. Data is then sent in a batch at configurable intervals, with the entire batch acknowledged at once. So asynchronous replication does not pose the performance bottlenecks involved with synchronous replication, but when a site failure occurs, you risk losing some data.

Solutions for asynchronous replication are implemented as features within your SAN storage or as software add-ons to VMs or a Hyper-V host. Each approach comes with benefits and drawbacks. Ultimately, you must weigh the possibility of data loss against reduced system performance to decide which option is best.

Architecting and implementing replicated storage between your two sites is arguably the most difficult part of creating a multi-site cluster. Once the storage is correctly configured, you’ll find the remaining tasks are trivial in comparison, including provisioning additional Hyper-V hosts at the secondary site, adding them into the existing cluster, and configuring failover and other cluster settings to ensure that VMs migrate only during a full-site failure.

Two other considerations that require special attention with multi-site clusters are the reconfiguration of the cluster quorum and the reconvergence of name resolution at the secondary site.

Quorums and Hyper-V failover

A cluster by nature is always prepared for failure. At its core, a Hyper-V failover cluster always watches for components to go down and, when a failure occurs, knows which action to take.

One way the cluster facilitates this task is through a quorum. In essence, a quorum is a collection of cluster elements that determine whether there are enough resources available for the cluster to function.

Quorums use a “voting system” to decide whether a cluster should remain online. There are several ways to configure the voting process:

• counting the number of votes cast by individual hosts;
• counting votes from hosts plus shared storage; or
• counting votes from hosts plus a file share witness in a third and separate site.

When creating a multi-site cluster, carefully consider your quorum options.

DNS resolution after Hyper-V failover

The final consideration for a multi-site cluster is the need for name resolution after a VM fails over from one site to another. Today’s Windows Failover Cluster service has the ability to span subnets (and IP address ranges). This process simplifies a cluster installation, because the network subnets no longer have to span between sites. But when failed-over VMs relocate to a new IP address range, the move complicates name resolution.

In short, when your VMs fail over from a primary site to a secondary site, they fail over to the secondary site’s IP address scheme. As a result, the IP address configuration for these VMs must be reconfigured at the time of failover. Also, clients must flush their local domain name server (DNS) cache to receive the server’s new address information.

Setting up virtual servers to use the Dynamic Host Configuration Protocol for address configuration simplifies their configuration update. For clients, this problem can be resolved by a reboot, clearing their cache with the ipconfig or flushdns commands, or by minimizing the time-to-live  setting for server DNS entries.

While multi-site Hyper-V failover clusters have special requirements for storage and data replication, the extension of your Windows Failover Cluster should not be difficult to set up. With the right technology and good planning, you can extend your Hyper-V high availability to protect against a full-site disaster.