Sign in or 
Share this
Live Case Study: Highly Available Hyper-V and Advanced DR
| Join a Live Case Study discussion presented by key team members who helped HotSchedules deploy a virtualized data center, disaster recovery (DR), and data protection solution to accommodate steady annual business growth. Who Is HotSchedules?HotSchedules provides online workforce management and scheduling for organizations in the restaurant, hospitality, and retail fields to streamline employee scheduling, facilitate communication between managers and staff, and reduce costs. In the face of rapid business growth, HotSchedules wanted to enhance the delivery of its software-as-a-service solutions and help ensure maximum uptime. To achieve these goals, Dell Global Infrastructure Consulting Services helped HotSchedules design and deploy a virtualized IT infrastructure based on Microsoft® Hyper-V™ technology. | |
| Advanced High Availability and Disaster Recovery HotSchedules consolidated servers and implemented advanced high availability and DR with Microsoft Hyper-V and Double-Take® for Hyper-V software. Working with Dell Global Infrastructure Consulting Services, HotSchedules used a multiple-phase approach for consolidating servers and implementing high availability and DR—key drivers for its data center solution—to help reduce power consumption by 77 percent and software licensing costs by 75 percent. In addition to consolidating Dell™ PowerEdge™ servers and Dell PowerVault™ storage, HotSchedules implemented the Microsoft Windows Server® 2008 OS with Hyper-V, the Microsoft System Center Virtual Machine Manager (SCVMM) and System Center Operations Manager (SCOM) suites, and Double-Take® for Hyper-V software to protect critical applications and data. Chat With the Teams That Made It Happen Join this discussion to hear from HotSchedules, Dell, and Double-Take Software team members about their approach and best practices for implementing: Phase 1: Capacity planning and consolidation on Microsoft Hyper-V
| Scheduled Presenters
|
Technical Community – Background Reading
- Visit Ray Weinstein’s Virtualization Frontier blog, “HotSchedules Leverages Dell OpenManage and Microsoft Hyper-V”
- Visit Brace Rennel's Double-Take Software blog, "Dell, Double-Take, HotSchedules Present Hyper-V Live Case Study"
- The DELL.COM Virtualization site provides these links:
Chat Transcript
| KongY-Dell | Looks like the experts are here. Let's get started |
| KongY-Dell | Any questions from our audience? |
| erson | When did you begin phase 1, and when was phase 3 completed? |
| Ray_Weinstein | Good afternoon all! On behalf of Dell and HotSchedules we welcome you to this Techchat |
| matt_woodings | Phase 1 started June 8 last year, and Phase 2 was completed February this year, and Phase 3 is in testing |
| Ray_Weinstein | Please let us know if you have any question about our highly available Hyper-V solution with Double-Take DR |
| KongY-Dell | Welcome Lance |
| brennels | So I understand HotSchedules was looking for a high availability solution to protect their Dell Hyper-V servers |
| Matt_Woodings_HotSchedules | We were looking for a full DR solution that included Hyper-V and SQL Server |
| brennels | Matt, how many Hyper-V servers did you set up originally for phase 1? |
| erson | With lots of customers depending on your Web site being available at all times? |
| Matt_Woodings_HotSchedules | For phase 1 we had approximately 50 servers of which about 40 were stateless application servers that didn't need replicating, leaving 10 critical servers |
| Ray_Weinstein | HotSchedules currently relies on Hyper-V to host its 380,000-user application architecture |
| Matt_Woodings_HotSchedules | We are online 24/7 365 days. We can't afford any downtime |
| erson | Okay, that sets the baseline :) |
| brent_douglas | How did the need for 100 percent uptime affect the design? |
| Ray_Weinstein | HotSchedules partnered with Dell Consulting Services to design, validate, and then protect the entire environment |
| brennels | So is that why you went looking for Double-Take Software to keep your system for Hyper-V highly available, and did you do any testing before you rolled it out? |
| KongY-Dell | Case Study 1: http://content.dell.com/us/en/enterprise/d/corporate~case-studies~en/documents~966-2008-hotschedules-810002930.pdf.aspx |
| KongY-Dell | Case Study 2: www.dell.com/content/topics/global.aspx/casestudies/en/us/us/fy2010_q1_id1194?c=us&cs=555&l=en&s=biz |
| brennels | Using Double-Take for Hyper-V allows the entire Virtual Host server to be failed over, and there were also some stand-alone SQL servers that Double-Take is protecting as well |
| Matt_Woodings_HotSchedules | Double-Take has a long history of virtually real-time replication over the wire, which made it a perfect candidate for off-site SQL and virtual machines |
| Ray_Weinstein | 100 percent uptime was critical; as defined in the case study, we used Hyper-V host clustering to ensure local availability and then used Double-Take for Hyper-V to provide a complete DR scenario for priority VMs |
| brennels | Flexible enough to support both Hyper-V and stand-alone servers with the same product then |
| erson | Is the Double Take replication asynchronous or synchronous? |
| Ray_Weinstein | HotSchedules can now recover from complete primary data center loss in a matter of minutes |
| KongY-Dell | You can click Action, Recent Room History to read what’s been discussed so far |
| brennels | Double-Take is asynchronous and provides byte-level replication as the changes occur |
| erson | Were there any thoughts on using EqualLogic arrays for replication instead of MD3000i and Double-Take software? EqualLogic replications also being asynchronous |
| brennels | The SQL transactions are written in write order preservation to maintain data integrity and with the built in compression; SQL transactions can be compressed upwards of 80 percent to reduce overall bandwidth requirements |
| Ray_Weinstein | Erson, HotSchedules was a Hyper-V early adopter from March 2008, and at that time the MD3000i was the preferred storage solution |
| Matt_Woodings_HotSchedules | Our database is over 300 GB and replicates with Double-Take over a 10 MB pipe to be within a second or so of the actual running database |
| brennels | I believe that Double-Take has been used in combination with EqualLogic to provide additional options for host-based as well as device-based replication |
| erson | Sounds like the Double-Take software gives you smart replication instead of just dumb replication |
| brennels | Matt, that is great performance; must be the strong Dell architecture :) |
| brennels | It is intuitive; here is some additional information about replication on Hyper-V though |
| Ray_Weinstein | Erson, the Double-Take console is very intuitive and powerful, and we were confident within hours |
| Matt_Woodings_HotSchedules | As the Double-Take representative stated, its buffers write in the order they were written, and only writes them on the target after it’s collected them in the order they were written. This prevents corruption and allows the WAN to be a good transport |
| brennels | Because Hyper-V writes changes to the VHD files, they are written to disk in blocks and then replicated in that order |
| KongY-Dell | For more information on the case study, refer to www.dell.com/content/topics/global.aspx/casestudies/en/us/us/fy2010_q1_id1194?c=us&cs=555&l=en&s=biz |
| brennels | This isn't quite the same when using Double-Take on a non- virtualized server because the changes are written directly to the disk and can be replicated at the byte level |
| Ray_Weinstein | Matt, what was the final physical-to-virtual consolidation ratio? |
| Matt_Woodings_HotSchedules | We consolidated 50 physical servers onto three PowerEdge servers with a peak consolidation ratio of about 20:1 |
| brennels | Looks like the case study says there were 50 physical servers that were all virtualized on the three PowerEdge nodes |
| erson | Are there any plans to take advantage of the much improved features of Hyper-V in Windows 2008 R2 that will be released later this summer? |
| erson | 50 VMs on three PowerEdge 2950s |
| Matt_Woodings_HotSchedules | Hyper-V 2 offers some great improvements that most notably is live migration |
| erson | A PowerEdge 2950 can have 64 GB of memory with 8x 8 GB configuration (not cheap). Is that the configuration being used? |
| Ray_Weinstein | Erson, HotSchedules currently has Windows Server 2008 R2 (with Hyper-V 2.0) in production as a Tap customer. Live Migration and CSV has been in production since December 2008 |
| erson | Great. We're testing that same setup right now |
| Matt_Woodings_HotSchedules | Having seen live migration in practice, its very impressive |
| Ray_Weinstein | Excellent, Erson |
| brennels | That is a good point. Because you are on the Hyper-V infrastructure you can easily move the workloads between virtual machines using Live Migration |
| erson | Waiting for the PowerEdge M710 to take things into production. When demoing Live Migration in public for the first time they live migrated a VM playing a 720p video...not a single frame dropped |
| Ray_Weinstein | Erson, true. Failover is within milliseconds |
| Matt_Woodings_HotSchedules | I have tested our application and RDP, of which I think RDP was the best test of it. I had someone typing in notepad while we moved their VM onto another server, only a moments pause was seen |
| brennels | There is a link to a blog post on the chat session main page that has an image of the overall architecture, just can't copy and paste it for some reason |
| brennels | Very cool. Matt, how far away is your disaster recovery data center? |
| erson | Brennels, this pdf from Double-Take has the same image: www.doubletake.com/documents/case_studies/hotschedules_dbtkcasestudy_final.pdf. Being from Double-Take I suspect you already knew that :) |
| Matt_Woodings_HotSchedules | We currently have it on a metro Ethernet link 20 miles from the main data center |
| brennels | It's called working it ;) |
| KongY-Dell | @brennels, here's the link that you mentioned: http://userblog.doubletake.com/2009/05/14/dell-double-take-hotschedules-present-hyper-v-live-case-study/ |
| brennels | Thanks, Kong |
| ranjith_purush | Does your recovery data center also have an identical hardware setup (PowerEdge 2950s and PowerVault MD3000i’s) as your primary data center? |
| erson | When you click the image on that blog, you get to a page when you can get that pdf I linked above :) |
| Matt_Woodings_HotSchedules | Yes, we have two PowerEdge servers and a PowerVault MD3000i at the DR site |
| erson | Is there a backup solution? |
| Matt_Woodings_HotSchedules | I never keep all my eggs in one basket; I also have backups on the VMs and database on site |
| Matt_Woodings_HotSchedules | We also keep tape backups safely off site also |
| erson | What backup solution do you use for the VMs? |
| erson | We're currently testing System Center Data Protection Manager with much success |
| Ray_Weinstein | Erson, cool, using VSS? |
| erson | Keeping all the VMs backed up on a PowerVault MD3000i |
| erson | Ray, yes |
| brennels | Good solution. I see more and more companies rolling out VSS with off-site archiving |
| Ray_Weinstein | Erson, what are your workloads? |
| ranjith_purush | Erson, what kind of servers do you have in your environment? |
| Ray_Weinstein | (applications) |
| erson | We're going to transition four pretty heavily used SQL servers to Hyper-V currently one is used in testing. We're still waiting for three PowerEdge M710 servers so we're using two PowerEdge 2950s while waiting |
| brennels | Matt, are you planning on migrating any more physical servers to Hyper-V, and what are you planning on using to do the physical-to-virtual conversion? |
| erson | We have about 45 employees with Small Business Server 2008 with plans to migrate to Essential Business Server 2008 when we get the new hardware set up |
| Ray_Weinstein | Erson, cool, are you using a tool to baseline the resource requirements? |
| Matt_Woodings_HotSchedules | I currently have two physical servers left to migrate, which I plan to use Double-Take’s new physical-to-virtual offering |
| brennels | Double-Take Move, I assume? |
| erson | Ray, the SQL servers is the heavy workload by far, so we're working on getting some good measurements of the expected I/O |
| Ray_Weinstein | Have you seen the latest Microsoft white paper on Hyper-V and SQL? |
| erson | Yes |
| ranjith_purush | Erson, you are very well positioned with the PowerEdge M710s |
| Ray_Weinstein | It's pretty good |
| erson | About a month old, yes? |
| Ray_Weinstein | Yes, end of December |
| erson | We have an EqualLogic PS6000e as storage for the VMs, btw |
| erson | Matt, what problems have you encountered? Any tips when moving this kind of solution into production? |
| Matt_Woodings_HotSchedules | Up to this point I haven't experienced any problems, but I have seen the solution save my butt |
| Ray_Weinstein | Erson, use PlateSpin Reconn to assess the environment prior to moving it over. This way you can be predictive based on dynamic models |
| Matt_Woodings_HotSchedules | We had two VMs fail on a cluster because of a bad DIMM, but they were migrated to the next node keeping them online |
| erson | Ray, will take a note on that |
| Ray_Weinstein | Nice safety net |
| erson | I'm actually glad that we don't host our software for our customers. It sure is a challenge to host an application like HotSchedules does |
| brennels | How many of the virtual machines do you have using failover clustering? |
| Matt_Woodings_HotSchedules | About 20 machines are in a failover cluster between two clustered nodes |
| erson | What type of memory configuration do you have to handle 20 VMs on a single PowerEdge 2950 (if the other node fails)? |
| Matt_Woodings_HotSchedules | We have 32 GB memory in each one (the maximum at the time). However, currently the most cost-effective size today is 64 GB memory |
| Ray_Weinstein | Erson, we also recommend using Datacenter edition to get the most out of VM guest licensing |
| erson | Not with 11th-generation servers. We're a Microsoft partner so we have a couple of Datacenter edition licenses. 11th-generation with 18 memory slots is cheap with 4 GB for a total of 72 GB |
| Matt_Woodings_HotSchedules | Each of the VMs is 32 bit. I am in the process of testing our environment in a 64-bit realm. So far with awesome results |
| erson | We're going 64 bits on all fronts. Hell, 32 bit is dead on servers anyway. Windows Server 2008 R2 does only come in 64 bit |
| Ray_Weinstein | Good point |
| brennels | Matt, any tips or suggestions when looking for this type of Dell, Double-Take, and Microsoft Hyper-V solution? |
| Matt_Woodings_HotSchedules | I can't speak to everyone, of course, but the MD3000i plus PowerEdge 2950III and Double-Take provides a very cost-effective solution to the SMB company |
| erson | I'm sure Dell is eager to get you a couple of PowerEdge R710s soon enough ;) |
| Matt_Woodings_HotSchedules | The PowerVault MD3000i in my experience is a great workhorse and coupled with Double-Take then provides replication |
| erson | That CPU power is impressive coupled with 18 memory slots |
| Ray_Weinstein | And some PowerEdge R900s :-) |
| erson | And four NICs as well |
| KongY-Dell | Just to remind the audience, the awesome case study on which this chat was based is located at: www.dell.com/content/topics/global.aspx/casestudies/en/us/us/fy2010_q1_id1194?c=us&cs=555&l=en&s=biz |
| erson | As usual, there will be a transcript of this chat tomorrow (or on Thursday) on the Dell TechCenter. Everyone here from the U.S.? |
| KongY-Dell | Yes, except for you...right, Erson ;) |
| Ray_Weinstein | Austin, TX here |
| erson | I actually wasn't the only Swede here last chat |
| Matt_Woodings_HotSchedules | I am actually representing Queen and Country, but live in the U.S. |
| Brady_Lambert | Houston here |
| Ray_Weinstein | Tak (sp) |
| erson | Almost, it's spelled tack :) |
| KongY-Dell | Thank you folks for your participation |
| Ray_Weinstein | LOL, I couldn't recall from my time spent in Stockholm |
| Brady_Lambert | Thanks, see you next week |
| erson | Anyway, great talking to you all. Very nice to hear from a company that actually has implemented something very similar that we're going for |
| brennels | Good talking to you guys, see you next week |
| Ray_Weinstein | Thanks all, cheers! |
| erson | Next time it's VMware vSphere |
| brennels | See ya, Erson |
| Matt_Woodings_HotSchedules | It’s great talking to like-minded people with a need for uptime and DR |
| brennels | Yea Baby... |
| erson | Brennels, yes, I'll be here for sure |
| brennels | Ta |
| ranjith_purush | Thank you. Bye. |
| KongY-Dell | Just as a reminder... transcripts of this chat will be posted tomorrow. Thank you kindly |
| KongY-Dell | Thanks erson as always :) |
|
TDA-Terry |
Latest page update: made by TDA-Terry
, May 20 2009, 9:53 AM EDT
(about this update
About This Update
Edited by TDA-Terry
2096 words added 2 words deleted view changes - complete history) |
|
Keyword tags:
None
More Info: links to this page
|
| Started By | Thread Subject | Replies | Last Post | ||
|---|---|---|---|---|---|
| andreaserson | The chat link isn't working. | 0 | May 19 2009, 3:57 PM EDT by andreaserson | ||
|
Thread started: May 19 2009, 3:57 PM EDT
Watch
|
||||
Showing 1 of 1 threads for this page
