Twin Cities #tweetup 9/24, Before Minneapolis VMUG

Short and sweet:

Location: Kozy’s Steak and Seafood
Tim: 11:30 – 12:45
What: Uh, a tweetup
How: Hash tag #tctweetup
What else: Five Guys burgers after the VMUG!

vSpecialist’s Delight

Almost forgot to get this out after all of the VMworld excitement. I had the pleasure of a last minute invite to the vGeekFest party at which a video put together by the vSpecialist team was debuted. This was by far the funniest thing I’d seen all week, check it out!

[Lyrics]
Im with Chad’s Army on the virtual patrol
Gotta make sure that my cloud is gold
Analysts say we hit it onn the nose
You from the IT department, Im tha C.I.O.
Adding Ram for your JAVA code
We be celebrating the budget had enough dough
CAP-EX, OP-EX is outa control..
Tweetin’ banana bread because Wade said so
We got beef with security, they don’t use RSA
They don’t know ’bout no S.L.A. SO!
Tier-1 apps running on my assets
vSphere saves ‘em cash for tech refresh,

We don’t know what you take us as,
Or understand the intelligence that Sak-kac has
We are the vSpecialist crew we get the job done…

[Chorus]
We’re number #1
vSpecialist we get the job done
We’re number #1
vSpecialist we get the job done
We’re number #1
vSpecialist we get the job done
We’re number #1
vSpecialist we get the job done

[Verse Two]
The year’s twenty-ten you’ve gots problems again.
Better journey to cloud just like John Glenn
You gots two choices, yall virtualize or
Drop some cash on some datacenter floor
Now I ain’t trying to start a backup race
Data Domain and Avamar will win that race So
I have a meeting with my database guy He says
“I don’t get it… why virtualize?”
Cause it’s fast an’ efficient and it saves me dough and it gives me much mo’
rackspace on my flo’
Should we P2V or should I guess some mo’?
“Well you was runnin’ 10000 vm’s on a vBlock and more “License of application
and step off the console”…
“Are you got a MacBook on you I know a lot of you do”
Yah, Im in Chad’s Army and I know a little bit…
Enough that you won’t login to my kit…
“We’ll see how smart you are when the competitors come”
We are the vSpecialist crew we get the job done…

[Chorus]
We’re number #1
vSpecialist we get the job done
We’re number #1
vSpecialist we get the job done
We’re number #1
vSpecialist we get the job done
We’re number #1
vSpecialist we get the job done

Read more: http://nickapedia.com/2010/08/30/vmworld-2010-rap-vspecialists-delight/

Site Recovery Manager Host Based Replication

Host Based Replication (HBR) was announced at VMworld 2010 “BC8432 SRM Futures: Host Based Replication” as a future, but nearly production ready, feature of the next version release of Site Recovery Manager (SRM). Impressions are that this will not be an additionally licensed feature but will be a native feature of SRM. HBR will be a user selectable object of a virtual machine that allows the elimination of SAN array based replication while enabling protection of the VM. VMware will be implementing technology for native replication between SRM sites for performing full and incremental copies of virtual machine disk objects. Initial configuration of HBR will require a full sync of the vm with subsequent delta copies based on user selectable RPO values between 15 minutes and 24 hours.

Following are some of the highlights I recall:

  • HBR will create (or require?) one or more Protection Groups
  • Protected vms can not be mixed with Protection Groups with array based replication
  • The technology monitoring the deltas of a protected vm is not CBT but something like it
  • Replication is asynchronous, time based and user definable
  • No WAN compression
  • No snapshot/journaling feature
  • Possibly supports replication to/from local datastore
  • Consistency with vm disks
  • No option for consistency across multiple vm
  • No support for RDM (very logical)

I see this feature as being huge for the SMB market as the array based replication is unarguably one of the larger costs of a traditional SRM. I see use case even for larger deployments and full intend to alter my environment for some lower priority virtual machines. For me this will free up licensed capacity on RecoverPoint, minimize disk space used for journaling and allow me to increase the RPO. This all sounds perfect for non top tier, static content application servers or largely read only file servers!

Speculation is that the Beta will be out late 2010 and that the new version of SRM will go GA near the middle of next year. Keyword there is speculation, this might have been from VMware employees but is not authoritative.

VMworld 2010, Day One Thoughts

Day One

I’ve really been trying to go over the events of day one at VMworld 2010 and it’s been a lot of information to digest.  There’s certainly a lot of cool technology and I’m not really sure I understand any of it well enough to really speak authoritatively and I’m actually Ok with that for now.  The real take away for me on this first day has been the people I’ve met.  Hitting up a few different venue I’ve had the opportunity to meet far too many guys I’ve communicated with exclusively via Twitter or through reading blogs.  I don’t think there’s been a single person i wouldn’t love to sit down with, tip a pint and have a lengthy discussion with.  After the crew of tweeps I ended up with a crew from the vSpecialist team at EMC.  I need to say vie been following that team a fair amount since it’s inception and watching the talent being picked up.  These guys really. Are the best of the best in the virtualization, storage and networking disciplines.  Although i didn’t suspect I’d be meeting a bunch of elitist, pretentious guys I certainly wasn’t prepared for the team I met.  Every single one of the people I’ve met have been some of the most gracious, sincere individuals I’ve come across in a long time.

The solutions exchange was awesome but insanely crowded, couldn’t take it for very long.  Due to the crowds, I only attended one session and what I saw I loved.  Stepping into a session on View Performance and Best Practices we ended up being introduced to View Planner, a workload generator, profiler and reporting tool.  Awesome stuff to be seen and is available now, or will be with the GA availability of View 4.5.

Enough technology, day two should be bringing a lot more of that.     

Mike Laverick’s Book Signing at VMworld 2010 – Wednesday, 1st Sept 4pm-4.30pm

Author of Administering VMware Site Recovery Manager 4.0, Mike Laverick, will be doing a book signing at VMworld! Don’t know about you but I’ll be bringing my copy of the SRM book!

VMworld Prep Thoughts

First, this is NOT a complaint. Understood?

Now; Wow, too many sessions to actually grok.  I printed out the entire session list, got a beer, spread list out on bar table and poured through everything that looked interesting.  At some point I made the decision I had to either commit to nothing but the Desktop track or exclude it completely.  I eventually chose the latter even though there is strong interest.  Desktop virtualization just seems as though it will be the easiest of the product line to gather more information on from my local VMware technical team due to VMware’s large push on the product line.

Even excluding the entire Desktop track I’ve managed to fill my schedule and have secondary sessions in case of a full room.  I still haven’t quite figured out what to give up so that I can squeeze in a few hands on labs.  What I’ve gotten from my local SE is that the Advanced Lab Tutorials are really going to be follow ups to the hands on lab sessions that will be going throughout the days.  I need to make the absolute best out of this, I want it all…I think I need to drop some of my expectations and accept that I’m going to miss out on some sessions, some social networking and maybe a bit more.

That said, I’m still more than just excited!  See you there.

VMworld 2010 Schedule

Schedule removed…

VMworld 2010 Events

Tons of social networking events for tweeps and mere mortals are planned for VMworld, see the VMworld 2010 Gatherings & Activities document for complete details.  This is a living document, frequently being updated.  Check often!

Note; Yes, somehow the URL changed recently…

Social Media Contributors at VMworld 2010

Just discoverd that the social media contributors page has been put up for VMworld 2010.  Awesome stuff, follow it for all the dirt at VMworld!

VMworld 2010 is going to be a whirlwind of technology and social activity. Let’s keep in touch! If you are going to VMworld 2010 in San Francisco and use social media (blogs, Twitter, etc), please add yourself to the list below.

We’re going to use this in a few ways at the event:

  • We’ll pull your blog and Twitter feeds from this list to aggregate and feature on VMworld.com.
  • If you’re blogging about VMworld, you may qualify to be an ‘official blogger’ at VMworld 2010, with access to the blogger lounge and blogger/press briefings.
  • Promote your participation and to let others know to look for you at the lounge or around the conference.

vCenter does not support hosts of this type (VMware ESX 4.0.0)

First, let me say that I failed to troubleshoot this alone and ended up making call for assistance.

Environment: vCenter 4.0 U1 with ESX 4.0 U1, two locations.

I encountered an odd error that occurred when I had detached an ESX server from an existing cluster and attempted to add it back in.  The host had come down for hardware repairs, I removed it from the cluster so it wouldn’t show up on my vCheck reports.  Upon attempting to add the server back I received a pop-up from the vCenter client with the message “vCenter does not support hosts of this type (VMware ESX 4.0.0).”  I spent several hours, far too many maybe, troubleshooting this as a host issue but was not finding anything of interest in the logs.  I could see that the vpxuser account was being created, key exchanges occurring but then the vpxa scripts were never installed.  In frustration I rebuilt the ESX server with exactly the same outcome.  I found logging in the ESX server ‘Authd error: 551 There is no VMware process running for config file vmware-vpxa’.  This lead to VMware KB Article 1007337 which indicated that there was insufficient free space on the ESX server file system.  This was not my issue.

Next I pulled another ESX server out of the cluster and attempted to add back without the same result.  Adding these hosts to my other vCenter server was successful.

Finally on the right track I started troubleshooting my vCenter server and found the following relevant logging.

[2010-07-26 16:35:01.592 02728 error 'App'] [VpxdHostUpgrader] Unrecognized bundle version 15. The bundleversion.xml file may be out of date
[2010-07-26 16:35:01.592 02728 info 'App'] [VpxdHostUpgrader] vpxa not reporting correct version. bundleVersion = 15, build = 208111, expected bundleVersion = -1
[2010-07-26 16:35:01.715 02728 error 'App'] [VpxdHostUpgrader] Unrecognized bundle version 15. The bundleversion.xml file may be out of date
[2010-07-26 16:35:01.715 02728 info 'App'] [VpxdHostUpgrader] vpxa not reporting correct version. bundleVersion = 15, build = 208111, expected bundleVersion = -1
[2010-07-26 16:35:01.776 02728 info 'App'] [VpxLRO] — FINISH task-18739 — domain-c34 — vim.ClusterComputeResource.addHost — 07063040-6EC0-44DE-9F79-2E1652BFA653(371258EB-2C3F-49F2-ACB8-147A610F6F0A)
[2010-07-26 16:35:01.776 02728 info 'App'] [VpxLRO] — ERROR task-18739 — domain-c34 — vim.ClusterComputeResource.addHost: vim.fault.NotSupportedHost:
(vim.fault.NotSupportedHost) {
dynamicType = <unset>,
faultCause = (vmodl.MethodFault) null,
productName = “VMware ESX”,
productVersion = “4.0.0″,
msg = “”,
}

So the error is stating that vCenter cannot compare vpx bundle version but what turned out to really be the case is that vCenter could not find it’s own bundleversion.xml file.  In fact, the entire contents of ‘C:\Program Files (x86)\VMware\Infrastructure\VirtualCenter Server\upgrade’ was missing.

Quick fix, since I was fortunate enough to have another vCenter server, was to copy the contents of the directory to my problematic server.  At this point I was able to add the ESX servers into the cluster without incident.  In fact, all seems to be well now and I didn’t even have to restart services.  Now, the real question I need to be asking is “How did the contents of my directory go missing?”

I have asked VMware to review, clean up and publish an internal KB article that lead me to the ultimate solution.