Installing vCenter Single Sign On with SQL Server

In the wake of my post on the introduction of Single Sign On, I noticed many people were searching for installation help. So I figured I’d write up how I did it in my lab.

There are two ways to go about this. The installer gives you the option of SQL Express or using a supported database (MS SQL, Oracle, DB2, or Postgres). In this post, I’ll show you how to install SSO with an existing SQL Server instance. The easiest way to install SSO is to use the SQL Express as the installer handles the DB creation and connectivity. From a backup perspective, an existing SQL Server instance is far better.

For this, you’ll need an existing SQL Server instance. This can run on either the vCenter itself or as a separate server. I won’t bog through installing SQL Server, we’ll just assume you’ve installed it and have it running.

Step 1: Create the database.

Inside [vSphere 5.1 Install Media]\Single Sign On\DBScripts\SSOServer\schema\mssql is a set of scripts that will build or change the Single Sign On database. Open SQL Server Management Studio, connect to your SQL Server instance and drag rsaIMSLiteMSSQLSetupTablespaces.sql into Management Studio.

This will open a new query window in SQL Server Management Studio. We need to change the file locations for the database.

Change this to the location you wish to save the file into. Do this for the other two file location lines. These should all stay together unless you specifically (and in production, should) separate log files from data files. In this case, this is in my lab so I kept them in the same location.

Execute the procedure and a new database called RSA will be created.

We will also need to enable Mixed Authentication. The installer uses a built in user to the database (RSA_DBA) to connect to the database. Right click on the SQL Server in the left pane and click Properties. Click Security in the left pane of the new window and select SQL Server and Windows Authentication mode.

You can close SQL Server Management Studio now.

Step 2: Install Single Sign On

Run the installer from the install media. This can be done via the “Simple Install” (which installs SSO, vCenter Inventory Service and vCenter Server) or by installing just SSO.

Create a new primary node for a vCenter Single Sign On Installation

In this instance, we’ll create a primary node for a new installation. Because we’re doing this, you could add another node later for high availability. Without this, you’re stuck with one node. If that node fails, so does vCenter. If you set up basic, I would suggest running it on the same machine as vCenter as it eliminates a point of failure. The only failure points are the service  failing or the backend database not being available.

Create a password. Make it a good one and save it somewhere safe (like KeePass or LastPass).

Because we’ve already run the SQL script to create the database, we’ll select the Use an existing supported database option. The SQL Express option is far easier, but doesn’t scale up as well and backups are a bit trickier.

Here’s where the fun begins. Set the database type to Mssql, insert the database name of RSA (which is the database the script creates), enter the IP or hostname of the SQL server, and check Use Windows Authentication. Make sure you’re an admin on the SQL server or you’ll get an authentication failure. If you knew a SQL user that had admin rights, you could enter those credentials here.

Click Next on the Fully Qualified Domain Name or IP Address of the host. If it can’t resolve the hostname, that’s OK.

Next, select a user account to run the Security Support Provider Interface server as. I took the default in my lab. If you have an application-only user you prefer to use, here’s where to enter it. I would make sure it has rights to the Single Sign On database. In my case, I was running it on the same machine as Single Sign On so the default was sufficient.

Select the location you wish to install the Single Sign On service to

Select the port you wish to use for the service. Taking the default is preferred here.

Click Install. If you get any errors, run them through the VMware KB. The documentation is well done on common errors. I got a “Error 29115: Cannot authenticate to DB” error, which was resolved by setting up SQL and Windows authentication on the database server.

That’s it! Install the Inventory Service and vCenter Server and you’ll have an up and running vCenter 5.1 install.

If you’re upgrading your existing vCenter and will run SSO with it, I would HIGHLY recommend adding memory to the VM or machine running vCenter. It works with 4 GB but is much more responsive with 6 GB at a minimum, 8 GB if you can spare it. The other option is to split SSO off onto its own VM or machine. You could build a multi-node install for availability this way.

Leave comments below and I’ll try to answer your questions!

How To Re-enable A Disabled Method in vSphere 4.1

I ran into a fun issue with a customer that caused us some consternation with attempting to do some storage migrations from one array to another. Here’s the error we ran into (names changed to protect the innocent):

Descriptive, right?

A quick Google found this KB article from VMware that explained the issue. When your backup program runs, it sets this flag in the vCenter database that prevents migrations from happening so it can take a clean backup. If it isn’t able to remove the flag, migrations fail with this error. This affects the entire VM, not just the individual disks.

Run through the KB article and you’ll be all set!

Changing The Port of the VMware Health Analyzer

Working for a VMware Partner (*ahem* Varrow *ahem*), I get the opportunity to use tools that help us figure out issues in customer environments. One of them is VMware Health Analyzer. It’s a ThinApp appliance that partners can use to compare a customer’s environment against VMware best practices.

When it runs, it starts a Tomcat server instance that runs the Health Analyzer web application. It’s a fantastic tool, but has a default setup that may not play well with other installed web servers (say, your vCenter box). When the application starts, it tries to run the application on port 80. If you already have something running on port 80 (like vCenter), the Tomcat instance will fail. This was an issue for me, but I figured out how to change the port it uses to start on so it can run without disabling services that need to use port 80.

1. Start the ThinApp and let it fail. We need to do this as it creates a folder structure for the Tomcat instance.

2. Go to C:\users\*the user you ran the ThinApp under*\AppData\Roaming\Thinstall\*random identification string*\%drive_c%\usr\share\vha\tomcat\conf

3. Edit server.xml

4. Find ConnectorPort=”80″

5. Change 80 to whatever port number you know is free. 

6. Save the changes and start the ThinApp. 

7. You should see Initializing Coyote HTTP/1.1 on http-(port number you chose)

8. Access the interface from http://localhost:port

Now you should be able to access VMware Health Analyzer from a different port than port 80. Remember to let the ThinApp remain running while you’re working in the Health Analyzer. If you close it, your session will end. 

ESXi 5 on VMware Fusion: Running Nested 64-bit VMs

I’ve blogged about my home lab before, and it’s not on the same scale as many other home labs. In fact, it’s rather small. I use VMware Fusion 4.1 on my Mac and have a pair of ESXi 5 VMs and an Openfiler VM providing the storage. I ran into a slight issue the other day where my machine kernel panicked while the VMs were running and caused a virtual disk failure on one of the ESXi VMs. This meant I had to rebuild it. No big deal, right?

I created a new VM and installed the OS. Put the 2008 R2 (64-bit) VM that was running on the host back onto it and moved on. I received a pink screen of death a few hours later and remembered having the same issue when I first built it. The issue (from this KB article) is a key line is not added to the VMX file of the host VM when the VM is created, even if you select ESXi 5.0 as the guest OS:

vhv.enable = “TRUE”

This line passes the virtualization instructions from the CPU through to the VM. Without it, you can only run 32-bit VMs and that’s no fun at all. Here’s the tricky part (and what the KB article fails to tell you): this line must be added BEFORE you install the OS.  For some reason, if you add the line AFTER installing the OS ESXi does not recognize the instructions are being passed. It’s like it only checks for the instructions once during installation and never again. Once that line is in and you install the OS, everything runs beautifully. Have fun!

UBER Network Fuser: Tips and Tricks

If you follow me on Twitter (@ajkuftic), you probably have read my laments of missing functionality between VMware Workstation and Fusion. Specifically, the lack of a tabbed interface (mostly an organizational complaint) and Virtual Network Editor. I considered going with one network and dealing with it, but then one of the greats, Nick Weaver, came through with a wonderful tool to restore the missing Virtual Network Editor called UBER Network Fuser. If you haven’t been to his blog, you’re missing out on one of the biggest treasure troves of amazing tools to make your home lab awesome. Once you get all of the fun toys, go on Twitter and thank Nick Weaver (aka @lynxbat). One of the most helpful and smartest guys in the business.

Back to the tool. As Nick explains in his blog post, the tool is designed to give Fusion users the ability to create multiple virtual networks beyond the basic NAT, Bridge, and Host Only options. While those are good for most users just wanting to virtualize a Windows/Linux box or two, it’s very limited when trying to build an “Inception” style home lab. What Nick’s tool does is give a nice GUI for configuration options that required knowledge of how VMware Fusion handles its networking setup.

It works REALLY REALLY well. The options are straight forward and everything is well thought through. Nick’s walkthrough is really good and you should absolutely watch it before using the tool. I’ve been using it since the day it came out and it’s made my lab a really well organized place to be. I figured I’d document some tips and tricks here to help others who want to use the tool.

Some Tips:

• Each created network is a full Class C subnet (255 hosts). If you decide to use Nick’s built in option for DHCP, the range starts at 128 and goes to the end of the subnet. If you want to set static IPs on the subnet, set them before 128 and you’ll be fine.
• DO NOT try to edit the config files to add your own DHCP options (like a local DNS box). If you want more granular control past handing out addresses, build a DHCP VM. You’ll need it if you want to mess with PXE or vSphere Auto Deploy anyways.
• The router address for the subnets is 192.168.xxx.2, not .1. .1 responds to ping, but 2 is the gateway.
• The creation tool uses a random number generator to determine the third octet of the subnet. If you’re clumsy and click “Reset to Defaults”, you will lose all of your networks. You will need to create new networks, re-assign the networks to the vNICS, and readdress everything, Yes, I did this. Yes, I tried to get the random number generator to give me back my networks. No, it didn’t work.

One frustration is that it requires Fusion to not be running to make changes. This is to make sure Fusion has control of the network configuration while it’s running. Shutting down your entire lab to make one tiny little change isn’t very fun. Suspending and resuming takes a lot of time when you run more than 3-4 VMs. There are ways to get more granular to the individual VM, but require being VERY careful as you will be editing critical files to your VMs.

To add a vNIC and place it on one of your custom networks, do the following:

1. Shut down the VM. Fusion cannot hot add hardware.
2. Add the vNIC.
3. Go to the location of your VM , right click it and click “Show Package Contents”
4. Find the <VM Name>.vmx file, right click it and click “Open With” and select TextEdit
5. Scroll to the bottom of the file and you should see an area with a bunch of “ethernet1.” options.
6. Add the following two lines:
   1. ethernet1.connectionType = “custom”
   2. ethernet1.vnet = “vmnet10″
      1. This value is determined by going to /Library/Preferences/VMWare Fusion and opening the “networking” file
      2. Look for the subnet that matches to the vNetwork
7. Save the file and close TextEdit
8. Go to Fusion and open the Virtual Machine Library
9. Right click on the VM, click Delete.
10. Make sure to click Keep File
11. Drag the VM back into the Virtual Machine Library
12. Fire up your VM

I hope this helps others get the most out of Nick’s awesome tool. Thanks Nick for putting it together and making Fusion a better place for home labs! If you have things to add to this list, please leave them in the comments!

Changing IPs on your vCenter host

I ran into a few little issues in the lab today that I’d thought I’d document. I was working on migrating my hosts from the vCenter Appliance (which is awesome in my ways) to a more robust, dedicated vCenter VM with accompanying SQL Server. The idea (provided by the great @MBLeib) was to minimize the amount of RAM vCenter would eat and provide a usable platform for other VMware products that utilize SQL Server, like VUM. Also, it allows some target practice at SQL Server without getting my hands too dirty.

Issue #1: Why oh why oh why would my hosts connect to, and subsequently disconnect from, my vCenter server? Over and over again? Seemed like a fun way to get a Tuesday evening started, I guess. As I was moving the servers from one vCenter to another, I decided to rip them away by connecting to them without disconnecting them from their previous (and now offline) vCenter. Strike 1. I then proceeded to disable and enable the Windows firewall in some hope that that would be the fix. Strike 2. I finally fired up the old vCenter and reconnected to the hosts and their connections remained stable. But one little thing popped out at me: the IP of the “previous” vCenter server. It was an odd 192.xxx IP that I don’t use on my home network. I’m staring at my monitor saying “Where did that come from?”. So I checked the settings on the VM running my vCenter server and sure enough, I was using NAT. So I turned that off and changed the IP to something in my home network IP scope. Which leads us to….

Issue #2: VirtualCenter services can be finicky. Once the IP was changed I figured I would fire up the vCenter client and get to configurating (A far more fun word to say than type). No bueno. If you tried to start the service, it would sit and spin. The vpxd.exe process would show up in Task Manager, but not really do anything. It was clearly getting hung up on something. So I started to dig around and hit some log files when I noticed this:

Seems like something is locking up the vCenter database. I hopped over to the SQL server and bounced the SQL services and ta-da, VirtualCenter services were able to start. I’m sure there’s a cleaner way to bounce the old connection from the SQL server, but as this is my lab, I can bounce the services when I want to.

Next up: Storage fun!

Database Fun In The Lab

I got a chance to play around in my lab and rebuild a few things I had been meaning to. One was to separate my vCenter server and its associated database. I split the two into separate servers to give me the flexibility to add other databases on without issue (ex. VUM). I had the SQL box setup and ready to go, but ran into an issue where the installer wouldn’t recognize the DSN I had created. I got an error that read “‘vcenter’ is using an unsupported ODBC driver” and wouldn’t continue. I created the DSN correctly but did not have a 64-bit ODBC driver. So I googled around and found a download for the 64-bit SQL Server Native Client.

Once I installed the driver and recreated the DSN, the install recognized it and allowed me to continue.

Oh, and then I ran into an issue regarding database logging setups that Hersey Cartwright already documented. Set the database logging to simple or deal with a full log drive later!

Backing Up Server 2008 VMs with vSphere 4.1

From: http://www.vmware.com/support/vdr/doc/vdr_110_releasenotes.html

Windows Server 2008 Virtual Machine Backups May Fail If Disk UUIDs Are Enabled
Data Recovery can only back up a Windows Server 2008 virtual machine if it does not have disk.EnableUUID=TRUE in its VMX file. This entry only appears in for Windows Server 2008 virtual machines that were created in vSphere 4.1. Before backing up such virtual machines, power off the virtual machine, and then remove the disk.EnableUUID=TRUE entry from the VMX file. After removing this entry, the virtual machine backs up as expected.

Fun part: This must be done on all Server 2008 VMs, even if you’ve made the fix in a template. Not sure if it affects any storage protocol other than NFS, but it certainly breaks using NFS.

Also, syntax is helpful to remember. Remember to hit “Enter” after removing that line in Notepad or else you can’t power the VM back up.

Double Check The Basics

I’ve been recently tearing my hair out trying to get an HA Netapp filer to use all of its links in a portchannel config using Cisco’s Virtual PortChannel technology.

I’ve been getting an error stating that “VPC detected mis-config between 2 different partners”. Once again, Cisco’s errors are useful and detailed (/sarcasm). I started double and triple checking configs on ports, destroying and recreating the VIFs, trying to team together two NICs instead of all four, anything.

Then running a “show interface status” command caused me to notice something. The ports down were coming from two different heads. I was trying to port channel two different heads, causing the switches to see two different MAC addresses being reported. I went to the rack and moved the cables (that I, admittedly, cabled myself) to the ports they were supposed to be in. Once I did that, the VPC came up just fine.

I owe myself some new hair.

Updating to ESXi 4.1 Update 1

Went against my better judgment and decided to update vSphere and the hosts on a Friday night.

vCenter update went fine. Hosts 1 and 3 went fine. Host 2, not so much.

Got an error called “vim.fault.NoHost”. Tried everything I could think of.

• Disconnected and reconnected the host
• Completely removed the host from the cluster
• Tried different IPs (yeah, no idea why I thought that would work)

Nothing. Then I found this.

Turns out I had an orphaned VM that I didn’t need any more. Once I removed it, the update finished successfully.

One pothole down, millions to go.