Configuring CA signed SSL certificates for VMware vCenter Single Sign-On in vSphere 5.1 (2035011)

Purpose

Note: This article is specifically for vSphere 5.1 using vCenter Single Sign-On 5.1. If you are using vSphere 5.5 with vCenter Single Sign-On 5.5, see Configuring CA signed SSL certificates for vCenter Single Sign-On in vSphere 5.5 (2058519).

This article guides you through the configuration of Certificate Authority (CA) certificates for the vCenter Single Sign-On (SSO) service on vSphere 5.1. VMware has released a tool to automate much of the described process below.

Before following the steps in the article, see Deploying and using the SSL Certificate Automation tool (2041600).

If you cannot use the VMware SSL Automation Tool, proceed with this article for configuration steps and details for implementing custom certificates in your environment.

The information in this article may also help you avoid some common configuration issues.

Resolution

Note: This article is part of a resolution path. See Implementing CA signed SSL certificates with vSphere 5.x (2034833) before following the steps in this article.

Creating CA assigned certificates for vSphere is a complex task. In many organizations it is required to maintain proper security for regulatory requirements. There are several different work flows required for successful implementation:

• Creating the certificate request
• Getting the certificate
• Installation and configuration of the certificate for Single Sign-On

These steps must be followed to ensure successful implementation of a custom certificate for vCenter server. Before attempting these steps, ensure that:

• You have a vSphere 5.x environment
• vCenter Single Sign-On has been installed and that all components configured
• You have completed the steps in Creating certificate requests and certificates for the vCenter 5.x components (2037432).
  
  Note: This article discusses generating the certificates and supporting files.

Installation and configuration of the certificate for vCenter Single Sign-On

After the certificate has been created, follow these steps to complete the installation and configuration of the certificate.

Note: If you are running vCenter Server in a virtual machine, it is a best practice to take a snapshot before starting this process to ensure that you can revert back to it if necessary. Remember to delete the snapshot after the process is complete.

To replace the vCenter SSO certificates:

1. Log in to the vCenter SSO server.
   
   Note: If you followed the steps in Creating certificate requests and certificates for the vCenter 5.x components (2037432), all components are installed on the same server. All files should be located in C:\certs.
   
2. Double-click the C:\certs\Root64.cer file and import the certificate into the Trusted Root Certificate Authorities > Local Computer Windows certificate store. This ensures that the certificate server is trusted.
   
3. Create a new directory to store the certificates. In this example, the directory used is C:\ProgramData\VMware\SSOCERTS.
   
   Note: You can use any name for this directory.
   
4. Copy Root64.cer file to the new directory from C:\certs.
   
5. Copy rui.crt, rui.key, root-trust.jks, server-identity.jks, and rui.pfx from C:\certs\sso\ into this new directory.
   
6. Open a terminal window on the system where Single Sign-On is installed and run:
   
   SET JAVA_HOME=C:\Program Files\VMware\Infrastructure\jre
   
7. Run this command to list all service entries from the Lookup Service::
   
   SSO_install_directory\ssolscli\ssolscli.cmd listServices Lookup_Service_URL
   
   Where:
   • SSO_install_directory is the directory in which SSO is installed (typically C:\Program Files\VMware\Infrastructure\SSOServer\)
   • Lookup_Service_URL is https://SSOserver.domain.com:7444/lookupservice/sdk.
     
     Note: Be sure to use the Fully Qualified Domain Name (FQDN) for the Lookup Service URL or the command will fail.
     
     The output appears similar to:
     
     
     
8. Locate these three services. You can identify the service by looking at the type field.
   
   
   • Group Check. This screenshot shows example output:
     
     
     
   • SSO Admin. This screenshot shows example output:
     
     
     
   • Security Token Service (STS). This screenshot shows example output:
     
     
     
9. Use a text editor to create three separate properties files for each of the services, replacing the items in red where appropriate. This is an example using the three services above. Ensure the uri= URL is correct to ensure the certificates function properly.
   
   Note: This article uses the C:\certs directory for temporary use.
   
   
   • gc.properties file:
     
     [service]
friendlyName=The group check interface of the SSO server
version=1.0
ownerId=
type=urn:sso:groupcheck
description=The group check interface of the SSO server

[endpoint0]
uri=https://SSOserver.domain.com:7444/sso-adminserver/sdk
ssl=C:\ProgramData\VMware\SSOCERTS\Root64.cerprotocol=vmomi
     
   • admin.properties file:
     
     [service]
friendlyName=The administrative interface of the SSO server
version=1.0
ownerId=
type=urn:sso:admin
description=The administrative interface of the SSO server

[endpoint0]
uri=https://SSOServer.domain.com:7444/sso-adminserver/sdk
ssl=C:\ProgramData\VMware\SSOCERTS\Root64.cer
protocol=vmomi
     
   • sts.properties file:
     
     [service]
friendlyName=STS for Single Sign On
version=1.0
ownerId=
type=urn:sso:sts
description=The Security Token Service of the Single Sign On server.

[endpoint0]
uri=https://SSOserver.domain.com:7444/ims/STSService
ssl=C:\ProgramData\VMware\SSOCERTS\Root64.cer
protocol=wsTrust
     
10. Locate the service ID for each of the three services in the list you generated in step 8. The service ID is located in the serviceId field of the service listing.
    
11. Use a text editor to create a separate service ID file for each of the three services. Using the example here, the three service ID files look like:
    
    
    • gc_id file:
      
      
      
    • admin_id file:
      
      
      
    • sts_id file:
      
      
      
      Note: The files must not contain any other data.
      
12. Open the Services applet and stop the vCenter Single Sign-On service.
    
13. Navigate to the SSO_Install_directory\security. By default, this is C:\Program Files\VMware\Infrastructure\SSOServer.
    
14. Backup the root-trust.jks and server-identity.jks files.
    
15. Copy the new root-trust.jks and server-identity.jks files. These will be located in theC:\ProgramData\VMware\SSOCERTS folder if you used the examples in this article.
    
16. Update Single Sign-On with the new keystore by running the command:
    
    Note: This command uses the example directory structure used in this article.
    
    SSO_Install_directory\utils\ssocli configure-riat -a configure-ssl --keystore-fileC:\ProgramData\VMware\SSOCERTS\root-trust.jks --keystore-password testpassword
    
    Where --keystore-file is the path to the JKS file.
    
    Note: Ensure that the JAVA_HOME variable is still set to JAVA_HOME=C:\Program Files\VMware\Infrastructure\jre, as instructed earlier in this procedure.
    
17. You are prompted to enter the master password that was configured during the installation of vCenter Single Sign-On. If the password is correct, the output is similar to:
    
    C:\Program Files\VMware\Infrastructure\SSOServer\utils>ssocli configure-riat -a configure-ssl --keystore-file c:\ProgramData\VMware\SSOCERTS\root-trust.jks --keystore-password testpassword
Enter master password: ********

Executing action: 'configure-ssl'

Updating SSL configuration

Successfully executed Action: 'configure-ssl'

C:\Program Files\VMware\Infrastructure\SSOServer\utils>
    
18. Start the vCenter Single Sign-On service from the Services applet. You can validate that the certificate is correct by accessing this URL in a web browser:
    
    https://ssoserver.domain.com:7444/sso-adminserver/sdk
    
    Note: An XML error displays. However, the certificate can be checked from the interface.
    
19. These commands apply the new certificates to the three services:
    
    
    • For the STS service, run the command:
      
      SSO_Install_directory\ssolscli\ssolscli updateService -d https://ssoserver.domain.com:7444/lookupservice/sdk -u SSO_administrator -pSSO_administrator_password -si c:\certs\sts_id -ip c:\certs\sts.properties
    • For the GC Service, run the command:
      
      SSO_Install_directory\ssolscli\ssolscli updateService -d https://ssoserver.domain.com:7444/lookupservice/sdk -u SSO_administrator -pSSO_administrator_password -si c:\certs\gc_id -ip c:\certs\gc.properties
      
    • For the Admin Service, run the command:
      
      SSO_Install_directory\ssolscli\ssolscli updateService -d https://ssoserver.domain.com:7444/lookupservice/sdk -u SSO_administrator -pSSO_administrator_password -si c:\certs\admin_id -ip c:\certs\admin.properties
      
      Example output for the STS Service:
      
      
      
      If you do not see a successful Return code, the process was not successful.
      
20. Repeat step 7 to list the three services again and validate that they are correct.
    
21. If you see OperationFailed as a return code, there is a problem with the certificate installation. If you see the OperationFailedreturn code:
    
    
    1. Take a backup of the SSL directory located at C:\ProgramData\VMware\. This folder contains two files,ca_certificates.crt and a hash file, 8_characters.0.
       
    2. Create the new SSL directory for the SSO certificates by running this command:
       
       C:\> mkdir C:\ProgramData\VMware\SSL
       
22. Copy the root certificate from the certification authority to the new VMware SSL directory. If you are following this resolution path, you must copy the C:\certs\Root64.cer file to C:\ProgramData\VMware\SSL\. This certificate is the root certificate for the certification authority that is being used.
    
23. In the C:\ProgramData\VMware\SSL\ directory, back up the current file by changing the file extension on ca_certificates.crt toca_certificates.bak.
    
24. Copy and rename Root64.cer to ca_certificates.crt by running this command:
    
    C:\> more C:\certs\Root64.cer >> C:\ProgramData\VMware\SSL\ca_certificates.crt
    
25. To compute the hash of the Root certificate, run this command:
    
    openssl x509 -subject_hash -noout -in c:\certs\Root64.cer
    
    The output appears similar to:
    
    
    
    Important: The hash must be created with OpenSSL v0.9.8 because this is the version that vCenter Server uses. If created with another version, the hash may be incorrect.
    
26. Create a file named hash_value.0. In this example, the file name is 78835296.0.
    
27. Open the Root64.cer file in a text editor and copy the contents into the hash_value.0 file and save it.
    
28. Repeat this for any other intermediate certificate authorities. In this example, there is only a single authority, so there is only one file. However, if there are intermediate certificate authorities, there will be a file for each intermediate authority with the content of the intermediate certificate in the file.
    
29. If you are using intermediate certificate authorities, you must also append each certificate authority to the ca_certificates.crt file. To do this, run this command:
    
    more intermediateCA.cer >> ca_certificates.crt
    
    Where intermediateCAis the certificate for the intermediate CA. Repeat this step for each intermediate CA in the certificate chain.
    
30. Restart the vSphere Web Client and open Windows Services via the command prompt by running the command:
    
    services.msc
    
31. Click the VMware vSphere Web Client service, then click the Action tab and click Restart.
    
32. Log into the vSphere Web Client as admin@system-domain.
    
33. Navigate to Administration > Sign-On and Discovery > Configuration, then click the STS Certificate tab.
    
34. Click Edit.
    
35. Click Browse.
    
36. Navigate to the SSO Security Directory and select root-trust.jks. By default, this directory is located at:
    
    C:\Program Files\VMware\Infrastructure\SSOServer\Security\
    
37. When prompted, enter testpassword as the password and click OK. The rui key chain is shown in the interface.
    
38. Select rui.
    
39. Click OK.
    
40. When prompted for the password, enter testpassword.
    
    Note: Ensure that you enter only testpassword as the password.
    
    The chain is added, and the certificate is available in the GUI.
    
    Notes:
    
    • If you encounter the An error occurred while updating server configuration error message, this may indicate that the certificate chain was not fully exported. For more information, see step 20 in the Getting the certificate section in Creating certificate requests and certificates for vCenter Server 5.1 components (2037432), which outlines steps to export and concatenate multiple certificates.
      
    • As an alternative to performing this operation in the GUI, you can add the JKS file by running the this command:
      
      ssocli.cmd configure-riat -a configure-sts --keystore-file "C:\Program Files\VMware\Infrastructure\SSOServer\Security\root-trust.jks" --keystore-type JKS --keystore-password testpassword -u admin -p master_password
41. If you have all services on the same server, restart the server for the changes to take effect. If the services are on separate servers or you cannot restart the server, stop and restart the services in this order:
    
    
    • Stop the VMware Log Browser service
    • Stop the VMware vSphere Web Client service
    • Stop the VMware VirtualCenter Server service
    • Stop the VMware vCenter Inventory service
    • Start the VMware vCenter Inventory service
    • Start the VMware VirtualCenter Server service and the VMware VirtualCenter Management WebServices service
    • Start the VMware vSphere Web Client service
    • Start the VMware Log Browser service
      
42. Wait until all the services are started. This normally takes approximately 5 minutes.
    

The SSL certificate for vCenter Single Sign-On (including the Security Token Service, the SSO Admin service, and Group Check) has been successfully updated. Next, continue to install the custom certificates for the Inventory Service.

Source KB 2035011