Quickly configuring vCenter alarms using PowerCLI

To quickly configure all vCenter alarms to send an email, use this snippet:

Get-AlarmDefinition | New-AlarmAction -Email -To 'admin@example.com'

Once that is in place, set up an AlarmActionTrigger to send an email when the monitored item moves from green to yellow. The default AlarmAction created earlier already contains a setting for moving from yellow to red:

Get-AlarmDefinition | Get-AlarmAction -ActionType SendEmail | where-object {$_.To -like 'admin@example.com'} | New-AlarmActionTrigger -startstatus 'Green' -EndStatus 'Yellow'

To have the alarm action repeat, use the -Repeat flag on the New-AlarmActionTrigger.

To add other transitions, use the New-AlarmActionTrigger and change the -StartStatus and -EndStatus to the desired transition. For example, -StartStatus ‘Red’ -EndStatus ‘Yellow’ will set the alarm to notify “Once” when moving from Red down to Yellow.

To see all Alarms configured to send to a certain email address along with triggers:

Get-AlarmDefinition | Get-AlarmAction -ActionType SendEmail | Where-Object {$_.To -like 'admin@example.com'} | Select AlarmDefinition,To,Trigger

To remove all SendEmail AlarmActions for a particular email address:

Get-AlarmDefinition | Get-AlarmAction -ActionType SendEmail | where-object {$_.To -like 'admin@example.com'} | Remove-AlarmAction

Further information at https://pubs.vmware.com/vsphere-51/index.jsp?topic=%2Fcom.vmware.powercli.ug.doc%2FGUID-3758D090-C799-4BA0-8F6C-1C2E7A055D1B.html

37,741 total views, 10 views today

Bulk configuration of HP iLO4

Found myself in a situation where I had 38 new HP DL380 Gen8 servers that needed their iLO4‘s configured to match the standard. While I’m always a fan of repetitive point and clicking (not really), I decided my time would be better spent finding a more efficient way to configure them all at once.

Enter HPQLOCFG.EXE: The HP Lights-Out Configuration Utility. Installed it on a random Win2k8 R2 VM and then downloaded the accompanying scripting examples. Coupled with the HP iLO 4 Scripting and Command Line Guide, it becomes fairly easy to cobble together a master script which remotely fully configures an iLO4 from scratch knowing only the DNS hostname and default Administrator password. Obviously DHCP comes in handy here on the iLO network so the iLO4 can get on the network without ever touching the server.

Reviewing the XML scripting examples, there are many good ones for doing individual configuration of certain features of the iLO4 using RIBXML. Putting them all together in to one script is a fun exercise, but the examples are good and the PDF guide is helpful.

The way it all works is an XML file is prepared by you and then passed to HPLOCFG.exe along with remote iLO4 hostname, Administrator username, and default Administrator password.

Here is an example XML file which fully configures features of the iLO4 to meet the customer’s standard:

<RIBCL VERSION="2.0">
	<LOGIN USER_LOGIN="adminname" PASSWORD="password">
		<RIB_INFO MODE="write">
			<LICENSE>
				<ACTIVATE KEY="1111122222333334444455555"/>
			</LICENSE>
			<ADD_FEDERATION_GROUP GROUP_NAME="Servers" GROUP_KEY="servers">
				<ADMIN_PRIV VALUE="Yes"/>
				<REMOTE_CONS_PRIV VALUE="Yes"/>
				<RESET_SERVER_PRIV VALUE="Yes"/>
				<VIRTUAL_MEDIA_PRIV VALUE="Yes"/>
				<CONFIG_ILO_PRIV VALUE="Yes"/>
				<LOGIN_PRIV VALUE="Yes"/>
			</ADD_FEDERATION_GROUP>
			<MOD_SNMP_IM_SETTINGS>
				<OS_TRAPS value="Yes"/>
				<SNMP_PASSTHROUGH_STATUS value="No"/>
				<RIB_TRAPS value="No"/>
				<CIM_SECURITY_MASK value="3"/>
				<SNMP_ADDRESS_1_ROCOMMUNITY VALUE="public"/>
				<AGENTLESS_MANAGEMENT_ENABLE value="Yes"/>
				<SNMP_SYS_CONTACT VALUE="Server Team"/>
				<SNMP_SYS_LOCATION VALUE="XYZ Corp"/>
				<SNMP_SYSTEM_ROLE VALUE="Server"/>
				<SNMP_SYSTEM_ROLE_DETAIL VALUE=""/>
				<COLD_START_TRAP_BROADCAST value="Yes"/>
				<TRAP_SOURCE_IDENTIFIER value="iLO Hostname"/>
				<SNMP_ACCESS_ENABLED value="Yes"/>
				<SNMP_PORT value="161"/>
				<SNMP_TRAP_PORT value="162"/>
				<SNMP_V1_TRAPS VALUE="Yes"/>
			</MOD_SNMP_IM_SETTINGS>
			<MOD_GLOBAL_SETTINGS>
				<REMOTE_SYSLOG_ENABLE VALUE="Yes"/>
				<REMOTE_SYSLOG_PORT VALUE="514"/>
				<REMOTE_SYSLOG_SERVER_ADDRESS VALUE="%syslogIP%"/>
				<ALERTMAIL_ENABLE VALUE="Y"/>
				<ALERTMAIL_EMAIL_ADDRESS VALUE="support@example.com"/>
				<ALERTMAIL_SENDER_DOMAIN VALUE="example.com"/>
				<ALERTMAIL_SMTP_SERVER VALUE="mail.example.com"/>
				<ALERTMAIL_SMTP_PORT VALUE="25"/>
			</MOD_GLOBAL_SETTINGS>
			<MOD_NETWORK_SETTINGS>
				<ENABLE_NIC value="Yes"/>
				<REG_DDNS_SERVER value="Yes"/>
				<PING_GATEWAY value="Yes"/>
				<DHCP_DOMAIN_NAME value="No"/>
				<SPEED_AUTOSELECT value="YES"/>
				<DHCP_ENABLE value="No"/>
				<IP_ADDRESS value="%iloIP%"/>
				<SUBNET_MASK value="255.255.255.0"/>
				<GATEWAY_IP_ADDRESS value="%iloGW%"/>
				<DNS_NAME value="%servername%-ilo"/>
				<DOMAIN_NAME value="example.com"/>
				<DHCP_GATEWAY value="No"/>
				<DHCP_DNS_SERVER value="No"/>
				<DHCP_WINS_SERVER value="No"/>
				<DHCP_STATIC_ROUTE value="No"/>
				<DHCP_SNTP_SETTINGS value="No"/>
				<DHCPV6_SNTP_SETTINGS value="No"/>
				<REG_WINS_SERVER value="Yes"/>
				<PRIM_WINS_SERVER value="10.10.10.51"/>
				<SEC_WINS_SERVER value="10.10.11.51"/>
				<PRIM_DNS_SERVER value="10.10.10.25"/>
				<SEC_DNS_SERVER value="10.10.11.25"/>
				<SNTP_SERVER1 value="10.10.10.100"/>
				<SNTP_SERVER2 value="10.10.11.100"/>
				<TIMEZONE value="America/Detroit"/>
			</MOD_NETWORK_SETTINGS>
		</RIB_INFO>
		<SERVER_INFO MODE="write">
			<SET_HOST_POWER_SAVER HOST_POWER_SAVER="1"/>
			<SERVER_NAME value="%servername%"/>
			<SERVER_FQDN value="%servername%.example.com"/>
		</SERVER_INFO>
		<USER_INFO MODE="write">
			<MOD_USER USER_LOGIN="Administrator">
				<PASSWORD value="adminpwd"/>
			</MOD_USER>
			<ADD_USER USER_NAME="monitoring" USER_LOGIN="monitoring" PASSWORD="monitoring">
				<ADMIN_PRIV value="N"/>
				<REMOTE_CONS_PRIV value="N"/>
				<RESET_SERVER_PRIV value="N"/>
				<VIRTUAL_MEDIA_PRIV value="N"/>
				<CONFIG_ILO_PRIV value="N"/>
			</ADD_USER>
		</USER_INFO>
		<DIR_INFO MODE="write">
			<MOD_DIR_CONFIG>
				<DIR_AUTHENTICATION_ENABLED value="Yes"/>
				<DIR_LOCAL_USER_ACCT value="Yes"/>
				<DIR_SERVER_ADDRESS value="ldap.example.com"/>
				<DIR_SERVER_PORT value="636"/>
				<DIR_USER_CONTEXT_1 value="OU=Teams,OU=EXAMPLE,DC=example,DC=com"/>
				<DIR_ENABLE_GRP_ACCT value="Yes"/>
				<DIR_GRPACCT1_NAME value="CN=ILO_Admins,OU=EXAMPLE,DC=example,DC=com"/>
				<DIR_GRPACCT1_PRIV value="1,2,3,4,5,6"/>
				<DIR_GRPACCT1_SID value="S-1-0"/>
			</MOD_DIR_CONFIG>
		</DIR_INFO>
		<SSO_INFO MODE="write">
			<MOD_SSO_SETTINGS>
				<TRUST_MODE VALUE="CERTIFICATE"/>
			</MOD_SSO_SETTINGS>
		</SSO_INFO>
	</LOGIN>
</RIBCL>

In the example above, note that some values are represented by variables such as %iloIP%. This allows the actual value to be filled in on the command line dynamically using the -t flag versus hardcoding it in the file. Also note that the <LOGIN> tag is ignored if username & password are used on the HPQLOCFG.exe command line (-u and -p flags).

To run this RIBXML against a virgin iLO4, the command is simply:
C:\Program Files (x86)\HP Lights-Out Configuration Utility>HPQLOCFG.exe -s ILOUSEnnnmmmm.example.com -u administrator -p 12345678 -f c:\scripts\ilo-scripts\my_config.xml
-t servername=SuperServer,iloIP=10.10.15.13,iloGW=10.10.15.1,syslogIP=10.10.15.95

Note the -t flag and its key=value pairs. Any instances of %servername% will be replaced by “SuperServer”, and so on. Very handy.

Pairing this functionality with Powershell and a CSV file of servername, iLO default password, desired iLO Hostname, iLO IP, iLO Gateway, and anything else necessary gives a powerful bulk-configuration tool which can configure all the iLOs in minutes.

Create a CSV as shown:

servername,password,iloHostname,iloIP,iloGW,syslogIP
Server01,12345678,ilouse1112222,10.10.15.12,10.10.15.1,10.10.15.95

Due to some formatting and syntax issues I was running in to with passing values through Powershell, I opted to make a simple “ilo.bat” batch file to take the values from import-csv and ultimately pass them on to HPQLOCFG.exe:

"C:\Program Files (x86)\HP Lights-Out Configuration Utility\HPQLOCFG.exe" -s %1 -u administrator -p %2 -f c:\scripts\ilo-scripts\my_config.xml -t servername=%3,iloIP=%4,iloGW=%5,syslogIP=%6

Then, to run it, use Powershell’s import-csv to parse the CSV, pass the values to the batch file which then passes them to HPQLOCFG.

import-csv -path .\ilo_config.csv | %{ & '.\ilo.bat' $_.iloHostname $_.password $_.servername $_.iloIP $_.iloGW $_.syslogIP }

All iLOs will then be configured! And luckily, if anything gets messed up or the script didn’t work, there is an example Factory_Reset.xml which can set an iLO4 back to defaults (don’t forget the default password – you’ll need it after it’s been reset!)

79,568 total views, 20 views today

Sending Horizon View logs via Syslog

Been experimenting with the vCenter Log Insight beta and have all the devices capable of sending syslog sending it to the Log Insight appliance. Pretty cool stuff. However, I deal a lot with Horizon View connection servers and their various logs, and given the ability of Log Insight to ferret out and find details in logs, I thought it would be great to pipe the View logs in to Log Insight for digestion.

Turns out this is pretty simple using Datagram SyslogAgent which not only can send Windows event logs, it can also follow any text log file and pipe it to a remote Syslog server. The great thing is the agent is free and a breeze to install.

Simply download the zip and extract it someplace permanent (Program Files), then run the SyslogAgentConfig.exe utility. Install the service, enter the syslog server IP and port, enable forwarding of event logs if you like, and then enable forwarding of application logs. I created two application log entries – pcoip and Vmware View, as they are in different folders. Editing one allows you to choose the directory and parsing options:

I used the ‘Suggest Settings’ button which will parse the files and determine the appropriate options.

Once done, just start the service and watch syslog data flow to your syslog server!

Also super handy, the settings are stored in the registry which can be exported to other View servers to make installation and configuration a snap. Just export “HKEY_LOCAL_MACHINE\SOFTWARE\Datagram\SyslogAgent” and its child keys, delete the “LastRun” entry from the .reg file and then import the .reg file to the other hosts, then run SyslogAgentConfig.exe which will show the copied configuration. Install and start the service.

29,365 total views, 5 views today

View Admin dashboard for vCenter Server 5.1 displays the message: VC service is not working properly

Regarding VMware KB: View Admin dashboard for vCenter Server 5.1 displays the message: VC service is not working properly, I encountered this when upgrading a vCenter 5.0 host to 5.1.

When it was all said and done, I checked the View 5.1.3 console and found the vCenter objects in a red error state, saying the service was not working properly.

I edited the vCenter server in View and it gave me the incorrect username/password. Since that is one of the issues in the KB, I followed the steps to move the View domain to be above the System domain in SSO.

Seems like an obvious thing to do for a new SSO installation – wish the SSO installer would have done this.

Once moved, the vCenter objects turned green and provisioning was able to run.

32,998 total views, 5 views today

Change root password on all vSphere ESXi hosts

Adapted from Change root password on all (or some) vSphere h… | VMware Communities.

$vCenter = "vcenterserver"
$oldpw = "oldpwd"
$newpw = "newpwd"

connect-viserver -server $vCenter -Credential (Get-Credential)
$hosts = @() 
write-host "Querying for ESXi hosts..."

Get-VMHost | sort | Where {$_.ConnectionState -eq "Connected" -or $_.ConnectionState -eq "Maintenance"} | Get-View | Where {$_.Summary.Config.Product.Name -match "i"} | % { $hosts+= $_.Name }

Disconnect-VIServer -confirm:$false

foreach ($vmhost in $hosts) {
    write-host "Connecting to $vmhost..."
    connect-viserver -server $vmhost -user root -password "$oldpw"
    write-host "Changing root password on $vmhost..."
    Set-VMHostAccount -UserAccount root -password "$newpw"
    Disconnect-VIServer -confirm:$false
}

21,445 total views, 10 views today

Cpu.PcpuMigrateIdlePcpus

Don’t forget to set the advanced ESXi setting “Cpu.PcpuMigrateIdlePcpus” back to its default value of 4 if it was changed to 0 as a workaround for the issue described in http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2033780.

In over-committed CPU environments (VDI), having Cpu.PcpuMigrateIdlePcpus disabled can increase CPU Ready time. It should be set back to its default of 4 as the official patch for the issue (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2051208) states.

Here is a 6-month graph for a single host (CPU | Ready) showing the 5-month period where Cpu.PcpuMigrateIdlePcpus was disabled in mid-February and re-enabled (back to default) a few days ago:

period of high cpu ready

The setting can be changed quickly and easily across all hosts via PowerCLI: Set advanced settings on all hosts.

23,089 total views, 10 views today