Skytap recently announced the availability of the Skytap Automation Pack for IBM UrbanCode Deploy. You can read about it in this Skytap blog post from earlier this year.
You’ll recall that this plugin adds the following functionality into the IBM UrbanCode Deploy Application Process web UI to create and manipulate templates and configurations in Skytap:
- Skytap Cloud Authentication
- Add Configuration to Project
- Add Template to Project
- Change Configuration State
- Connect to Network in another Configuration (ICNR)
- Connect to VPN Tunnel
- Create Configuration from Template
- Create Published Service
- Create Published URL
- Create Template from Configuration
- Delete Configuration
- Get VM ID
- List Published URL for Configuration
- List VM Published Service
- Merge Template into Configuration
- Set VM User Data
In this blog post we’ll explore how to use some of the advanced capabilities available in Skytap Cloud along with the functions available in the Skytap Automation Pack for IBM UrbanCode Deploy. This will enable us to create a Linux autoconfiguring IBM UrbanCode Deploy Agent and save it as a reusable Skytap Environment Template.
Prerequisite Software
As we begin this exercise, the following software must already be installed on a CentOS 6.4 VM in a Skytap Environment:
- The IBM UrbanCode Deploy Agent software
- The “curl” command package to enable interaction with the Skytap API
- Install into /usr/local/bin, “jq”, a a command line JSON processor to enable parsing Skytap Environment metadata returned from the Skytap API. This application is available from http://stedolan.github.io/jq
Managing Self-Registration of IBM UrbanCode Deploy Agents
When the IBM UrbanCode Deploy Agent service starts on a compute engine, it seeks to register itself with an IBM UrbanCode Deploy Server. As part of this registration process, each agent that is registered with an IBM UrbanCode Deploy Server must have a unique name by which it will be referenced by the IBM UrbanCode Deploy Server. Both the server to which the agent registers itself, as well as the name it will be assigned upon registration, are specified on the IBM UrbanCode Deploy Agent in the configuration file located at IBM UrbanCode Deploy Agent Install Path >/conf/agent/installed.properties. In our case, this path is as follows:
/opt/ibm-ucd/agent/conf/agent/installed .properties
The default contents of this configuration file are as follows (items in italics are set during the agent install process):
IBM UrbanCode Deploy/java.home=/usr/java/jdk1.7.0_51/jre
locked/agent.brokerUrl=failover:(ah3://localhost:7918)
locked/agent.home=/opt/ibm-ucd/agent
locked/agent.http.proxy.host=
locked/agent.http.proxy.port=
locked/agent.id=randomized id string >
locked/agent.jms.remote.host=localhost
locked/agent.jms.remote.port=7918
locked/agent.keystore=../conf/ibm-ucd.keystore
locked/agent.keystore.pwd=pbe{randomized key string >}
locked/agent.mutual_auth=false
locked/agent.name=hostname >
Proper operation of an IBM UrbanCode Deploy agent depends on the following prerequisites:
- Each IBM UrbanCode Deploy agent connecting to a common IBM UrbanCode Deploy server must have a unique agent name. This agent name is set in the “locked/agent.name=…” parameter in the installed.properties file you see above.
- The IBM UrbanCode Deploy server to which an agent is to connect must be specified in the “locked/agent.jms.remote.host=…” parameter in the installed.properties file you see above.
In order to ensure proper operation of our Skytap IBM UrbanCode Deploy Autoconfiguring Agent, we will set these parameters during the initial boot of the agent VM. To enable this operation we will alter the installed.properties files as follows:
locked/agent.brokerUrl=failover:(ah3://xxxUCD_SERVERxxx:7918)
locked/agent.jms.remote.host=xxxUCD_SERVERxxx
locked/agent.jms.remote.port=7918
locked/agent.http.proxy.host=
locked/agent.http.proxy.port=
IBM UrbanCode Deploy/java.home=/usr/java/jdk1.7.0_51/jre
locked/agent.home=/opt/ibm-ucd/agent
locked/agent.mutual_auth=false
locked/agent.keystore=../conf/ibm-ucd.keystore
locked/agent.keystore.pwd=pbe{some-unique-random-string}
locked/agent.name=xxxINSTALLED_AGENT_NAMExxx
The entries in bold above will be changed upon boot of the Skytap VM to the appropriate value:
xxxUCD_SERVERxxx will be changed to the hostname of the IBM Urbancode Deploy Server to which the Skytap IBM UrbanCode Deploy Autoconfiguring Agent is to connect
xxxINSTALLED_AGENT_NAMExxx will be changed to a unique name to allow multiple Skytap IBM UrbanCode Deploy Autoconfiguring Agents to register with a common IBM UrbanCode Deploy server. Note that in the Skytap environment, using the hostname as the agent name does not guarantee uniqueness because Skytap Environments that are created from the same Skytap Template will have the same hostname.
We will save this modified “installed.properties” file as our reference file, “installed.properties.orig” for use in the Skytap IBM UrbanCode Deploy Autoconfiguring Agent startup script later in this article.
Configuring Skytap User Data
We will use the Skytap VM User Data to specify values to be used for the “xxxUCD_SERVERxxx” and “xxxINSTALLED_AGENT_NAMExxx” fields in the Skytap IBM UrbanCode Deploy Autoconfiguring Agent installed.properties file:
It the above screenshot, we see a value of “Skytap-Agent” assigned to a “UCD-Group” variable. We will use this UCD-Group variable as a mnemonic identifier string to prepend to the unique ID assigned to the Skytap IBM UrbanCode Deploy Autoconfiguring Agent that is registered with the IBM UrbanCode Deploy Server. In practice, we could preconfigure several different autoconfiguring agent Skytap Templates to serve as database servers, application servers, etc. Use of this mnemonic string allows us to easily discern the role of the agent once it is registered with the IBM UrbanCode Deploy Server.
In the above screenshot, we also see a value of “10.0.0.1” assigned to the “UCD-Server” variable. This value is the IP address of the IBM UrbanCode Deploy Server.
You can read more about the Skytap VM User Data feature here.
The IBM UrbanCode Deploy Agent Boot-time Startup Script
The script to startup the Skytap IBM UrbanCode Deploy Autoconfiguring Agent process at system boot is located in the file IBM UrbanCode Deploy Agent Install Path >/bin/init/ibm-ucdagent. In our case this file is located as follows:
/opt/ibm-ucd/agent/bin/init/ibm-ucdagent
Recall that we modified the installed.properties file to parameterize the values required to dynamically configure the agent and saved this modified file as “installed.properties.orig” for use in the Skytap IBM UrbanCode Deploy Autoconfiguring Agent startup script. Prior to starting the Skytap IBM UrbanCode Deploy Autoconfiguring Agent we must alter the startup script to use the Skytap REST API to retrieve from the Skytap VM User Data, the IBM UrbanCode Deploy Server designation and the UCD-Group mnemonic string to use in creating a unique name for the Skytap IBM UrbanCode Deploy Autoconfiguring Agent.
Creating the Skytap IBM UrbanCode Deploy Autoconfiguring Agent Name
Since the Skytap IBM UrbanCode Deploy Autoconfiguring Agent name must be unique, we can use the Skytap VM ID, which is guaranteed to be unique in the Skytap environment, for this purpose. This VM ID, along with the UCD-Group mnemonic string specified in the VM User Metadata are both accessible via the Skytap REST API. By concatenating these values together in the form – we can construct a name that is both descriptive and unique.
Modifying the Skytap IBM UrbanCode Deploy Autoconfiguring Agent Startup Script
The modifications we make to the Skytap IBM UrbanCode Deploy Autoconfiguring Agent startup script are made in the start() routine of the script. These modifications as well as annotations in the comments are seen below.
start() {
touch $agent_pid
echo -n “Starting ibm-ucdagent: ”
# — Skytap Autoconfig Section Begin ——————————————
#
# This pause is necessary to allow initial boot-time system processes time to start up
#
sleep 15
#
#Set the home directory of the “installed.properties file
#
PROPFILE_HOME=”$AGENT_HOME/conf/agent”
#
# Save any existing installed.properties file with a time-date stamp for future reference, if required
#
mv -f ${PROPFILE_HOME}/installed.properties ${PROPFILE_HOME}/installed.properties.`date +%F-%T`
#
# create a fresh copy of the installed.properties file from a parameterized installed.properties file
# (saved in installed.properties.orig)
#
cp ${PROPFILE_HOME}/installed.properties.orig ${PROPFILE_HOME}/installed.properties
#
# remove any previous intermediate ConfigData that might be left from a previous run of this script
#
rm -f /tmp/ConfigData_*
#
# issue a call to the Skytap API to get the Skytap Configuration metadata for this VM and save it in
# in an intermediate file
#
curl -s -X GET “http://gw/skytap” -H “Content-Type: application/json” -H “Accept: application/json” >/tmp/ConfigData_$$
#
# use the jq command line JSON parser along with some sed pattern matching and text substitution
# to extract the VM ID from the metadata retrieved from the SkytapAPI
#
vmID=`cat /tmp/ConfigData_$$ | /usr/local/bin/jq .id | sed s/\”//g`
#
# remove any previous intermediate UserData file that might be left from a previous run of this script
#
rm -f /tmp/UserData_*
#
# use the jq command line JSON parser along with some sed pattern matching and text substitution
# to extract the UserData field from the VM Metadata
#
cat /tmp/ConfigData_* | /usr/local/bin/jq .user_data | sed s/\\\\r\\\\n/\”\\n\”/g > /tmp/UserData_$$
#
# Now use some grep, awk and sed pattern matching and text parsing to extract the UCD-Group and
# UCD-Server designations from the VM User Data
#
ucdGroup=`grep UCD-Group /tmp/UserData_$$ | awk -F= ‘{print $2}’ | sed s/\”//g`
ucdServer=`grep UCD-Server /tmp/UserData_$$ | awk -F= ‘{print $2}’ | sed s/\”//g`
#
# echo what we found to the boot log
#
echo “Installed Agent Name: ${ucdGroup}-${vmID}”
echo “UCD Server: ${ucdServer}”
#
# Now use sed pattern matching and substitution to modify the installed.properties file
#
# first set the Agent Name and save to an intermediate file
#
sed s/xxxINSTALLED_AGENT_NAMExxx/`echo ${ucdGroup}-${vmID}`/ < ${PROPFILE_HOME}/installed.properties > ${PROPFILE_HOME}/new.installed.properties
#
# next use that intermediate file as input and set the IBM UrbanCode Deploy Server to which
# the agent is to connect and save to another intermediate file
#
sed s/xxxUCD_SERVERxxx/`echo ${ucdServer}`/ < ${PROPFILE_HOME}/new.installed.properties > ${PROPFILE_HOME}/newer.installed.properties
#
# move the modified file into place overwriting the existing installed.properties file (Note the –f
# argument to suppress any warning or user prompt)
#
mv -f ${PROPFILE_HOME}/newer.installed.properties ${PROPFILE_HOME}/installed.properties
# — Skytap Autoconfig Section End ——————————————
# AMD — Made lock location variable.
if [ -f $agent_lock ]; then
if [ -f $agent_pid ]; then
…
The steps in the above script that create intermediate files could be combined and the intermediate files reduced or eliminated but these intermediate files are useful for debugging or further study of the process and have been left in for purposes of this writeup.
Notable steps in the script include the use of the curl command to interact with the Skytap API to retrieve information about the currently running Skytap VM. In particular, note that the running Skytap VM can get information about itself via the Skytap API without requiring authentication.
Enable the Boot-time Startup of the IBM UrbanCode Deploy Agent
To enable Skytap IBM UrbanCode Deploy Autoconfiguring Agent startup script to run at boot time, add it to the /etc/init.d directory as follows:
ln –s /opt/ibm-ucd/agent/bin/init/ibm-ucdagent /etc/init.d
For our CentOS system, we also need to issue the appropriate chkconfig command to enable the script to run at system boot:
chkconfig –-add ibm-ucdagent
Different procedures may be required for alternate Linux implementations.
Testing the Skytap IBM UrbanCode Deploy Autoconfiguring Agent
Now that we have put all the pieces in place, we can test our Skytap IBM UrbanCode Deploy Autoconfiguring Agent. To do this, we must first shutdown our Autoconfiguring Agent Skytap Environment. Do this by navigating to the Configuration Details page that contains the Skytap IBM UrbanCode Deploy Autoconfiguring Agent and click the “Shutdown” button on the Agent VM:
When the agent VM has shutdown, we will open a browser window to the UrbanCode Server and navigate to the Resources->Agents tab:
In the above screenshot we can see that there is already one agent named “host1.skytap.example” running, the one hosted on our IBM UrbanCode Deploy Server VM.
Now we start our Skytap IBM UrbanCode Deploy Autoconfiguring Agent VM by clicking the “Run” button on the Agent VM:
After the VM has started, we will see the VM Status is “running”:
Reviewing the IBM UrbanCode Deploy Server web interface, we should see the Autoconfiguring Agent appear in the “Agents” tab:
To enable reuse of this IBM UrbanCode Skytap Autoconfiguring Agent we will first stop the VM:
Once it has stopped, we can press the “Save as Template” button in the Skytap Configuration Details page:
Complete the Template Creation dialog:
When the template has been created, we will see the following screen:
The template will appear in the template listing page for “My” templates and available for reuse:
In my next blog post, I’ll discuss how to use the Skytap Automation Pack for IBM UrbanCode Deploy to instantiate, configure and use Skytap IBM UrbanCode Deploy Autoconfiguring Agent Template in a complex IBM UrbanCode Deploy process.
Summary
We’ve explored an advanced scenario where we created a Skytap Environment containing an IBM UrbanCode Deploy Agent that automatically configures itself at boot-time for successful registration and use by and IBM UrbanCode Deploy Server.
- Install the software prerequisites on a VM in a Skytap Environment
- Modify the Skytap IBM UrbanCode Deploy Autoconfiguring Agent “installed.properties” file to parameterize the IBM UrbanCode Deploy Server to which the agent is to connect and establish a unique agent name
- Modify the Skytap User Data associated with the Skytap IBM UrbanCode Deploy Autoconfiguring Agent to specify the IBM UrbanCode Deploy Server to which the agent process is to connect and to specify an identifier string to prepend to the agent name.
- Modify the Skytap IBM UrbanCode Deploy Autoconfiguring Agent startup script to modify the parameterized “installed.properties” file prior to starting the agent process
- Save the modified Skytap Environment as a Skytap Template to allow reuse.
The principles demonstrated in this exercise and the additional Skytap actions provided by the Skytap Cloud Automation Pack for IBM UrbanCode Deploy can easily be expanded further into even more complex scenarios to satisfy even the most intricate and complex deployment requirements.
The next blog post will expand on this topic to demonstrate how to use the Skytap Automation Pack for IBM UrbanCode Deploy in conjunction with the Skytap IBM UrbanCode Deploy Autoconfiguring Agent Template we created in this discussion to dynamically provision a complex deployment environment.
Please contact Skytap ( 1-888-759-8278 or https://www.skytap.com )if you want more information about Skytap or the Skytap Cloud Automation Pack for IBM UrbanCode Deploy.
