So you have this idea, started testing it on your laptop, and it looks good.  Now you want to test how it scales out on multiple machines. In cases where your IT department doesn't have available machines inhouse, you might want to consider cloud providers. According to Bernard Golden (from CIO.com) this starts becoming quite common:

Make no mistake about it, developers are using public cloud computing-a lot. We certainly see it in our daily business interactions. The high level of usage (and its somewhat clandestine nature) was illustrated at a conference earlier this year when one CIO noted that he had gone through the expense reports turned into him for reimbursement and found 50 different cloud computing accounts being used by developers in his organization. The reason? It's a lot easier for a developer to obtain computing resources from a public cloud provider than to undergo the extended waits typical of the existing compute environments.
http://www.cio.com/article/618385/Cloud_Computing_The_Truth_About_What_Runs_on_Amazon?page=2

In some cases, you may want the cloud to have the same setup as your inhouse lab, you want to install the same VMs in both environments or you want to tune your network architecture so that it would be closer to your production system.

Here at GigaSpaces, during the integration with one of our partners, we got an opportunity to work with SoftLayer which is a hosting and cloud provider. SoftLayer's "Bare-Metal Cloud Computing" service, provides hosted physical machines preinstalled with Xen Server virtualization software. Since a bare metal cloud exposes the virtualization APIs and VM images we could "duplicate" some of our lab into SoftLayer.

Proper disclosure: We got the SoftLayer service without charge through our partner, so we were not looking for any other "Bare Metal Cloud" providers. For a list of other bare metal cloud providers follow this quora disussion http://www.quora.com/Who-are-the-leading-bare-metal-cloud-providers

Softlayer also has a 24/7 support service. This is really important if you are moonlighting or working in a non-US timezone.I (unintentionally) "stress-tested" the SoftLayer support team. I opened quite a few network topology questions and change requests and got very fast and detailed responses. In some cases I got instructions on how to perform the changes myself through the management portal. Two of the less trivial tickets were quickly escelated to a senior engineer, and in one case, I was politley asked if the issue could wait for Monday morning (for the cloud developers to arrive). Apparantly : – ) the first working day in Israel (Sunday morning) is Saturday night in the US.The support portal has a cool 5 stars rating system, so when the case was closed I thanked them and clicked the 5 star thingy.

Ok… this blog post was meant to explain the difference between different IaaS flavors. So let's take a step back and compare "Infrustructure as a Service", "Build your own cloud Service" and the "Bare Metal cloud Service".

Infrustructre as a Service

Infrustructure as a Service (ala Amazon EC2, Rackspace, GoGrid, SoftLayer Cloud,  and all the others I forgot to mention) provide virtual machines (start machine/stop machine), preconfigured network services ( DHCP,Firewall,Load Balancer) and storage (File System, Key/Value storage). Each cloud has different APIs and a different network architecture (which are probably also different from your inhouse lab). The service is attractive due to its low entry costs and the premise that it "just works".

IaaS charge per new machine (per hour) which could become expensive if you run tests that start virtual machines automatically. Shortly after joining Gigaspaces, much like the Sorcere's Apprentice , my first instinct was to shout "I command you to stop". More and more virtual machines started automatically and "taskkill /f /im java.exe" had zero effect. So if you do choose IaaS for your scalability tests, make sure to memorize the "killall -9 java" spell.

Build your own cloud Service

IaaS might also introduce SLA uncertainties, due to the fact you are sharing the physcial machine and network resources with others.A more fine grained IaaS alternative is what SoftLayer calls "build your own cloud" service. This service allows specifying the exact ammount of memory,disk and CPU for virtual machines. It provides fine grained control over the lab performance (why pay for extra CPU and disk if all you need is extra RAM).  The APIs are still cloud vendor specific and are probably incompatible with your inhouse infrustructure.

I haven't tried this service … so there's not much I can report.

Bare metal cloud Service

The third alternative is "Bare Metal Cloud" which is the most geek oriented. It has the advantage of using raw virtualization APIs (XenServer). Which means that the same code and installation scripts can run on a XenServer hosted inside your organization. The flipside is that you need to do some plumbing yourself, and this is what this blog post series is all about.

One word of advice, the instructions are meant for quick ramp up projects. Not described here are security measures and single point of failure mitigations, that are essential for production ready deployments.

So let's start the step-by-step guide ….

Step 1 -physical machine and some network storage

After registering for a http://www.softlayer.com account, login to the softlayer portal.
To order a new machine with XenServer preinstalled, click the Sales menu, and choose "Add Bare Metal Instance".

 Also order NAS Storage to store the operating system installation ISO files.

 After a few hours (or a business day) you should get an email saying the physical machines are ready. The Bare Metal Computing page (under the Cloud Computing menu) reveals the login credentials of the new xenserver machine.

Step 2 – VPN connection

A VPN connection is required to manage the xenserver machine. A VPN connection can be established from softlayer's homepage by selecting the data center's SSL VPN before clicking the login button.

Step 3 – Time zone

SSH to the xenserver, and check the timezone

Here is how to show the date and the timezone
(For more info see http://support.citrix.com/article/CTX116082 )

date
ls -l /etc/localtime

And here is an example how to modify the timezone:

mv /etc/localtime /etc/localtime.bak
ln -s /usr/share/zoneinfo/America/New_York /etc/localtime

Step 4 – Network Speed

Check network speed (you can order faster speeds through the portal)

dmesg | grep -i duplex

Step 5 – ISO download

Mount the network access storage as described in https://manage.softlayer.com/PrivateNetwork/storagelayer/NAS  (click the "Mount in Linux" link). It should look simmiliar to the following (replace the XXX below with your details) :

su root
mkdir /mnt/share
mount -t cifs //nasXXX.service.softlayer.com/SLXXXX-X -o username=SLXXXX-1,password=XXXXXX /mnt/share

Then download the operating system installation ISO file. We will use the 32bit CentOS 5.3 ISO which is compatible with Xen Server 5.5.

The commands below links the downloaded ISO file to the Xen Server's ISOs folder.

mkdir -p /var/opt/xen/iso_import
ln -s /mnt/share/CentOS-5.3-i386-bin-DVD.iso /var/opt/xen/iso_import/CentOS-5.3-i386-bin-DVD.iso
xe sr-create name-label=ISOs type=iso device-config:location=/var/opt/xen/iso_import device-config:legacy_mode=true content-type=iso
xe-mount-iso-sr /var/opt/xen/iso_import

Step 6 – XenCenter management

Managing Xen Server is done via API calls, commandline or with Xen Center, (a .NET application requires a windows machine). SoftLayer has a download mirror of XenCenter v5.5 and v5.6 which can be found at https://manage.softlayer.com/Software/downloads . Match the version of XenCenter with the version of XenServer you ordered.

Once XenCenter is running click "Add New Server". Type in the IP addresses and login credentials of the Xen Server … POOF …

Notice that your xenserver has an ISOs folder which contains the downloaded CentOS ISO file. That means you can now start a new virtual machine.

That's it for today.

In the next parts of the series, we will configure a DHCP server and install centos on a new virtual machine.

Itai