Archive for the 'Java' Category
Integrating GigaSpaces persistency service into an existing tier based system
April 23rd, 2008A common issue I’m facing recently is how to integrate existing tier based applications with GigaSpaces persistency service, AKA persistency as a service (Paas) or mirror . The motivation is often a result of the acknowledgment that a standard tier based application fails to scale when facing the database throughput limitation.
Software Caching technologies (overlooking their [...]
Cloud Aware Classpath
March 12th, 2008One of the most challenging aspects in deploying large distributed systems is making sure the right jar files are located where they are needed. For a cluster that contains many nodes (tens and hudreds and even thousands) having the ability to deploy the application code easily is very important.
This problem is not new and indeed there have beens several solutions for it in the past:
The first one is of shared file-system. All hosts have access to a shared drive. A classppath variable is set to this shared drive and upon change this shared drive is the only place to put new jar files. This simple solution is ok when it comes to static deployments and when we know all hosts are on the same physical LAN. Another drawback of this simple solution is that it represents a SPOF in the system. Also, the singularity imposes a contention point that in many cases becomes a bottleneck. Also, it is imperative to make sure the FS is as available as our SLA requirements. This may become very costly.
The second approach is of a codebase server. This is the approach used by RMI and by JINI. As java class loader is a URLClassLoader, all we make sure is that our jars are behind a reliable HTTP server and when the JVM starts it looks for the classes at the codebase server as well. The codebase server is a first step in virtualization. A url is abstracted by a host name controlled by a DNS and can be changed without impacting the application. There are few implementation problems with this approach: The first is that when it comes to downloading the jars, in real life, this may lead to a network storm which will result in system hickups as jars are downloded fully and in real life systems those jars may be very large. Another problem with the codebase server approach, is that the URLClassLoader behavior is static. Once a class has been loaded into the class loader it can not be changed, and in order to replace it the class loader needs to be destroted, which practically means in many cases, the process needs to be restarted.
What we’ve introducing in 6.5 and is already available in M7, is what we call "Cloud Aware ClassLoader". Once a VM in the cloud requires to load a class that it does not have it’s class defition, it calls to the "cloud" classloader to retreive the class definition. This mechanism is fully integrated with the JVM class loader, in many cases the sequence of loading a class is transitive. For example if class A implements interface I, the classloader starts loading class A and as it idetifies other class dependecies such as interface I are missing, it looks for it in the cloud for the interface’s class defintion.
The full power of this feature unlocks dynamic capabilities of patterns such as MapReduce and Master Worker. Where is such cases the implementation of the Work object (M/W) can be changed at runtime while the system is running without the need to re-deploy the code on all nodes again.
Cosider the following scenario. Task interface is defined with one execute mehtod. Worker implementations resides on cloud nodes know how to consume a Task object and to call it’s execute method. The first client connects to the cloud and submit an implementation of Task which is called TaskA. All he’s doing is connecting to the cloud and submitting his task. No need for task deployment on cloud nodes is required. A second client connects to the cloud as well and submits his own implementation of Task named TaskBTask and he’s not deploying as well. The beauty of it is that if client A would like to be able to use TaskB the class information of TaskB will be shipped automatically to his JVM upon demand without any configuration.
The funny part, is that we didn’t plan in advance to solve this problem in the release, we had other issues to attend. However, as we had several painful experiences in deployment of huge clouds in our lab and on EC2, the Runtime Platform team, decided that they are too spoiled to work with non-elegant solutions, and in few days work, put this great feature into the product.
You’re all encourage to give it a spin and share you’re thoughts on this with us.
Cheers,
Guy
.Net Customer Announcement: Susquehanna (SIG)
March 11th, 2008In the past few months we've made several exciting announcements, such as our partnership with SpringSource , the expansion of our executive team, the launch of our community site OpenSpaces.org , the OpenSpaces Developer Challenge and the Start-Up Program . But there is nothing like a customer announcement, as in today's press release [...]
Why not use maven?
January 16th, 2008I received an (anonymous) comment regarding the command-line project creator.
(anonymous)asks:
“
why dont you use Maven archetypes for this ? See http://maven.apache.org/guides/introduction/introduction-to-archetypes.html
“
I answer:
First of all, if you like Maven I have good news!
The official GigaSpacesXAP product will support Maven and supply some archetypes very soon. (I believe the next early-access build will contain some support for this)
Now, you may, like Charles Miller , not like Maven.
The command-line project-creator is designed to be extremely light-weight and simple for anyone to use.
To use it, you need the jar file ~80kb and a probably two or three batch or shell scripts.
[Once I write some decent documentation, it will be more obvious what is required to execute the functionality.]
While it does create eclipse files, it also creates a build.xml suitable for execution from any ide or even emacs.
That said, once the maven stuff is supported and available, I plan to contribute my service templates to the available archetypes where they can add value.
I do not really see these as competing, but rather co-existing, like items on a menu: some people prefer fish, some meat, both have to eat.
I hope this helps to explain the existence of CPC
BTW: from now on, I will try to have news regarding CPC posted at openspaces and hope further comments can also live there, just for ease of information access and organization.
Cheers,
Owen.
Update on Virgin Mobile post (thanks to Julian Browne)
December 13th, 2007A while ago I posted a note about the success Virgin mobile experienced from switching to a Space-Based Architecture.
I received a comment from Julian Browne that I think is informative and so I make it public here:
***
Hi Owen - a backend system was unavailable for a few hours (honestly I
can’t remember the exact time). It was towards the end of the day’s
activity and it wasn’t back online until after midnight.
We knew this because one of the business managers thought they spotted
a bug in our reporting software after it informed him that more than
100% of orders had just been fulfilled. Never having seen anything
more than a 100% fulfilment rate, he naturally suspected an error.
On investigation we discovered that some of the previous day’s late
orders had been held in the space until the legacy system came back up
and submitted to the warehouse that morning. The system that was
replaced by the space-based approach didn’t do this and would often
lose orders entirely. What he really loved about this was that
managing orders safely in the event of a backend system not being
ready hadn’t even on his list of requirements.
I think there’s been some confusion created by other posts about the
project around order numbers generally. The number of online orders
increased by around three times almost immediately we went live (IT
wasn’t aware of this for a while, the system just scaled up without
any complaint) and over the Christmas period (a stressful time under
the pre-SBA solution) we handled a massive increase in orders
completed (unfortunately I’m not at liberty to say by how much because
it’s commercially sensitive).
I spoke to the online director only a couple of weeks ago and he still
loves it nearly two years later.
***
Thanks Julian,
It is great to know that what was once only a few lines of test code has survived and blossomed into a reference implementation.
Cheers,
Owen.
