The GigaSpaces Blog » Owen Taylor

Author Archive

« Previous Page — Next Page »

SyncRemoting cookbook

June 4th, 2008

I wrote up a cheat sheet for myself when building scatter-gather or map reduce apps with GigaSpacesXAP.

It goes like this:

Create business interface (as part of shared classes)

Space-side stuff:

Create the Space-side implementation (logic) [ This may use a GigaSpace reference ]
Configure the implementation on the Space-Side:
Declare the business implementation as a [bean]
Declare a filter as part of the space
The filter references a ServiceExporter
Declare the os-remoting:service-exporter
The exporter references the business implementation [the bean]
If impl uses space: Declare the giga-space-context

Client-side stuff

Declare os-remoting:sync-proxy with its attributes:
The business interface it implements
The giga-space it uses as a transport layer
Whether broadcast is true or false
If necessary –declare the reducer for the proxy
Inject the client with the service proxy (ie: public void setMyService(MyService ref){} )

Let’s say we want to do a Map reduce exercise where each node in the cluster will process its share in parallel and return a total result for my digestion…

Let’s say I want to know the average price for all my widgets in all my inventory, but my inventory is scattered across a dozen servers because I do so much business…

I have my widgets:

package com.test.common;

import com.gigaspaces.annotation.pojo.SpaceRouting;

public class Widget {

private Double price;

private String description;

private Integer routingValue;

public Double getPrice() {
  return price;
}
public void setPrice(Double price) {
  this.price = price;
}
public String getDescription() {
  return description;
}
public void setDescription(String description) {
  this.description = description;
}
@SpaceRouting
public Integer getRoutingValue() {
  return routingValue;
}
public void setRoutingValue(Integer routingValue) {
  this.routingValue = routingValue;
}
}

My Service needs an interface such as:

package com.test.common;
public interface PricingService{

public Double getAveragePriceOfAllWidgets();
}

Next, I need to implement the service:

package com.test;

import org.openspaces.core.GigaSpace;
import org.openspaces.core.context.GigaSpaceContext;
import com.j_spaces.core.client.SQLQuery;
import com.test.common.PricingService;
import com.test.common.Widget;

public class PricingServiceImpl implements PricingService{
@GigaSpaceContext
GigaSpace space;

public Double getAveragePriceOfAllWidgets() {
  Object[] allWidgets = space.readMultiple(new Widget(), Integer.MAX_VALUE);
  double priceTotal = 0.0d;
  for(int x=0;x<allWidgets.length;x++){
   priceTotal=priceTotal+((Widget)allWidgets[x]).getPrice().doubleValue();
  }
  double averagePrice = priceTotal/allWidgets.length;
  return new Double(averagePrice);
}
}

Now I need a client that uses this service:

package com.test;
import java.util.TimerTask;
import com.test.common.PricingService;

public class WidgetSurfingClient extends TimerTask{

private PricingService service;
public void setPricingService(PricingService ref){
this.service=ref;
}
public void run(){
System.out.println(
”The average price of all widgets is now: $”+
service.getAveragePriceOfAllWidgets());
}
}

Now I need something to do the “reduce” side of the map-reduce: (it also performs the final average calculation because I am looking for the average price…

package com.test;
import org.openspaces.remoting.RemoteResultReducer;
import org.openspaces.remoting.SpaceRemotingInvocation;
import org.openspaces.remoting.SpaceRemotingResult;

public class AveragingDoubleResultReducer implements RemoteResultReducer<Double,Double>{

public Double reduce(SpaceRemotingResult<Double>[] results,
SpaceRemotingInvocation remotingInvocation) throws Exception {
  double totalDouble = 0.0d;
  int size = results.length;
  for (int x=0;x<size;x++){
   if(results[x]!=null){
    if(results[x].getResult()!=null){
      totalDouble += (Double) results[x].getResult();
    }
   }
  }
  double averageDouble=totalDouble/size;
  return new Double(averageDouble);
}
}

That takes care of the Java code, now we have to configure this system:

First: the Space-side configuration:

<!– declare the transaction manager –>
<os-core:local-tx-manager id=”transactionManager”
space=”widgetTransport”/>

<bean id=”PricingService” class=”com.test.PricingServiceImpl”/>

<!– provide a gigaspace context to the pricing service –>
<os-core:giga-space-context/>

<!–
Provide the binding to the Space-side
transport for the PricingService :
–>
<os-remoting:service-exporter id=”PricingServiceExporter”>
<os-remoting:service ref=”PricingService”/>
</os-remoting:service-exporter>

<!– configure a big or small cluster
(here we have 20 partitions with 1 backup for each) –>
<os-sla:sla cluster-schema=”partitioned-sync2backup”
number-of-instances=”20″ number-of-backups=”1″
max-instances-per-vm=”1″>
</os-sla:sla>

</beans>

Next: the client-side configuration:

<!– bootstrap the basic proxy for the serviceCluster: –>
<os-core:space id=”widgetTransport” url=”jini://*/*/widgetsystem” />

<!– wrap the proxy in Spring-ready OpenSpaces API –>
<os-core:giga-space id=”gigaspace” space=”widgetTransport” />

<!– WidgetSurfer Service –>
<bean id=”WidgetSurfer” class=”com.test.WidgetSurfingClient”>
<property name=”pricingService” ref=”PricingService”/>
</bean>

<os-remoting:sync-proxy id=”PricingService”
            giga-space=”gigaspace”
            broadcast=”true”
            interface=”com.test.common.PricingService”>
<os-remoting:result-reducer ref=”AveragingDoubleResultReducer”/>
</os-remoting:sync-proxy>

<!– this is regular Spring TimerTask configuration stuff: –>
<bean id=”widgetClientTask” class=”org.springframework.scheduling.timer.ScheduledTimerTask” >
<!– wait 1 seconds before starting repeated execution –>
<property name=”delay” value=”1000″ />
<!– run every 5 seconds –>
<property name=”period” value=”5000″ />
<property name=”timerTask” ref=”WidgetSurfer” />
</bean>

<bean id=”widgetClientTimer” class=”org.springframework.scheduling.timer.TimerFactoryBean”>
   <property name=”scheduledTimerTasks”>
    <list>
      <!– This wires the factory to the scheduledTask bean above –>
      <ref bean=”widgetClientTask” />
    </list>
   </property>
</bean>
</beans>

Cheers,

Owen.

Amazon gets more elastic

March 27th, 2008

Amazon EC2 now makes it possible for failure to be less noticeable within EC2 due to the new ability offered to have an IPAddress move from box to box upon failure. This is really cool because it means you can host your webserver as a service within the GigaSpaces AMI and when it automatically fails over due to our service management and self-healing behaviors, the end user will have a hiccup but will be able to refresh and continue without changing the URL!

Another cool new thing EC2 is offering is the ability to host primary and backup nodes in different physical “Zones” - meaning Amazon will guarantee that the backup location is not on the same power grid as the primary. Using GigaSpaces to create and deploy the instances makes it even easier to ensure this desired behavior is achieved.

See Amazon’s Feature Guides for Elastic IP Addresses and Availability Zones.

Learn how to build your first EC2 Applications using GigaSpaces

Cheers,

Owen.

Why not use maven?

January 16th, 2008

I received an (anonymous) comment regarding the command-line project creator.

(anonymous)asks:

“

why dont you use Maven archetypes for this ? See http://maven.apache.org/guides/introduction/introduction-to-archetypes.html

“

I answer:

First of all, if you like Maven I have good news!

The official GigaSpacesXAP product will support Maven and supply some archetypes very soon. (I believe the next early-access build will contain some support for this)

Now, you may, like Charles Miller , not like Maven.

The command-line project-creator is designed to be extremely light-weight and simple for anyone to use.

To use it, you need the jar file ~80kb and a probably two or three batch or shell scripts.

[Once I write some decent documentation, it will be more obvious what is required to execute the functionality.]

While it does create eclipse files, it also creates a build.xml suitable for execution from any ide or even emacs.

That said, once the maven stuff is supported and available, I plan to contribute my service templates to the available archetypes where they can add value.

I do not really see these as competing, but rather co-existing, like items on a menu: some people prefer fish, some meat, both have to eat.

I hope this helps to explain the existence of CPC

BTW: from now on, I will try to have news regarding CPC posted at openspaces and hope further comments can also live there, just for ease of information access and organization.

Cheers,

Owen.

Command-line Project creator (CPC) now on openspaces.org

January 15th, 2008

I have done it. I committed the project creator - now with support for pre-existing and shared domains - to openspaces.org.

I created a new tutorial video which currently acts as the documentation.

That can be seen here

I will be updating docs and such over the next few days.

The user interface now looks like this:

Welcome to the GigaSpaces Project Creator To create a new project, please answer the following questions:


What would you like to name your project [Worker]? myProcessor


Root directory for project creation [C:\_D\_wrk\projectcreator\out]?


GigaSpaces root directory [c:\GigaSpacesXAP6.0]?


Is Space embedded in this PU? [yes]


Name of space [gigaspace]


Do you wish to create the common domain [yes]?


*


What is the package name for the common domain [com.test.common] or ‘no’? org.test


*


Name of Common Object []? Message


*


Package name for generated service [com.test]?


*


ClassName of Service [MyService]? Processor


Created directory C:\_D\_wrk\projectcreator\out\myProcessor


commonDomainFilePath= commonDomain1177f763b16\common\domain


projDir = C:\_D\_wrk\projectcreator\out\myProcessor


Created directory C:\_D\_wrk\projectcreator\out\myProcessor\..\commonDomain1177f763b16\common\domain


commonDir = C:\_D\_wrk\projectcreator\out\myProcessor\..\commonDomain1177f763b16\common\domain


domainPackageDir = org\test


Created directory C:\_D\_wrk\projectcreator\out\myProcessor\..\commonDomain1177f763b16\common\domain\org\test
        DomainCreationHandler: Debug C:\_D\_wrk\projectcreator\out\commonDomain1177f763b16\common\domain


Created file C:\_D\_wrk\projectcreator\out\myProcessor\..\commonDomain1177f763b16\common\domain\org\test\Message.ja


Created directory C:\_D\_wrk\projectcreator\out\myProcessor\src


Created directory C:\_D\_wrk\projectcreator\out\myProcessor\src\test


Created directory C:\_D\_wrk\projectcreator\out\myProcessor\src\java


Created directory C:\_D\_wrk\projectcreator\out\myProcessor\src\java\com\test


Created file C:\_D\_wrk\projectcreator\out\myProcessor\src\java\com\test\Processor.java


Created directory C:\_D\_wrk\projectcreator\out\myProcessor\pu\myProcessor\META-INF\spring


Created directory C:\_D\_wrk\projectcreator\out\myProcessor\pu\myProcessor\META-INF\spring\mode


Created directory C:\_D\_wrk\projectcreator\out\myProcessor\src\java\META-INF\spring


Created file C:\_D\_wrk\projectcreator\out\myProcessor\pu\myProcessor\META-INF\spring\pu.xml


Created file C:\_D\_wrk\projectcreator\out\myProcessor\src\java\META-INF\spring\pu.xml


Created file C:\_D\_wrk\projectcreator\out\myProcessor\build.xml


Created directory C:\_D\_wrk\projectcreator\out\myProcessor\.eclipse


Created file C:\_D\_wrk\projectcreator\out\myProcessor\.eclipse\myProcessor.launch


EclipseTemplatePath= templates/eclipse/


Created file C:\_D\_wrk\projectcreator\out\myProcessor\.project


Created file C:\_D\_wrk\projectcreator\out\myProcessor\.classpath
        THE END.


        (We have completed all processing possible…)


        Hit Enter to continue…

So head over to

openspaces.org

-and check out the new version! (There is a zip file in the downloads section.)

Cheers,

Owen.

Opinionated architecture - blue prints without the middleman

December 16th, 2007

While at the Spring Experience this past week, I was delighted to learn a few things.

I got a good look at an OSGI environment, discovered what it felt like to turn rapidly in a circle on a cigarette boat doing well over 40 knots, and was introduced to the term “Opinionated Architecture”.

To my new understanding, an opinionated architecture, or framework, or solution, is one that proclaims a mandatory set of conventions in various things such as directory structure, configuration and code artifacts, naming conventions, and patterns of use. Due to the specification of these and possibly other things, the adopter of the solution will benefit from the use of the naturally occurring wizards and reusable templates that exemplify the mandatory bits and pieces. More than the traditional “blue-prints” offered by some, the essence of the available and proper use of the

opinionated solution depends on the erasure of alternative paths lest disaster strike.

This is not a trivial course to take. The author of the solution must have effective and real knowledge of the problem domain addressed by the solution - hopefully informed by enough failures that the successful path as declared is completely valid to the limits of the conceit of the solution.

For instance, web frameworks such as Grails provide rapid application development to those who agree to adopt the programming paradigm made possible and pushed by the elected framework. The result is a religious fervor adopted as those who share the faith and bend to the will of the higher, more-knowing, technological power are validated by their successes and spurred to greater adoption and recognition of the awesome power of the solutions’ prescription.

Caution: some wordiness follows:

The topic of the effect of poor planning and proceeding with lack of effective guidance down the enterprise architectural path was touched on in a BOF on high-performance Spring apps that took place late Friday night. (After the keynote filled with Monty Python references) During this talk, it was made clear that only experience can fulfill the needs of the architect challenged by a huge problem. What was also revealed is that humans frequently fail to avoid making the same mistakes numerous times even when presented with compelling alternatives. Old habits die hard it seems.

I am (tonight) of the opinion that unless your solution infrastructure mandates a path of best results and guides you to a proven successful state, you are more than likely to wander into the woods of competing best practices overgrown with budget constraints, lack of innovative and experienced resources, and just plain lack of understanding. It is therefore better to adopt a winning solution set whole-heartedly and when all the Kool-Aid is gone, agree to be dictated to by a benevolent dictator who has the right stuff.

The experience of a journeyman or artist is really very similar. In classical Ballet, for instance, the student rarely questions the validity of the techniques demonstrated by the Maestro leading the class/company. There is an element of necessary blind faith in the chosen path and the guides you choose to follow down it. Once one determines that the task at hand is indeed Ballet, one really has no choice but to turn out hips, point feet, and develop habits of posture that will sustain them through the most grueling choreography - for it will surely come and in spades. The student/apprentice is not well served to strike out on their own until a mastery is attained of the subject matter *through experience* that validates new choices and proves that through their adoption, greater success is to be found. The opinion therefore exhibited must be one based on hard-earned experience and still based predominately on a foundation made real and sustainable by the experiences of many others.

As it happens, I have watched the growth of OpenSpaces over the last many months with the skeptical eye of a - er skeptic - not realizing that the purpose of the framework is not only buzzword compliance and the implementation of popular development practices, but in the context of GigaSpaces and what is offered to our prospects and customers, a truly opinionated framework that takes the vastly flexible GigaSpaces infrastructure and service implementations and distills *the* successful use of it all into enforceable use of the Spring programmatic approach and technologies, thus empowering the neophyte and thought-leader alike with the unmistakably opinionated path to success.

The beauty of the switch to the prescriptive and formulaic from the free and often flailing style, is the stunningly rapid improvement in adoption rate, early prototyping successes, and sustained production-systems that continue to reinforce my growing certainty that we got our bit of it right this time.

Yes, OpenSpaces and by our support of it, GigaSpaces, has become an opinionated architecture/solution. As more and more tools and samples become available, and more wizards like my somewhat primitive, yet highly effective Project-creator, are built and offered, that fact will be obvious to all and joyfully so to the initiates and practitioners of the Linearly Scaling Aplication Arts. (The LSAA)

Write Once, [our way]

Scale Anywhere

Cheers,

Owen.

« Previous Page — Next Page »