The GigaSpaces Blog » Owen Taylor

Author Archive

Video is worth a million words. . . Latest GigaSpacesXAP demo

August 12th, 2008

With my wife and kids away, I am having a geeky bachelor time that many who read this would probably find quite thrilling. For example: I spent this last weekend producing a new MonteCarlo application example for GigaSpaces.

It is rich in the patterns it utilizes:

SBA (Colocated spaces with workers)
Master-worker (remote workers which scale dynamically when the workload is too high)
Spring Remoting on top of GigaSpaces – the aggregation work is achieved utilizing our Sync Remoting

The app goes like this:

Historical information regarding several funds is used to predict the future behavior of those funds over a 20 year time period with Minute by Minute fluctuation in prices being calculated and applied.
These funds are grouped in different formations into Portfolios and the entire Portfolio assessed for its growth over the same 20 year period.

In the end, the best combination of funds within a portfolio is determined and reported and the best and worst behaving funds are also shown.

Technically, several portfolios and their associated funds are written as tasks into the system for analysis – the logic needed to process them and their historical information is sent along with them as well. Workers take the tasks, process the associated logic and return a result showing the outcome of the application of the expected variation in price for each fund in the portfolio over the designated 20 year period.

This is performed over a set of possible portfolios – each containing different variations of funds.
Each set is repeatedly simulated to avoid undo skewing and allow for better control of the randomness.

In the end, the client requests a summary of all the results which is created by a service that operates in parallel across the available spaces to take all the results and sort them and aggregate them to determine the most common, best [highest value portfolio], and worst [lowest value portfolio]

This information is returned to the client and printed out to the console.

This morning, I produced a nifty 8 minute flash presentation showing my Monte Carlo application being executed on the GigaSpaces platform.

NB: To keep the video short, and because I only used my laptop, I kept the size of the sets and the iterations at very low levels. Nevertheless, the application scales very nicely and provides fault-tolerance as well.

You can check out the 8 minute video using this link: THE MONTECARLO VIDEO

[you may have to wait a while for the video to load, if it stops
mid-way, play it again from the beginning after it loads completely]

I hope you enjoy watching the fruits of my labors – what could be a better way to spend a bachelor weekend eh?

Cheers,

Owen.

Bridging the gap between the clouds

August 9th, 2008

Dekel Tankel of GigaSpaces spoke recently to a hip cloud crowd regarding the risks associated with moving an application to the cloud or grid environment.

Without a GigaSpaces Space-based architecture (what I refer to as a TPC Architecture ) applications running in a cloud suffer from an inability to scale and start to erode the benefits of the cloud. In addition there are questions regarding the ability to maintain state reliably in a cloud: especially a pay by the drink cloud where it can get expensive both in terms of $$ and time to utilize the persistence mechanisms provided by the vendor.

Dekel speaks about and demonstrates how GigaSpaces makes it possible to manage your application services -moving them from machine to machine as needed and also addresses scaling the application by spreading it out [partitioning it while removing the traditional bottlenecks of the database and network boundaries between services]

video

He does a nice job – way to go Dekel!

Cheers,

Owen.

Geva makes a strong argument…

August 1st, 2008

In his blog post: here Geva Perry speaks about the dollars and sense regarding setting up shop in a cloud and how GigaSpaces’ Elastic Appserver [my term], provides incredible efficiency which leads to greater dollar value realized from your it investment. After reading Geva’s blog, I have nothing to add except I am *so* glad I work for GigaSpaces!

BTW, These days, I am conducting various educational webinars designed to assist new users in understanding the nature of architecting and programming GigaSpaces solutions. Please, feel free to check them out.

Other news: I created a new introductory video series with a new and improved [or so I think] HelloWorld application example. The videos are < 20 minutes each and should also prove useful in helping users get started with the GigaSpacesXAP way of doing things. Check out this quick start guide to see what I am talking about.

Hope you are having a great summer, my family is back in FL (remember the wedding pics?) so, I should be able to get a tan and lose some weight playing tennis and swimming. -ahh life is good.

Cheers,

Owen.

SyncRemoting cookbook

June 4th, 2008

I wrote up a cheat sheet for myself when building scatter-gather or map reduce apps with GigaSpacesXAP.

It goes like this:

Create business interface (as part of shared classes)

Space-side stuff:

Create the Space-side implementation (logic) [ This may use a GigaSpace reference ]
Configure the implementation on the Space-Side:
Declare the business implementation as a [bean]
Declare a filter as part of the space
The filter references a ServiceExporter
Declare the os-remoting:service-exporter
The exporter references the business implementation [the bean]
If impl uses space: Declare the giga-space-context

Client-side stuff

Declare os-remoting:sync-proxy with its attributes:
The business interface it implements
The giga-space it uses as a transport layer
Whether broadcast is true or false
If necessary –declare the reducer for the proxy
Inject the client with the service proxy (ie: public void setMyService(MyService ref){} )

Let’s say we want to do a Map reduce exercise where each node in the cluster will process its share in parallel and return a total result for my digestion…

Let’s say I want to know the average price for all my widgets in all my inventory, but my inventory is scattered across a dozen servers because I do so much business…

I have my widgets:

package com.test.common;

import com.gigaspaces.annotation.pojo.SpaceRouting;

public class Widget {

private Double price;

private String description;

private Integer routingValue;

public Double getPrice() {
  return price;
}
public void setPrice(Double price) {
  this.price = price;
}
public String getDescription() {
  return description;
}
public void setDescription(String description) {
  this.description = description;
}
@SpaceRouting
public Integer getRoutingValue() {
  return routingValue;
}
public void setRoutingValue(Integer routingValue) {
  this.routingValue = routingValue;
}
}

My Service needs an interface such as:

package com.test.common;
public interface PricingService{

public Double getAveragePriceOfAllWidgets();
}

Next, I need to implement the service:

package com.test;

import org.openspaces.core.GigaSpace;
import org.openspaces.core.context.GigaSpaceContext;
import com.j_spaces.core.client.SQLQuery;
import com.test.common.PricingService;
import com.test.common.Widget;

public class PricingServiceImpl implements PricingService{
@GigaSpaceContext
GigaSpace space;

public Double getAveragePriceOfAllWidgets() {
  Object[] allWidgets = space.readMultiple(new Widget(), Integer.MAX_VALUE);
  double priceTotal = 0.0d;
  for(int x=0;x<allWidgets.length;x++){
   priceTotal=priceTotal+((Widget)allWidgets[x]).getPrice().doubleValue();
  }
  double averagePrice = priceTotal/allWidgets.length;
  return new Double(averagePrice);
}
}

Now I need a client that uses this service:

package com.test;
import java.util.TimerTask;
import com.test.common.PricingService;

public class WidgetSurfingClient extends TimerTask{

private PricingService service;
public void setPricingService(PricingService ref){
this.service=ref;
}
public void run(){
System.out.println(
”The average price of all widgets is now: $”+
service.getAveragePriceOfAllWidgets());
}
}

Now I need something to do the “reduce” side of the map-reduce: (it also performs the final average calculation because I am looking for the average price…

package com.test;
import org.openspaces.remoting.RemoteResultReducer;
import org.openspaces.remoting.SpaceRemotingInvocation;
import org.openspaces.remoting.SpaceRemotingResult;

public class AveragingDoubleResultReducer implements RemoteResultReducer<Double,Double>{

public Double reduce(SpaceRemotingResult<Double>[] results,
SpaceRemotingInvocation remotingInvocation) throws Exception {
  double totalDouble = 0.0d;
  int size = results.length;
  for (int x=0;x<size;x++){
   if(results[x]!=null){
    if(results[x].getResult()!=null){
      totalDouble += (Double) results[x].getResult();
    }
   }
  }
  double averageDouble=totalDouble/size;
  return new Double(averageDouble);
}
}

That takes care of the Java code, now we have to configure this system:

First: the Space-side configuration:

<!– declare the transaction manager –>
<os-core:local-tx-manager id=”transactionManager”
space=”widgetTransport”/>

<bean id=”PricingService” class=”com.test.PricingServiceImpl”/>

<!– provide a gigaspace context to the pricing service –>
<os-core:giga-space-context/>

<!–
Provide the binding to the Space-side
transport for the PricingService :
–>
<os-remoting:service-exporter id=”PricingServiceExporter”>
<os-remoting:service ref=”PricingService”/>
</os-remoting:service-exporter>

<!– configure a big or small cluster
(here we have 20 partitions with 1 backup for each) –>
<os-sla:sla cluster-schema=”partitioned-sync2backup”
number-of-instances=”20″ number-of-backups=”1″
max-instances-per-vm=”1″>
</os-sla:sla>

</beans>

Next: the client-side configuration:

<!– bootstrap the basic proxy for the serviceCluster: –>
<os-core:space id=”widgetTransport” url=”jini://*/*/widgetsystem” />

<!– wrap the proxy in Spring-ready OpenSpaces API –>
<os-core:giga-space id=”gigaspace” space=”widgetTransport” />

<!– WidgetSurfer Service –>
<bean id=”WidgetSurfer” class=”com.test.WidgetSurfingClient”>
<property name=”pricingService” ref=”PricingService”/>
</bean>

<os-remoting:sync-proxy id=”PricingService”
            giga-space=”gigaspace”
            broadcast=”true”
            interface=”com.test.common.PricingService”>
<os-remoting:result-reducer ref=”AveragingDoubleResultReducer”/>
</os-remoting:sync-proxy>

<!– this is regular Spring TimerTask configuration stuff: –>
<bean id=”widgetClientTask” class=”org.springframework.scheduling.timer.ScheduledTimerTask” >
<!– wait 1 seconds before starting repeated execution –>
<property name=”delay” value=”1000″ />
<!– run every 5 seconds –>
<property name=”period” value=”5000″ />
<property name=”timerTask” ref=”WidgetSurfer” />
</bean>

<bean id=”widgetClientTimer” class=”org.springframework.scheduling.timer.TimerFactoryBean”>
   <property name=”scheduledTimerTasks”>
    <list>
      <!– This wires the factory to the scheduledTask bean above –>
      <ref bean=”widgetClientTask” />
    </list>
   </property>
</bean>
</beans>

Cheers,

Owen.

Amazon gets more elastic

March 27th, 2008

Amazon EC2 now makes it possible for failure to be less noticeable within EC2 due to the new ability offered to have an IPAddress move from box to box upon failure. This is really cool because it means you can host your webserver as a service within the GigaSpaces AMI and when it automatically fails over due to our service management and self-healing behaviors, the end user will have a hiccup but will be able to refresh and continue without changing the URL!

Another cool new thing EC2 is offering is the ability to host primary and backup nodes in different physical “Zones” - meaning Amazon will guarantee that the backup location is not on the same power grid as the primary. Using GigaSpaces to create and deploy the instances makes it even easier to ensure this desired behavior is achieved.

See Amazon’s Feature Guides for Elastic IP Addresses and Availability Zones.

Learn how to build your first EC2 Applications using GigaSpaces

Cheers,

Owen.

« Previous Page — Next Page »