Archive for the 'Java' Category

Next Page »

Video is worth a million words. . . Latest GigaSpacesXAP demo

August 12th, 2008

With my wife and kids away, I am having a geeky bachelor time that many who read this would probably find quite thrilling. For example: I spent this last weekend producing a new MonteCarlo application example for GigaSpaces.

It is rich in the patterns it utilizes:

The app goes like this:

Historical information regarding several funds is used to predict the future behavior of those funds over a 20 year time period with Minute by Minute fluctuation in prices being calculated and applied.
These funds are grouped in different formations into Portfolios and the entire Portfolio assessed for its growth over the same 20 year period.

In the end, the best combination of funds within a portfolio is determined and reported and the best and worst behaving funds are also shown.

Technically, several portfolios and their associated funds are written as tasks into the system for analysis – the logic needed to process them and their historical information is sent along with them as well. Workers take the tasks, process the associated logic and return a result showing the outcome of the application of the expected variation in price for each fund in the portfolio over the designated 20 year period.

This is performed over a set of possible portfolios – each containing different variations of funds.
Each set is repeatedly simulated to avoid undo skewing and allow for better control of the randomness.

In the end, the client requests a summary of all the results which is created by a service that operates in parallel across the available spaces to take all the results and sort them and aggregate them to determine the most common, best [highest value portfolio], and worst [lowest value portfolio]

This information is returned to the client and printed out to the console.

This morning, I produced a nifty 8 minute flash presentation showing my Monte Carlo application being executed on the GigaSpaces platform.

NB: To keep the video short, and because I only used my laptop, I kept the size of the sets and the iterations at very low levels. Nevertheless, the application scales very nicely and provides fault-tolerance as well.

You can check out the 8 minute video using this link: THE MONTECARLO VIDEO

[you may have to wait a while for the video to load, if it stops
mid-way, play it again from the beginning after it loads completely]

I hope you enjoy watching the fruits of my labors – what could be a better way to spend a bachelor weekend eh?

Cheers,

Owen.

Bridging the gap between the clouds

August 9th, 2008

Dekel Tankel of GigaSpaces spoke recently to a hip cloud crowd regarding the risks associated with moving an application to the cloud or grid environment.

Without a GigaSpaces Space-based architecture (what I refer to as a TPC Architecture ) applications running in a cloud suffer from an inability to scale and start to erode the benefits of the cloud. In addition there are questions regarding the ability to maintain state reliably in a cloud: especially a pay by the drink cloud where it can get expensive both in terms of $$ and time to utilize the persistence mechanisms provided by the vendor.

Dekel speaks about and demonstrates how GigaSpaces makes it possible to manage your application services -moving them from machine to machine as needed and also addresses scaling the application by spreading it out [partitioning it while removing the traditional bottlenecks of the database and network boundaries between services]

video

He does a nice job – way to go Dekel!

Cheers,

Owen.

TSS Prague

June 20th, 2008

TSS at Prague was great. Great sessions, always great meeting up with people from all over the world. Sadly, I did not manage to get to too many sessions, had to work a bit on GigaSpaces upcoming 6.5 GA release, but the ones I went to were really good.
I also did a tech talk with [...]

SyncRemoting cookbook

June 4th, 2008

I wrote up a cheat sheet for myself when building scatter-gather or map reduce apps with GigaSpacesXAP.

It goes like this:

Create business interface (as part of shared classes)

Space-side stuff:

Create the Space-side implementation (logic) [ This may use a GigaSpace reference ]
Configure the implementation on the Space-Side:
Declare the business implementation as a [bean]
Declare a filter as part of the space
The filter references a ServiceExporter
Declare the os-remoting:service-exporter
The exporter references the business implementation [the bean]
If impl uses space: Declare the giga-space-context

Client-side stuff

Declare os-remoting:sync-proxy with its attributes:
The business interface it implements
The giga-space it uses as a transport layer
Whether broadcast is true or false
If necessary –declare the reducer for the proxy
Inject the client with the service proxy (ie: public void setMyService(MyService ref){} )

Let’s say we want to do a Map reduce exercise where each node in the cluster will process its share in parallel and return a total result for my digestion…

Let’s say I want to know the average price for all my widgets in all my inventory, but my inventory is scattered across a dozen servers because I do so much business…

I have my widgets:

package com.test.common;

import com.gigaspaces.annotation.pojo.SpaceRouting;

public class Widget {

private Double price;

 private String description;

 private Integer routingValue;
 
 public Double getPrice() {
  return price;
 }
 public void setPrice(Double price) {
  this.price = price;
 }
 public String getDescription() {
  return description;
 }
 public void setDescription(String description) {
  this.description = description;
 }
 @SpaceRouting
 public Integer getRoutingValue() {
  return routingValue;
 }
 public void setRoutingValue(Integer routingValue) {
  this.routingValue = routingValue;
 }
}

My Service needs an interface such as:

package com.test.common;
public interface PricingService{

 public Double getAveragePriceOfAllWidgets();
}

Next, I need to implement the service:

package com.test;

import org.openspaces.core.GigaSpace;
import org.openspaces.core.context.GigaSpaceContext;
import com.j_spaces.core.client.SQLQuery;
import com.test.common.PricingService;
import com.test.common.Widget;

public class PricingServiceImpl implements PricingService{
 @GigaSpaceContext
 GigaSpace space;

 public Double getAveragePriceOfAllWidgets() {
  Object[] allWidgets = space.readMultiple(new Widget(), Integer.MAX_VALUE);
  double priceTotal = 0.0d;
  for(int x=0;x<allWidgets.length;x++){
   priceTotal=priceTotal+((Widget)allWidgets[x]).getPrice().doubleValue();
  }
  double averagePrice = priceTotal/allWidgets.length;
  return new Double(averagePrice);
 }
}

Now I need a client that uses this service:

package com.test;
import java.util.TimerTask;
import com.test.common.PricingService;

public class WidgetSurfingClient extends TimerTask{

 private PricingService service; 
 public void setPricingService(PricingService ref){
 this.service=ref;
 }
 public void run(){
 System.out.println(
 ”The average price of all widgets is now: $”+ 
 service.getAveragePriceOfAllWidgets());
 }
}

Now I need something to do the “reduce” side of the map-reduce: (it also performs the final average calculation because I am looking for the average price…

package com.test;
import org.openspaces.remoting.RemoteResultReducer;
import org.openspaces.remoting.SpaceRemotingInvocation;
import org.openspaces.remoting.SpaceRemotingResult;

public class AveragingDoubleResultReducer implements RemoteResultReducer<Double,Double>{
   
 public Double reduce(SpaceRemotingResult<Double>[] results,
 SpaceRemotingInvocation remotingInvocation) throws Exception {
  double totalDouble = 0.0d;
  int size = results.length;
  for (int x=0;x<size;x++){
   if(results[x]!=null){
    if(results[x].getResult()!=null){
      totalDouble += (Double) results[x].getResult();
    }
   }
  }
  double averageDouble=totalDouble/size;    
  return new Double(averageDouble);
 }
} 

That takes care of the Java code, now we have to configure this system:

First: the Space-side configuration:

<beans … XML namespace declaration stuff not shown…>
<!– bootstrap the construction of the serviceCluster: –>
<os-core:space id=”widgetTransport” url=”/./widgetsystem”>
 <!– setup the PricingServiceExporter to handle calls –>
 <os-core:filter-provider ref=”PricingServiceExporter”/>
</os-core:space> 
 
<!– wrap the serviceCluster in Spring-ready OpenSpaces API –>
<os-core:giga-space id=”gigaspace” space=”widgetTransport”
  tx-manager=”transactionManager” />

<!– declare the transaction manager –>
<os-core:local-tx-manager id=”transactionManager”
  space=”widgetTransport”/>
       
<bean id=”PricingService” class=”com.test.PricingServiceImpl”/>

<!– provide a gigaspace context to the pricing service –>
<os-core:giga-space-context/>

<!–
Provide the binding to the Space-side
transport for the PricingService :
–>
<os-remoting:service-exporter id=”PricingServiceExporter”>
  <os-remoting:service ref=”PricingService”/>
</os-remoting:service-exporter>

<!– configure a big or small cluster
  (here we have 20 partitions with 1 backup for each) –> 
<os-sla:sla cluster-schema=”partitioned-sync2backup”
   number-of-instances=”20″ number-of-backups=”1″
   max-instances-per-vm=”1″>
</os-sla:sla>

</beans>

Next: the client-side configuration:

<beans … XML namespace declaration stuff not shown…>

<!– bootstrap the basic proxy for the serviceCluster: –>
<os-core:space id=”widgetTransport” url=”jini://*/*/widgetsystem” />

<!– wrap the proxy in Spring-ready OpenSpaces API –>
<os-core:giga-space id=”gigaspace” space=”widgetTransport” />

<!– WidgetSurfer Service –> 
<bean id=”WidgetSurfer” class=”com.test.WidgetSurfingClient”>   
  <property name=”pricingService” ref=”PricingService”/>
</bean> 

<bean id=”AveragingDoubleResultReducer” class=”com.test.AveragingDoubleResultReducer”/>

<os-remoting:sync-proxy id=”PricingService”
            giga-space=”gigaspace”
            broadcast=”true”
            interface=”com.test.common.PricingService”>
  <os-remoting:result-reducer ref=”AveragingDoubleResultReducer”/>
</os-remoting:sync-proxy>

<!– this is regular Spring TimerTask configuration stuff: –>
  <bean id=”widgetClientTask” class=”org.springframework.scheduling.timer.ScheduledTimerTask” >
   <!– wait 1 seconds before starting repeated execution –>
   <property name=”delay” value=”1000″ />
   <!– run every 5 seconds –>
   <property name=”period” value=”5000″ />
   <property name=”timerTask” ref=”WidgetSurfer” />   
  </bean> 
 
  <bean id=”widgetClientTimer” class=”org.springframework.scheduling.timer.TimerFactoryBean”>
   <property name=”scheduledTimerTasks”>
    <list>
      <!– This wires the factory to the scheduledTask bean above –>
      <ref bean=”widgetClientTask” />
    </list>
   </property>
  </bean>
</beans>

Cheers,

Owen.

Integrating GigaSpaces persistency service into an existing tier based system

April 23rd, 2008

A common issue I’m facing recently is how to integrate existing tier based applications with GigaSpaces persistency service, AKA persistency as a service (Paas) or mirror . The motivation is often a result of the acknowledgment that a standard tier based application fails to scale when facing the database throughput limitation.
Software Caching technologies (overlooking their [...]

Next Page »