GigaSpaces includes several runtime options and API to access the IMDG. These includes: – Local Cache – Local View (available only for GigaSpace) – Embedded Space – Remote space Where the API to access your data could be GigaSpace API or GigaMap API (JMS API and JDBC API are also available). This post will compare
READ MORE…
When you download the GigaSpacesXAP product (currently at version 6.6.1) you get a few really nice examples.
With those examples come some very detailed instructions and explanations as to how to use GigaSpaces. These instructions are found in the “docs” subfolder under the example root and are written in rich HTML with beautiful screenshots and graphics to help guide the reader. Example: GS_HOME/examples/helloworld/docs Despite the existence of these terrific resources, for some people who are new to the GigaSpaces experience, it might be helpful to have a set of instructions in the traditional readme.txt file. ( I can be so old fashioned sometimes…) I created the following in the hopes that it will prove useful and speed up the adoption of this wonderful application platform known as GigaSpacesXAP.Here goes…
(Imagine this readme is located in the root of the helloworld example like this: GS_HOME/examples/helloworld/readme )
This example shows the basic interaction between a processing unit (where objects are processed) and a simple client that feeds the processing unit with objects to be processed.
There are two ways to run this example: (provided as scripts in this directory)
The first way is simplest and shows the processing unit running in one process and the Feed running in another. Neither of the processes are managed and no clustering technology is employed to provide fail-over or scaling.
To try out the example in this simple way read and follow the first set of instructions below.
[To try out the more complex topology, read and follow the second set of instructions below.]
_______________________________________
FIRST SET OF INSTRUCTIONS::
_______________________________________
This example does the following:
1) starts the processing unit containing the processor and a space
[this is where the work of processing information goes on]
2) Starts the client-side Feeder
[this populates the system with 1000 objects to be processed]
Instructions:
1) Ensure you have GigaSpacesXAP6.6 or higher
2) Ensure you have JDK1.5
3) Navigate to the directory containing this example readme file
Execute the following
> build.sh dist
It will:
a) Build the application and create the jar file versions of the deployment units
Next Execute:
> ../../bin/puInstance.sh ./processor/pu/hello-processor.jar
This will:
b) Start the space-side system which includes a processor and a space
Edit the build.xml file so that this line:
<target name=”run-feeder” depends=”dist”>
Is changed to equal this:
<target name=”run-feeder” depends=”">
Next Execute:
> build.sh run-feeder
This will:
c) Start the client-side Feeder which populates the system with 1000 objects
If you want to use a GUI to examine what is running, execute:
>../../bin/gs-ui.sh
If you want to feed more information into the system,
you can execute the
> build.sh run-feeder
more than once.
Note: If you wish to open the three Eclipse projects that make up the source for
this example in Eclipse, be aware that they use a variable GS_HOME that must be
configured in your Eclipse workspace. It should point to the install
directory/folder of GigaSpacesXAP6.5
_______________________________________
SECOND SET OF INSTRUCTIONS::
_______________________________________
This example does the following:
1) Starts the GigaSpacesXAP runtime environment (Service Grid)
2) Deploys the space-side system as a “cluster” which is split into 4 processing units:
2 partitions
(which divide the total work up between them)
2 dedicated backups
(one for each active partition)
3) Starts the client-side Feeder
[this populates the entire system with 1000 objects to be processed]
Instructions:
1) Ensure you have GigaSpacesXAP6.6 or higher
2) Ensure you have JDK1.5
3) Navigate to the directory containing this example readme file
Execute the following:
> build.sh dist
It will:
a) Build the application and create the jar file versions of the deployment units
Next Execute: (add piping to a log file if you like)
> ../../bin/gsc.sh &
> ../../bin/gsc.sh &
> ../../bin/gsm.sh &
This will:
b) Start the service grid which is the Gigaspaces application server runtime environment
Next Execute:
> ../../bin/gs-ui.sh &
This will:
c) Start the gs-ui so you can see when the service grid is started
(Switch to the middle tab on the left-hand side of the GUI called “Deployments,Details” and you should see two boxes in the bottom left of the screen. Those are your empty GSC runtime containers.)
Edit the build.xml file so that this section:
<macrodef name=”deploy”>
<attribute name=”name”/>
<sequential>
<java classname=”org.openspaces.pu.container.servicegrid.deploy.Deploy” fork=”false”>
<classpath refid=”all-libs”/>
<arg value=”-groups”/>
<arg value=”${groups}”/>
<arg value=”-timeout”/>
<arg value=”15000″/>
<arg value=”@{name}”/>
</java>
</sequential>
</macrodef>
Is changed to equal this:
<macrodef name=”deploy”>
<attribute name=”name”/>
<sequential>
<java classname=”org.openspaces.pu.container.servicegrid.deploy.Deploy” fork=”false”>
<classpath refid=”all-libs”/>
<arg value=”-groups”/>
<arg value=”${groups}”/>
<arg value=”-timeout”/>
<arg value=”90000″/>
<arg value=”-locators”/>
<arg value=”localhost”/> <!– assumes GSM is on same host as you–>
<arg value=”-cluster”/>
<arg value=”schema=partitioned-sync2backup”/>
<arg value=”total_members=2,1″/>
<arg value=”@{name}”/>
</java>
</sequential>
</macrodef>
This edit changes the topology of the application being deployed so that there will be 2 primary active instances and 2 backup instances deployed into the service grid.
Next Execute:
> build.sh deploy-processor
This will deploy the newly re-defined 2 primary spaces and accompanying workers and their backup service instances into the service grid (you will see them appear in the gs-ui GUI)
Wait for all the nodes to appear. (There should be 2 nodes with a ‘p’ for primary and 2 nodes with a ‘b’ for backup)
This can take a while… (up to 2 minutes the first time depending on the machine and network)
Next Execute:
> build.sh run-feeder
This will:
c) Start the client-side Feeder which populates the system with 1000 objects
If you want to feed more information into the system,
you can execute the
> build.sh run-feeder
more than once.
Note: If you wish to open the three Eclipse projects that make up the source for
this example in Eclipse, be aware that they use a variable GS_HOME that must be
configured in your Eclipse workspace. It should point to the install
directory/folder of GigaSpacesXAP6.6
HTH
Owen.
I recently created an integration with SunGrid Engine. This was easy to do – requiring only that a listener be written that hears JMX events that are produced by our product. As you may know, with GigaSpaces you can create watches for the services you deploy. These watches are populated with information coming from getter methods on those services.
For example: you could have a service that exposes the method: getBacklog() that returns a long
Then in your pu.xml you set up the watch for that property:
<os-sla:sla max-instances-per-vm=”2″>
<os-sla:scale-up-policy monitor=”backlog” max-instances=”12″ high=”200″ />
<os-sla:requirements>
<os-sla:cpu high=”.65″ />
<os-sla:memory high=”.75″ />
< /os-sla:requirements>
<os-sla:monitors>
<os-sla:bean-property-monitor name=”backlog”
bean-ref=”adaptiveOrderProcessor”
property-name=”backlog”
period=”2000″ />
< /os-sla:monitors>
< /os-sla:sla>
In this example, when the value returned from the getBacklog method is over 200, it triggers a scaling event which adds another instance of the service named adaptiveOrderProcessor to the running system.
NOTE: that the scaling event effects the entire population of the ProcessingUnit of which that service is a part. To scale one service only, you must define that one service alone in its own pu.xml file.If (as in this example) that service has a limitation on the number of instances that can run in a single GSC that is expressed here:
<os-sla:sla max-instances-per-vm=”2″>and means that at some point, the available GSCs will not be enough to host all the possible instances
<os-sla:scale-up-policy monitor=”backlog” max-instances=”12″ high=”200″/>
here we state we want a max of 12 instances so we need 6 GSC instances to host them all.
What I did with SUN was to write a JMX listener that listened for “ProvisionerFailureEvents” which are created by the GSM when it gets a scaling or relocation or failover event that provokes the GSM to seek a host for that service. When the GSM cannot find a suitable host because there are not enough GSCs running, the GSM sends out a ProvisionerFailureEvent which is what my code listens for. When the event hits, my code simply calls the API of the Grid technology in question and asks for the creation of a new GSC. In other words, the service in question says: “help me, I must relocate” or, “Help me, I must failover”, or “help me, I must have more of me running on the network because one of me is not enough!” The GSM says, ” I will start you somewhere else. . .” but then the GSM says, “Oh, golly! There is no where else to start you!” And then the GSM says, “Help me someone!” and sends the ProvisionerFailureEvent to JMX hoping some force in the universe will care.Once the universe shows an interest and starts a new GSC, the GSM will retry the scaling, failover, or relocation effort and utilize that new resource allowing the declared SLA to be satisfied.
Bottom line is: it is simple to integrate GigaSpaces with any grid management solution that exposes an API and in doing that integration, enable the dynamic addition of resources to allow the relocation (or failover or scaling) of an application on the fly. Other watches you might set up include: getLocalTimeOfDay() where the value measured causes a relocation event that could move applications to new machines in different timeZones – allowing you to “follow the sun”. getMemoryConsumption() where the value measured causes a relocation event that moves applications to new GSCs that have more memory. getCPULevel() where the value measured causes a relocation event that moves applications to new GSCs with more CPU capacity. etc.. Again, the choice if scaling in response to an event or relocating is yours but is also dependent on the type of service you are affecting. If the service involved has an embedded space in the same PU.xml file, you can not scale it using scaling events. To accomplish this behavior, you must relocate the processing unit which will allow you to scale to the limits of the number of partitions you defined for that space when it was deployed. Example: you deploy a space and worker to the system and define the space as <os-sla:sla cluster-schema=”partitioned-sync2backup” number-of-instances=”24″ number-of-backups=”1″ max-instances-per-vm=”1″>This means you have 24 partitions defined. You first deploy this processing unit to 3 GSCs each having 2gb ram where you run 16 instances in each GSC (8 partitions and 8 backups)
These instances run happily until they start running low on memory. At that point a watch on one of the workers could trigger a relocation event which asks the GSM to move one of the instances of the PU to a new GSC. Presuming there is a GSC with the necessary memory and CPU available … (as defined in the following section where it is specified that the PU will not be deployed unless no more than 25% of the CPU and memory is utilized in a target GSC) <os-sla:requirements>
