The world’s most valuable currency is data and every business is becoming a data business. Cisco notes that by 2020, 50B devices and 212B sensors will join the internet. In 2020, it is expected that the average internet user will generate ~1.5 GB of traffic per day, more than doubling since 2015. According to Forbes, a Smart Hospital will generate 3,000 GB per day, self-driving cars are generating over 4,000 GB per day, a connected plane will generate 40,000 gigabytes per day, and a connected factory will generate 1 million gigabytes per day.

That’s a lot of data. And it is changing the nature of competition.

Enterprises are seeking the most efficient ways to process and analyze the burgeoning data volumes within their organizations to gain the competitive edge.

The Competitive Edge

In today’s high-speed world, it has become essential for businesses to transform how they collect, manage, and analyze data to make decisions. That is, they must become an insight-driven organization in order to maintain a competitive edge.

The ability to extract “insight” from the data and instantly act in real time helps reduce costs, gain a better understanding of your customers’ behaviors and needs, create new innovations, launch new businesses, and react more quickly in today’s digital economy.  

According to a recent report from Frost & Sullivan, financial markets, adtech, cybersecurity, and others, now find themselves competing in “the millisecond economy,” where survival is no longer possible without putting the freshest insights at the fingertips of their people.

This shift towards using real-time (hot) data to make real-time or near-real-time business decisions is happening as organizations begin to converge their operational systems with their management systems, with the goal of becoming more efficient and data-driven by pulling actionable insights at the same time that data is born.

GigaSpaces InsightEdge Platform 12.3

The Necessary Components

We believe that the value of insight-driven innovation is best realized through the convergence of three fundamental components that, when combined, can transform virtually every element of an enterprise.

Component 1:  Eliminate the three most expensive letters in fast data analytics: E-T-L by co-locating analytics and transactional workloads that handles low-latency, multi-tier data storage (over RAM, SSD and Storage-class Memory), while giving flexibility to a multitude of data management and query APIs.

Component 2:  Leverage and simplify cloud-native development patterns using microservices architecture through XAP In-Memory Data Grid for real-time applications.

Component 3: In-data integration of Apache Spark workloads and transactional data provide online machine learning and fast analytics against data in motion.

This represents a new and exciting computing paradigm that allows unprecedented speed and scale for applications that run the business and those that manage it to seamlessly integrate.

InsightEdge Platform 12.3

In this release of InsightEdge, we continue to strive to meet the needs of the real-time millisecond enterprise. We are focusing on simplifying and scaling our platform so that our customers can innovate with confidence to become insight-driven.

Take Control: Enhance Performance and Scale as Your Data Grows

The latest release introduces a unique multi-tiered data storage approach to the MemoryXtend module that allows customized preferences for data prioritization per application and lowering RAM footprint. Customers can configure the system to ensure that the most important data resides in the fastest data storage tier.

In addition to the existing MemoryXtend storage driver for storing data in SSD (or other disk types), we’re introducing a new storage driver for Off-Heap RAM. Data stored in Off-Heap RAM consumes significantly less memory than data stored on Java’s heap, which boosts the grid’s capacity. A smaller heap means less work for Java’s Garbage Collector, resulting in systems which are stabler and more predictable.

The MemoryXtend module includes an on-heap caching layer, to optimize performance for frequently queried data. Starting in v12.3, users can define which data should be cached on-heap, by specifying a 1 or more SQL-like queries to determine the caching criteria. Since those queries rely on the data itself, it means queries can enter or leave the on-heap cache according to changes in the data. This on-heap cache can be used with all storage drivers – the existing one for SSD, the new one for off-heap RAM, and future drivers.

The SSD storage driver has been enhanced with an off-heap cache as well, to store indices values. This allows operations or queries which use only indexed data to be executed using heap and off-heap alone, with no SSD access, providing better performance.

Real-life systems tend to be complex, and often require a mixture of storage capabilities which defer from system to system.

This new set of MemoryXtend features put the developers in control more than ever before, allowing them to tailor the InsightEdge platform according to their requirements to optimize the business results of each application while lowering total cost of ownership (TCO).  

Simplicity Accelerates Innovation

Real-time Analytics Made Easy

InsightEdge 12.3, a single-tier, in-memory insight platform, also supports the most recent release of Apache Spark 2.3. By leveraging the complete Spark ecosystem, collocated with transactional data and applications running in the grid, across one distributed fabric, organizations can accelerate real-time AI and machine learning innovation.

“The traditional batch-oriented cycle of business intelligence insight is giving way to new deployment

architectures delivering real-time access to data and the expected insights that it will provide,” said Adam M. Ronthal and Roxane Edjlali in  Gartner Research, Delivering Digital Business Value Using Practical Hybrid Transactional/Analytical Processing (April 2017). “In-memory computing provides performance that removes the need for competing operational and analytical strategies for accessing data. When all the data can fit in active memory, complicated aggregations, summaries and even predictive modeling can now be performed virtually, with no requirement to update and build the supporting structures that previously required dedicated platforms to accomplish.”

You Choose: Cloud, On-Premise or Hybrid

The first release of the InsightEdge Platform provided two separate command lines – one for InsightEdge and another for XAP. In InsightEdge 12.3 we’re introducing a new unified and seamless command line interface for both XAP and InsightEdge. The new command line interface, in addition to being intuitive and sporting a modern syntax, is based on the recently-added Manager REST API, making it ideal for managing cloud-based systems from your machine with no extra effort.

In v12.3, there’s an official Docker image for both InsightEdge and XAP; open source and enterprise. The Docker images are available online at the docker hub, and make it easier to experiment with the products on your local machine, test elaborate distributed scenarios in your staging environment or upgrade production. What we found most attractive about Docker is that it’s cloud-friendly but not cloud-only.  Some of our customers are already on the cloud, while others are still preparing to begin the journey. Using Docker simplifies environment setup on-premise as well as moving to the cloud, since all the major cloud vendors support docker.

“With this InsightEdge Platform release, GigaSpaces continues to meet the needs of the real-time millisecond enterprise,” said Adi Paz, CEO of GigaSpaces. “A critical competitive advantage results from the ability to out-innovate its rivals by leveraging the data within the organization to gain instant insights that can immediately impact the business applications.  Insight Edge 12.3 enhancements continue to simplify and scale our platform so that our customers can innovate with confidence to become insight-driven.”

Further links:

We invite you to give the new InsightEdge a try or to join our upcoming webinar on What’s New in InsightEdge Platform 12.3 to learn more about our offering.

InsightEdge Platform 12.3: Accelerating Innovation, Taking control on the Cloud
Niv Ingberg on GithubNiv Ingberg on LinkedinNiv Ingberg on Twitter
Niv Ingberg
Architect & Lead Developer @ GigaSpaces
Solution architect & lead developer at core team in R&D division. Evaluating and adapting new technologies, assessment of future customer needs and designing new solutions.