Welcome!

Log Management Authors: Dana Gardner, Pat Romanski, Elizabeth White, David H Deans, Carmen Gonzalez

Related Topics: Containers Expo Blog, Microservices Expo, @CloudExpo

Containers Expo Blog: Article

Data Virtualization - RDBMS vs Middleware

Data virtualization platforms

As discussed in my previous article, Data Virtualization is the new enterprise data integration pattern for the petabyte enterprise that depends on Cloud for a dynamic allocation of resources to satisfy information needs. We have also seen several attributes of data virtualization that fits the needs of the new enterprise.

As an illustration of data virtualization at work, we have demonstrated the usage of SQL Server as a data virtualization engine that utilizes the relational database engine as a data virtualization server and delivers the results.

However, utilizing the existing RDBMS as a data virtualization engine is not the only option and we can utilize specialized engines outside of RDBMS as a data virtualization engine. This concept is not new.

ROLAP vs MDOLAP
In the decision-support systems, OLAP (Online Analytical Processing) refers to a multi-dimensional view of aggregate data to provide quick access to strategic information for further analysis.

MDOLAP or MOLAP, multi-dimensional OLAP servers, employ dedicated OLAP engines optimized to manage sparse matrices of data. MOLAP storage management maintains the physical storage of OLAP cubes. It has got the advantage of high performance and response time to queries while it is quite proprietary and cannot scale for large data sets.

ROLAP technology accesses data stored in a relational database to provide OLAP analysis without the requirement to store and calculate data in a multi-dimensional cube. Relational databases serve as the database layer for data storage, access and retrieval processes. ROLAP has the advantage of storing large data sets. However the response time is not fast as MOLAP.

While the idea is not to compare between ROLAP and MOLAP but it is more about , certain data integration and access patterns can be either achieved with pure RDBMS implementation or specialized middle tier servers specifically built for them.

In that context apart from analyzing the traditional RDBMS as a data virtualization engines in the last article, we also wanted analyze specialized data virtualization engines that specifically meant for the purpose.

Data Virtualization Using a Composite Platform
The Composite Data Virtualization Platform
provides a middle-tier platform outside of the relational databases that helps to integrate data from multiple disparate sources in a unified, logically virtualized manner for access by various front-end technologies.

This solution provides a virtual data abstraction layer on top of the disparate data sources.

At the heart of this virtualization platform, the Composite Information Server acts a virtual database layer and facilitates the following core tenants of data virtualization.

  • Federates and queries data across disparate data sources. This provides integration across multiple data sources like Big Data, mainframes, RDBMS, web services, messages, Microsoft Office, etc.
  • Performance optimization for the queries that converges data from multiple data sources
  • QoS factors like caching and security
  • Abstraction layer to deliver data to consuming applications

Another interesting and useful feature of this data virtualization platform and that's not available out-of-the-box in the traditional RDBMS implementation of data virtualization is the Performance Plus Adapters.' This feature converges data from enterprise applications like SAP, Siebel, Oracle E Business Suite, as well as traditional OLAP platforms such as Oracle Essbase, SAP BW and newer analytical databases like HP Vertica, Netezza and even Big Data implementations like Hadoop. This feature enables the creation of entire enterprise application integration patterns using the data virtualization layer, replacing or augmenting the Enterprise Service Bus (ESB).

While most cases of data virtualization solutions are good to visualize, their performance always subject to issues due to the latency involved in connecting to disparate data sources. To solve this issue, this platform also provides a ‘Composite Active Cluster' that acts like MSCS for Microsoft SQL Server implementation, with features like.

  • Active/Active Clustering
  • Shared Cluster Cache
  • Replicated Metadata Repository

Summary
As seen in several articles, data virtualization will present a useful value proposition for enterprise data integration. Availability of multiple options will help the enterprises evaluate them for their needs and budgets and fit them accordingly. Apart from the traditional relational databases, specialized engines like the one by the Composite software provide multiple features to implement data virtualization in the enterprises.

More Stories By Srinivasan Sundara Rajan

Highly passionate about utilizing Digital Technologies to enable next generation enterprise. Believes in enterprise transformation through the Natives (Cloud Native & Mobile Native).

IoT & Smart Cities Stories
If a machine can invent, does this mean the end of the patent system as we know it? The patent system, both in the US and Europe, allows companies to protect their inventions and helps foster innovation. However, Artificial Intelligence (AI) could be set to disrupt the patent system as we know it. This talk will examine how AI may change the patent landscape in the years to come. Furthermore, ways in which companies can best protect their AI related inventions will be examined from both a US and...
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities - ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups.
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...
Chris Matthieu is the President & CEO of Computes, inc. He brings 30 years of experience in development and launches of disruptive technologies to create new market opportunities as well as enhance enterprise product portfolios with emerging technologies. His most recent venture was Octoblu, a cross-protocol Internet of Things (IoT) mesh network platform, acquired by Citrix. Prior to co-founding Octoblu, Chris was founder of Nodester, an open-source Node.JS PaaS which was acquired by AppFog and ...
The deluge of IoT sensor data collected from connected devices and the powerful AI required to make that data actionable are giving rise to a hybrid ecosystem in which cloud, on-prem and edge processes become interweaved. Attendees will learn how emerging composable infrastructure solutions deliver the adaptive architecture needed to manage this new data reality. Machine learning algorithms can better anticipate data storms and automate resources to support surges, including fully scalable GPU-c...
Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
The standardization of container runtimes and images has sparked the creation of an almost overwhelming number of new open source projects that build on and otherwise work with these specifications. Of course, there's Kubernetes, which orchestrates and manages collections of containers. It was one of the first and best-known examples of projects that make containers truly useful for production use. However, more recently, the container ecosystem has truly exploded. A service mesh like Istio addr...
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
Whenever a new technology hits the high points of hype, everyone starts talking about it like it will solve all their business problems. Blockchain is one of those technologies. According to Gartner's latest report on the hype cycle of emerging technologies, blockchain has just passed the peak of their hype cycle curve. If you read the news articles about it, one would think it has taken over the technology world. No disruptive technology is without its challenges and potential impediments t...