By Ananth Chakravarthy
We often focus on improving the speed and efficiency of our data infrastructures, but it is equally important to ensure their longevity and resilience. Especially in this world of rapidly evolving technologies, it is critical to adopt and maintain flexible infrastructures. Over the years, data infrastructures have undergone significant transformations, and this trend will continue.
In the 1970s, it began with flat-file data structures, followed by the emergence of databases in the 1980s. The 1990s introduced data warehouses, star schemas, and multi-dimensional data structures. The 2000s witnessed the rise of Web data, social data, and, more recently, blob storage, columnar storage, data lakes, and machine-generated or Internet of Things (IoT) data. As the list expands, data architectures become increasingly complex.
The Modern Era
How long should a technology solution last? If the current technology meets the requirements and serves its purpose, should we rush to replace it? While there has been significant emphasis on acquiring solutions that deliver faster data delivery and performance, such as OLAP cubes and MPP capabilities, these approaches only partially solve the challenges faced by businesses aiming to become more data-centric and accelerate data delivery for improved business outcomes.
How can we modernize our data landscape while leveraging existing technology investments? How can we invest in technology that will endure and minimize the need for frequent changes and disruptions across the organization? We need a data infrastructure that can carry us into the future with minimal disruption. We require an agile architecture capable of adapting quickly to change. It should enable us to seamlessly incorporate different technologies as needed while minimizing the impact on users.
The Logical Approach
One powerful way to build the foundations of a future-proof architecture is by adopting a logical approach to data management.
At the core of a logical data management approach is the ability to efficiently consolidate diverse data sources by delivering trusted data from these sources to relevant consumers through a single logical layer. Data virtualization is a data integration technique that enables this consolidation.
Data virtualization provides a single logical access point, eliminating the need for point-to-point connections between consuming applications and information sources. Serving as a centralized data access point, it enables organizations to seamlessly enforce access security and manage data governance across the entire enterprise.
Unlike traditional approaches that physically move data to a consolidated location, logical data management offers a real-time view of consolidated data while leaving the source data in its original location. It leverages a logical data integration approach that addresses today’s challenges and forms the foundation for future data integration.
With logical data management platforms like Denodo , the semantic layer is defined within the data virtualization layer and shared across all applications and tools. This ensures consistent data across the board, reducing inconsistencies and instilling trust. Data views defined within the data virtualization layer can be reused across multiple applications, resulting in reduced development efforts and faster time-to-delivery.
Logical data management platforms not only provide visibility into corporate data assets but also enable data to become central to the business by making it easily discoverable and accessible. It enables self-service by offering a data catalog that serves as a marketplace for all key corporate data assets.
Into the Future
The COVID-19 pandemic triggered a shift in technology strategies, requiring organizations to be adaptive, creative, and resilient. Logical data management has been recognized as an effective approach to modernizing data landscapes and empowering organizations to adapt to change while fostering a data-driven culture that enables decisive insight.
(The author is Mr. Ananth Chakravarthy, RVP Sales, Denodo India, and the views expressed in this article are his own)
