«We’re going to cease fixing knowledge within the BI layer beginning right now!»
I’ve heard some model of this chorus numerous occasions in my profession, and the final decade introduced a ton of improvements that at occasions made it appear attainable. Improvements together with the vast adoption of cloud computing, cloud warehouse and lakehouse architectures, and the shift from ETL to ELT. At each step of the method, well-meaning knowledge groups swore off the observe of fixing knowledge issues in BI because of the apparent points it creates: Metrics locked in BI, inconsistent and low-quality knowledge, and a excessive quantity of rework.
Did anybody succeed although?
Virtually universally, no. Enterprise actuality adjustments too quick for even probably the most adept knowledge crew. Information fashions begin shedding relevance the second they’re deployed. Finish customers run sooner than your governance program can sustain with. BI-layer knowledge transformation, modeling, and calculation is an inevitable and inescapable a part of knowledge, no matter our makes an attempt to forestall it.
The First Step
We took on this problem at GoodData by introducing one of many trade’s first and strongest open enterprise intelligence semantic layers. Not are these essential ‘final mile’ transformations or calculations locked right into a proprietary entrance finish. Due to our SDKs, APIs, and SQL entry level, any system can profit from the superb work of your knowledge analysts and BI engineers. It was an essential first step, however our prospects nonetheless struggled with complicated knowledge construction, pace, and price points that semantics alone merely couldn’t remedy.
The Subsequent Stage of our Imaginative and prescient
It’s for that cause, we’re excited to announce FlexQuery, the subsequent stage of our imaginative and prescient to offer highly effective, open instruments to knowledge builders fixing ‘final mile’ knowledge issues that merely can’t be addressed in knowledge warehouses. FlexQuery is a composable knowledge service layer constructed on widespread open-source applied sciences like Apache Arrow, Apache Iceberg, and DuckDB. We’ve pre-built probably the most helpful enterprise intelligence companies into it, corresponding to caching, knowledge federation, transformation, and computation. Nonetheless, consistent with our dedication to developer-first approaches, FlexQuery is open so that you can construct modules that meet your particular wants.
So what are you able to truly do with FlexQuery?
Lower your expenses with superior caching
FlexQuery mechanically caches full and partial outcome units for re-use, saving you vital cash in your underlying cloud knowledge structure. Based mostly on Apache Arrow, the ‘FlexCache’ helps super-fast in-memory computation on massive volumes of information. Shifting ahead, we plan to offer an increasing number of granular cache controls that can assist you stability knowledge recency vs. price financial savings.
Remodel and federate knowledge with agility
FlexQuery’s built-in transformation and federation capabilities make it simple to mix knowledge from your whole functions by way of SQL or APIs. You may materialize knowledge in reminiscence or on disk question it reside out of your databases, and even combine the 2 modes collectively. It’s simple to do knowledge engineering instantly inside FlexQuery to create the complicated pipelines essential to prep knowledge for consumption.
Construct with the very best open-source instruments
FlexQuery is constructed on the very best open-source analytics instruments, together with Apache Arrow, Iceberg, and DuckDB, pre-configured for core BI use circumstances. Nonetheless, at GoodData we all the time provide the keys to the dominion: It’s simple to develop your personal FlexQuery modules to handle knowledge engineering or knowledge science wants at low price instantly in your BI platform.
Welcome to the Analytics Lake
FlexQuery is now on the computational coronary heart of what we name the ‘Analytics Lake.’ Not like a knowledge lake, the Analytics Lake incorporates knowledge, metadata, knowledge fashions, metrics, and studies/visualizations in a single platform, making it simple to create and govern any sort of analytics software with out complicated software program integration challenges. Furthermore, due to our semantic layer, every person accesses the Analytics Lake via their favourite interface — Python for knowledge scientists, React for software builders, SQL for knowledge engineers and, no code/low code and chat interfaces for enterprise customers.
FlexQuery and the Analytics Lake idea characterize the subsequent step ahead for enterprise intelligence platforms, transferring past easy point-and-click desktop interfaces to embrace the complete complexity of information improvement within the fashionable period. With a mix of intuitive person interfaces, highly effective analytics-as-code instruments, and now the FlexQuery analytics engine, GoodData affords organizations massive and small the world’s strongest, versatile, and environment friendly enterprise intelligence platform.
Prepared To Be taught Extra?
To be taught extra about FlexQuery and the Analytics Lake head right here, or to discover it hands-on together with the remainder of the GoodData analytics platform, request a demo.
