Reinvention and Redefinition
Most areas of enterprise software are reinvented and redefined every 7-10 years. And in most cases when reinvention has occurred, it was an option; not an imperative. For instance, when server virtualization was invented by VMware it was an option, as otherwise we would have continued with existing physical-only server scaling. When ServiceNow redefined ITSM, it was an option, not an imperative. When Box redefined secure file storage in the cloud for enterprises, it was an option. In the area of Business Intelligence and data analytics however, we’ve now lived through decades of little reinvention outside of prettier visualizations. But in the last 2-3 years it is suddenly undergoing rapid transformation and it’s imperative and not an option to not have it occur. Why is that?
We live today in a unique data landscape context. We also live in a unique time of business competition. Both are in a state we’ve never seen or experienced before. Add to that fundamental technology advances that have occurred in cloud, user interfaces, and data processing, and together we are now seeing a kind of runaway acceleration in BI innovation that surpasses previous decades. As a result of the data environment, business environment, and core technology acceleration, Business Intelligence is being completely transformed. The transformation applies to “everything”; from the way disparate data is accessed, how it’s prepped, how it’s blended, how fast it happens, how easy it is, and how it’s freely explored and analyzed.
The Modernization of BI
The “modernization of BI and analytics” is experiencing its’ fastest pace of change ever. Today, “BI change” has become an integral part of every organization’s quest for a new way for business users to be self-reliant in accessing, discovering, and exploring more data and more insights. We are now reaching a market inflection point where rapid BI transformation is happening. Organizations of all kinds in quick-moving, competitive sectors such as CPG, national retail, healthcare and logistics/manufacturing. urgently need near real-time, business-ready insights. Further, two-thirds (64%) of companies are already trying to combine five to 15 sources of data.(1) And shockingly, 92 percent of business users are still using Excel for analysis for lack of better alternatives.1
Through 2017, Gartner predicts that the number of business users doing data analysis will grow five times faster than the number of highly skilled data scientists, (2) and most business users and analysts will access self-service tools to prepare data for analysis.(3)
Spark-speed, Fast Insights, Business-explorable
Paired with the urgent need to access more disparate data and accelerate business insights, the attractive economics and speed advantages of cloud-hosted solutions are quickening the industry’s shift to modern solutions. To address the need for scalable and fast data processing, Apache Spark has also now spiraled into the forefront as a catalyst and industry change agent. Early on, we, at ClearStory, saw the great potential in Spark when it was still a project at U.C. Berkeley’s AMPLab and built a Spark-native analytics solutions, that’s business-ready and cloud ready, including being cloud agnostic.
Today at O’Reilly Strata + Hadoop World in New York City, we unveil advances in our Spark-native, business-ready analysis solution to further speed disparate data analysis and dramatically accelerate data discovery and insights, to serve the unique data landscape and business environment organizations now face.
ClearStory Advances and What’s New
Our new advances brings more disparate data access, smarter system-driven data matching, and faster and more visual data harmonization and an even simpler user model so anyone can access, blend and analyze data. It also adds Google-like visual maps that let business users see the specific route that resulted in the holistic insights. ClearStory’s end-goal hasn’t changed; which is to make multi-source data analysis to be fast and simple, so business users can answer new questions quickly, across more data, and act fast. Try it here.
Highlights of our new capabilities:
- Turnkey access to a broad range of data sources and types and visual guidance for ad-hoc data blending and harmonization: Users can easily navigate a streamlined visual route to make changes on-the-fly and enrich insights with more data. Each step generates a dynamically updating “Data Story” that reveals insights that are explorable and flexible. The types of data that businesses need to combine and harmonize today include Excel files, relational data from existing data repositories, semi-structured data and data from cloud-based applications.
- Enhanced data intelligence and smart data matching with visual cues on data blending and best matches across complex data: The new ClearStory interface provides an interactive experience that informs the user about the auto-detected matching attributes of the datasets and relationship between attributes across data sources. The system additionally detects and alerts on data dimension inconsistencies and guides users to the right and best way to harmonize disparate data. In situations where disparate data is being combined that doesn’t fit together is recognized, users are alerted via visual cues and can review and resolve any ambiguity and inconsistencies while blending data in a few clicks.
- New Data Lineage Visualizer™ for deep visibility into data lineage to ensure data integrity and insight integrity: Business users’ preferred way of manipulating data is through a visual, self-service experience so they can answer their questions faster, versus being provided rigid IT-created dashboards that constrain their insights. ClearStory’s advanced interface provides users with a visual map and data-blending blueprint in one intuitive, unified view. Users can click on any dataset in the map to ascertain its underlying granular details – to see how the data was blended together, and to trace the data harmonization path that has been applied. In a single view, users can easily answer the questions and iterate fast, to answer new questions without relying on data specialists to do it for them. We encourage you to read more here.
More to Come, with Empathy for Business Users and Organizations Challenges
With today’s announcement and ClearStory’s advances in its Spark-based ‘smart data matching’ and the richness and simplicity of our user model, business users can be self-sufficient in generating fast, business-ready insights across more data. Our integrated, Spark-native architecture and visual, guided application interface speeds up every step from data access, prep, modeling and data harmonization.
At ClearStory, we continue to accelerate BI modernization on every front, from our underlying Spark-native capabilities, through to the user application model so tasks that were once overwhelming, complex and slow in traditional BI platforms are now fast, simple and accessible to everyone.
1 Harvard Business Review, Analytics Services Report: “Data Blending: A Powerful Method for Faster, Easier Decisions” – August 31, 2015
2 Gartner, Inc. Report: “Smart Data Discovery Will Enable a New Class of Citizen Data Scientist” – June 29, 2015
3 Gartner, Inc. Report: “The Rise of Data Discovery Has Set the Stage for a Major Strategic Shift in the BI and Analytics Platform Market” – June 15, 2015