Why Automated ETL Isn't ETL at All

By Special Guest
Taylor Barstow, CEO & Co-Founder, Bedrock Data
April 03, 2018

As the number of SaaS applications ascends ever upward, so too has the desire to integrate the cloud data nested within them. SaaS data is rich with insights about customers. The trick is how to unify the data, so it can be fed into reports, analytics, and business intelligence (BI) tools where you can use it to draw valuable insights.

Since the 1970s, businesses have used ETL (extract, transform and load) technology as a first step towards moving data between systems. Most IT professionals consider ETL complex and time-consuming — more turtle than hare. Half a century later, ETL has evolved to the extent that it’s finally stood upright, morphed into a far more advanced species.

This “new ETL” automates the process of data ingestion, preparation, and storage. At the click of a button, anyone can have all their SaaS application data unified, and a cloud data warehouse built to feed their analytics, reports, & dashboards.

Undergirding this evolution is a true revolution in real-time analytics — historically difficult to achieve. With automated ETL, businesses can finally combine data sources and trust their accuracy. Developers are free from having to write code that extracts data from applications via APIs. And IT teams that have had to manually standardize data formats and learn how multiple data models relate to one another, no longer have to.

For decades, these roadblocks have either stalled analytics projects or postponed them indefinitely. Now businesses can analyze customer records instantly, since system relationships may now be automatically mapped, matched, modeled, and merged into a single database. That 80% of analytics projects is spent on data ingestion and prep could very soon become a stat of the past.

What does all this mean for the SaaS software & IoT spaces?
The rise in devices has also produced more data sources. Customers engage with businesses through multiple channels, often simultaneously. One day a customer might visit your website on desktop, the next on their iPad. After they file a support software, they might talk about their experience with customer service on Twitter.

With an automated ETL approach, making use of all this data — regardless of origin — is no longer a multi-day, -week, or -month project. Every activity, event, deal, and opportunity — also called “objects” — can be automatically combined within a simple UI. It’s also possible to customize relationships, or mappings, between said objects. Customer data, once seen as an analytical non sequitur, now coheres, bestowing businesses with a more holistic understanding of user behaviors and preferences.

Modern ETL also builds a data warehouse that connects easily to reports, analytics, and BI tools. For developers, it’s a relief just to have a unified schema of SaaS data. For analysts, transforming many data sets into one is just shy of a miracle, since data is in near real-time. Executives getting ready for a board meeting can report their top KPI and refresh dashboards, assured that data is up-to-date, not weeks old.

According to a survey of 502 IT professionals, traditional ETL has long undermined organizations’ ability to achieve real-time analytics. Once moved via ETL, data is at least five days old by the time it reached an analytics database. These delays also grow as data sets do. The more data there is to process, the more time IT must exert on each step, transformation especially. These are just some of the obvious reasons why so many IT professionals are on the hunt for better alternatives.

One of the chief advantages of automated ETL is speed. As the number of data sources added increases, the time it takes to unify and warehouse is relatively unaffected when compared to non-automated antecedents.

Although predictions about new technology tend to err quickly, at play are two parallel trends which reflect the same intrinsic demand — the need to do more faster, and with fewer resources. At the office, SaaS has quenched our thirst for higher productivity. At home, the IoT era has engineered a spate of gadgets which renders home and daily life more convenient.

In light of these truths, we should expect traditional ETL to fade, leaving room for a superior solution, one whose automation so radically outpaces the current one that it isn’t ETL at all.

About the author: Taylor Barstow is CEO & Co-Founder of Bedrock Data. He is a technology innovator who has a proven track record as both a developer and an engineering executive.

Edited by Ken Briodagh

Related Articles

Synching Up A More Secure IoT, Syniverse Launches Single Focus Global Network

By: Cynthia S. Artin    6/22/2018

As global as the Internet is, and as resilient as the Internet was designed to be, Syniverse, a 30-year old mobile network services company that alrea…

Read More

AvePoint Citizen Services Launches in Richmond, Virginia

By: Ken Briodagh    6/21/2018

Six government agencies in Richmond, Virginia will hook into the AvePoint Citizen Services' RVA 311, centralizing requests and resolution tracking for…

Read More

Cybertrust Joins the CIP Project As Ecosystem for Large Scale Critical Infrastructure Initiatives Grows

By: Cynthia S. Artin    6/20/2018

When it comes to critical infrastructure, including development of the smart grid, smart cities, transportation systems, and more, security is no long…

Read More

Study Says: Mauritius, Ghana and Tunisia Lead African Telecoms

By: Ken Briodagh    6/20/2018

BuddeComm has launched a Telecoms Maturity Index (TMI), to analyze the broadband, mobile and fixed Line markets of a country, rank it and compare it t…

Read More

A Guide to Creating New Citizen Experience in IoT-Based Smart Cities

By: Special Guest    6/19/2018

IoT-based smart cities are creating value for citizens. In this article, we explore three components of a citizen-centric smart city and reveal how to…

Read More