We have to discuss metrics! Why? As a result of they’re probably the most essential issues within the information analytics world. Truly, with out metrics information analytics wouldn’t be very helpful in any respect. Metrics give everybody context, assist corporations develop, and handle points. All of us want them to make higher selections. Then again, regardless of the massive advantages, it isn’t simple to work with them. You need to retailer the definition, and it may be SQL, calculation in a spreadsheet, or (for instance) Python code. Metrics may be advanced and their definition (for instance, % Income from Prime 10% Clients) isn’t a straightforward process. These are the 2 greatest points — consistency and complexity. Right here come instruments and approaches like dbt metrics that declare:

by shifting metric definitions out of the BI layer and into the modeling layer, information groups can really feel assured that completely different enterprise items are working from the identical metric definitions, no matter their instrument of selection.

The query is: Is that this actually the fitting strategy?

Metrics in dbt

The rationale for introducing metrics in dbt is clear — consistency. Each firm must have constant numbers about issues like income, churn, clients, and so forth. The dbt strategy is to maneuver metrics definition from BI instruments to the so-called modeling layer.

What does it imply? You might have heard the time period ELT earlier than — extract, load, rework. Because of cloud information warehouses ELT began to change into increasingly common than the previous ETL — extract, rework, load. What’s the distinction?

Properly, earlier than the rise of cloud information warehouses, it was not simple to retailer all of your information in a database. Earlier than you could possibly retailer your information (i.e. loading your information into the database) you wanted to do some transformation in a separate course of and solely after the information was reworked, you could possibly then load it right into a database. These days, the ELT strategy permits us to run information transformation inside an information warehouse, which opens up the potential of instruments like dbt and that introduces the so-called modeling layer. You mannequin your information to the form your purchasers (BI instruments, Notebooks, and so forth.) want. Shifting metric definition from purchasers to the modeling layer makes good sense, proper? Properly, sure, however probably not…

Metrics within the modeling layer make good sense for easy use circumstances, like aggregating the overall variety of clients. Then again, if you wish to calculate extra advanced metrics and take a look at them from completely different factors of view, you want to have the ability to see metrics in several contexts. From the dbt documentation:

A metric is a timeseries aggregation over a desk that helps zero or extra dimensions.

You want a extra subtle instrument that means that you can work with a number of tables and that means that you can create a metric over a desk the place you should not have a date attribute.

Metrics on Steroids — GoodData Metrics (MAQL)

Let me ask you a couple of questions — have stakeholders ever wished to know firm income? Have stakeholders ever wished to know firm income in area A? Have stakeholders ever wished to know firm income in area A from product B? What do these questions have in widespread? Asking the identical query in a unique context. The ultimate output of a metric is a quantity, however the quantity modifications relying on the context during which it’s used. GoodData, because of its proprietary language MAQL (Multidimensional Analytical Question Language), brings a solution to outline metrics which might be context-aware. GoodData enables you to outline one metric that may be reused in a number of contexts (considered by areas, merchandise, and so forth.). In different phrases, GoodData means that you can outline metrics which might be advanced and solves complexity.

Headless BI — The Solely Supply of Reality

The decision for a single supply of reality is clear. It’s the cause why dbt got here up with the metric layer. The identical cause applies to headless BI as the one supply of reality and which means an ideal match between dbt and GoodData! You possibly can outline easy metrics in dbt (throughout transformation/modeling) and derive them in headless BI (GoodData). Moreover, you possibly can outline extra advanced and context-aware metrics in headless BI.

You’ll obtain consistency and resolve complexity. On high of that you could devour every part via API and SDKs in BI instruments, Internet functions, Notebooks, and so forth. Final however not least, headless BI will present you caching and authorization. Sounds good, proper?

Demonstration — GoodData vs dbt

As an indication let’s present how the metric % Income from Prime 10% Clients will look in GoodData versus the way it will look in dbt.

Metrics in dbt

Let’s begin with a easy definition of income:

It may be outlined in a schema.yml file in a declarative approach. It’s fairly easy, you simply must outline time grains and dimensions, setting the maximal scope of reusability.

Let’s strive a extra advanced instance and calculate the aforementioned % Income from Prime 10% Clients:

For extra advanced metrics in dbt it’s important to write a big and complex SQL assertion. You should utilize metrics.calculate() operate to make the most of the easy metrics outlined within the schema.yml file.

Metrics in GoodData (MAQL)

Let’s begin with a easy definition of income:

To calculate the metric % Income from Prime 10%, let’s create one assist metric Income Prime 10:

The ultimate definition of % Income from Prime 10%:

You possibly can examine each examples. As mentioned, dbt metrics are fully tremendous for easy use circumstances however for extra advanced use circumstances the prudent selection is to make use of a extra subtle instrument like GoodData. Moreover, all GoodData metrics are outlined with out context — which means that if you wish to use a metric with completely different attributes (for instance, sliced by area) you are able to do so simply. If you need to slice a dbt metric with completely different attributes, you would want to jot down one other lengthy SQL file. In my trustworthy opinion, GoodData is simply higher for superior use circumstances and may prevent a variety of time.

Closing Phrases

For individuals who need to actually dive deep, we’ve created a demo with dbt and GoodData. Additionally, for a clean expertise, I invite you to strive the GoodData trial the place you possibly can attempt to outline all metrics both on demo information or by yourself information.

Thanks for sticking with me via this text and don’t forget to observe us for extra demos and articles! See you!