Conversation: Machine Learning, Big Squid’s Kraken, Qlik and Solve

Solve and Big Squid recently presented a webinar – “Automating Your Machine Learning and AI Initiatives (How to Get Started)” – that focused on how organizations can augment their existing analytics capabilities using Kraken, Big Squid’s machine learning platform.

Featuring Solve’s Greg Woodard (GW), Big Squid Director of Strategy & Alliances Chris Wintermeyer (CW), and Big Squid Director of Data Sciences Jorge Zuloaga (JZ), the discussion offered a valuable discussion on how combine machine learning and Qlik technologies to accelerate insights and deliver accurate predictive analytics. Here are the highlights of this online conversation:


What is the data analytics journey and where does machine learning fit?

GW: Gartner and Big Squid describe the data analytics journey with four levels. Most of our customers, when they come to us, are focused on the first level: Descriptive. They have a lot of reporting. They’re trying to figure out how to analyze the data they’ve captured over time. And the Qlik products we use add a lot of value there.

But once customers get beyond that initial stage, and can look backward really well, they start thinking about how they can use these tools to look forward. So they start looking at Diagnostic capabilities, the second level, and build out dashboards that help them determine why something happened.

Then they start thinking they need to look at key metrics that will help drive their business. Usually they know many of their key metrics, but they don’t know which ones actually impact their business. And that’s where machine learning comes into play. With machine learning, they can take a very large data set, ask it a specific question, see how their data impacts that test, and uncover critical KPIs they didn’t know about.

Ultimately, they can use that information to look forward by adding Predictive capabilities – and eventually move to the Prescriptive level where they are able to influence outcomes.

JZ: Simply put, machine learning – ML – is the science of programming computers so they can learn from data. Today, organizations with business intelligence are consuming their data sets and using ML algorithms to find patterns and making business predictions based on those patterns. Many practical business questions can be answered with ML.


What are the steps to ML success?

JZ: There are five steps that determine whether machine learning delivers value:

Step 1: Define the business question precisely. You must be able to clearly state the use case – customer churn, lead scoring, customer lifetime value, etc. – and the precise definitions – timeframe of lifetime value, who is a customer, etc. – related to the use case. Ideally, this should be done with just a few sentences.

Step 2: Prepare your dataset properly. The data must be relevant for the questions you’re trying to answer. Not all BI data is organized properly for ML. When you pull data into Kraken, it must be denormalized, and organized into a single view, a single table or a single file so it precisely relevant for your precise question. (Solve can help you determine if you have the right data, and if it is prepared the right way for ML, and, if not, build the structures and schemas to organize it properly.)

Step 3: Determine how will the model and predictions be used. What actions will you take based on the predictions from the machine learning model? Will you take the predictions and feed them into an automated process?

Step 4: Know the benchmark you need to beat. How well does the ML model need to perform to exceed results of random chance or existing model/system?

Step 5: Quantify the ROI. This is often difficult to do. It will depend on what you’re trying to know and the actions you take. Did you use the insight from ML to reduce costs, realize savings, discover efficiencies, find new customers? Sometimes it is impossible to calculate.


Where should most organizations start with ML?

GW: At Solve, the first step is to do an ML audit. Are you tracking the right data in your core systems?
Do you need help determining the business question and use case? Is your data available and properly organized?

And then, if your organization is ready, we do a Kraken pilot. We populate and test an ML model to determine if there is a correlation between your data and what you’re trying to predict.

It’s important to know that if the audit shows your organization is not ready for ML now, the insight it provides will make it easier to make the move in the future.


How does Big Squid’s Kraken fit with Qlik?

CW: Kraken is really good at providing answers for important and difficult business questions, like predicting customer lifetime value, forecasting sales and inventory, optimizing marketing mix, predicting patient re-admission rates and more.

Many organizations have great reporting and dashboards, but business intelligence falls short when it comes to predicting what’s next. When we introduce Kraken into the process, we can look through the windshield, and not the rearview mirror.

Kraken delivers substantial value in your existing environment. You can bring data directly from your Qlik applications, and it also allows you to take the predictive information and push it directly back into your Qlik environment.

In short, Kraken maximizes your data and analytics investments, operationalizes your data, provides prescriptive actions, and enhances your analytics.


Do you need a data science background to work with Kraken?

CW: We built Kraken for BI people – people with analytics experience, people who are great at pulling in and organizing data, building reports and pushing dashboards out, people who probably do not have advanced data science skills and the ability to leverage tools like R and Python.

But if you are a shop that has in-house data science expertise, we have your back too. Kraken allows you to empower your analysts to take advantage of machine learning.

Think of Kraken as giving you the ability to augment what you are doing in BI. Kraken is focused on analysts with data skills and business acumen – and enabling them to focus on solving business problems, and leveraging the analytics skills they already have, not data science.


View the webinar here. And see a previous webinar that includes a detailed Kraken demo here.


Greg Woodard

Principal at Solve