Scottie Bryan Discusses Data Processes

In 2016, the company I worked for migrated to a new accounting software. We left Friday with the ability to run reports for budgeting and analytics and we walked in on Monday and realized we had lost our ability to answer even the basic questions around costs.

Even worse, our organization did not have a clear process for handling data requests. There was no team or individual that we could reach out to for help or assistance. Help Desk tickets placed our requests in a queue where they sat for years (literally) without being fulfilled. We’d meet monthly and watch our data requests get pushed further into the backlog. At one point, I was able to project that the entire team would be collecting social security before we had any insight into data.

Our company, like many companies 5-6 years ago, liked to talk about data as an asset. The reality was that data was a byproduct to be hoarded. Never to be thrown away but never made accessible.

When I stepped into the consulting world, I found that this was a reality across many organizations and industries. From landfills to gas stations to school yearbooks, straightforward access to data was at best a challenge and more than likely an impossibility.
‍

New Technologies Require New Strategies

‍In the last 5-10 years, the introduction of technologies have revolutionized how data can be handled and consumed. According to Flexera, over 50% of organizations have moved at least some portion of their data into the cloud in 2022. While this paves the way for companies to dramatically change how they interact with their data, it takes people and processes to implement change.

For a healthy organization to leverage their data as a true asset, a company needs to create clear and simple processes for data.

Any team member should inherently know 1) where to find the data they need for their job, 2) if they need more data there should be a clear process for requesting data, 3) that process needs to function in a timely manner, and 4) the organization should have right-sized security rules in place.

‍

Finding the Data

While some organizations can benefit from data governance and data management tools to assist with finding the right data, many organizations will have no issues with a low budget solution. Part of the key to finding the right data is to create a simple and straightforward enterprise data warehouse that contains a single source of truth for most of the data. This is generally done by ensuring that the single source of truth is the operating platform that manages that information. What did operations accomplish last week? That answer comes from the operations software. How much did those operations cost? That answer comes from the accounting software. How do we marry that data together? That’s a collaboration between operations, data engineers, and accounting.

‍

A Process for Data

There should be a clear and simple process for a team to request data. At a minimum, an organization needs to make sure that requests are captured in a formal system (like a Help Desk ticket), data is mapped from the correct data source to where it needs to land to be consumed, that the data is accurate, the data is given the appropriate level of security, and that quality testing is completed by the business to ensure the results in the data warehouse (and subsequent reporting/analytics) meets expectations.

Once this process is created, the organization needs to ensure that this process functions timely. Data requests shouldn’t languish for months on end unless there are critical missing components that prevent IT from offering the data up for consumption. Issues could include significant data errors like when two business units use a software program in very different ways or when multiple business units are using disparate software programs altogether. These issues aside, most data requests should take less than a month from request to test.

For organizations struggling to keep up with business data requests, there are few tricks that can take the stress from IT and empower the business.

First, consider giving technically savvy users access to a sandbox data warehouse for innovation. This works great when your data sits in the cloud. With just a few clicks of the mouse, and sandbox warehouse can be available for a team to innovate in. They can create the solutions they need, and IT just needs to wrap the solution in best practices.

Second, offload the data process onto the business. The more detailed a data request is, the easier it is to deliver a solution. I like to require the business to identify source systems and tables (when possible). I’ll even push the business to recommend a landing spot for the data.

Third, I make sure that the business provides me with all the business logic they need applied up front. If they can make the formulas work in Excel, my data engineers can code that in minutes.

Fourth, testing and signoff should always be done by the business. Its often tempting for a customer focused IT department to step through the testing on behalf of the organization, but it is critical that the requesting party reviews and approves once the project is complete.

Lastly, figure out how to make the business the bottleneck. As someone who spent most of their career in operations or finance, I know firsthand how easy it is to throw projects at IT. Creating reviews and signoffs after key steps ensures the result is correct, but it also forces the business to invest time and resources in their request. When accounting requests 10 data projects but can’t find the time to test request #1, you’ve put the bottleneck back on accounting.

By putting a few simple and low cost steps in place, any IT department can quickly set their organization up to treat data as an asset, use data to drive decisions, and set the stage for a real digital transformation.

At Novus, we have a solid history of working with companies across industries on putting processes together and helping with change management so that the broader organization understands how to leverage data as an asset. Feel free to reach out to us for a consultation or an engagement to get you started.