How creative and unconventional audit analytics can help you take your audits to the next level

"We just need to come up with some data analytics tests to find out about ... that thing they mentioned in the board meeting today..."

That "thing" is usually the latest hot topic and could range among a variety of sensitive matters, from weeding out travel abuse to finding out if executives share passwords, whether contractors are overbilling the company, or even if anti-money laundering measures are effective.

Whatever the problem is, you realize right away that the request is anything but simple and that your weekend plans will have to be put on hold. When issues are discussed in the boardroom, it's usually because something complex and unpleasant has happened already. You guess that your manager isn't telling you the whole story.

Your manager probably also omitted a few other important details, such as the fact that: it is a hugely political matter, the problem has never occurred before, five people have already tried and failed to solve this before you, nobody really knows what systems the data related to the problem resides in, and the units involved are famously difficult to work with.

Your boss walks away with a casual, "have a nice weekend!" Suddenly, this "thing" that you didn't even know about five minutes ago has become your problem. You find yourself with a big challenge ahead of you. But the problem could also become a huge opportunity if handled correctly.

Your first thoughts are probably not too enthusiastic, however, as you hear the same questions repeatedly echoing in your head: Why? What? How?

We all know this feeling of helplessness. This is when you have to start collecting yourself together and analyze the situation. Ironically, this apparent call of despair is actually the right way to start tackling a problem like this. Let's break it down, one question at a time:

Why? Why are we doing this test? Answering this question will give you clear indications as to what senior management is really after.

What? What are you going to do? Well, you still want to be employed on Monday, so you need to come up with a practical, pragmatic, and defensible approach.
You will never find any checklist or audit work-program on this topic. The normal ways of data analysis are not going to work either, so what are you going to do?

How? How can you crack that problem? You will have to innovate. You will have to do something that has likely never been done before. Basically, you will have to find creative ways to use analytics to get your audit to the next level.

Leveling Up
Let's be candid, audit analytics is a hot topic right now, has been for several years, and it's only gaining in popularity and acceptance. These days, the conventional use of analytics tends to receive a high level of management support. It relies on a proactive approach, which requires proper planning, and is used to successfully achieve your test objectives.

In contrast, a more outside-the-box, innovative, or unconventional approach is much harder to sell to management. It is often reactive, unplanned (relying heavily on improvisation), and regularly offers unexpected insights that were not part of the original objective, hence generating an increased risk of "scope creep."

So why bother? Why take the risk of going for added complexity, higher failure rates, longer turn-around time, and uncertain results that take you out of your comfort zone, while fighting against your hierarchy to sell your ideas?

Well, because the amazing opportunity offered by getting that unique, unexpected insight will really make all the difference. If you manage to control the scope creep, unconventional analytics will give you the rare ability to not only find difficult answers to problems that nobody could crack, but more importantly, the ability to ask new questions.

I also firmly believe that when you get outside of your comfort zone, you push yourself to the next level. Pushing yourself will also push your audits to the next level, and you will contribute to getting your organization to the next level.

Change Your Perspective
A famous quote often attributed to Albert Einstein states: "The definition of insanity is doing the same thing over and over again and expecting different results." Since calling people insane does not usually yield the most constructive results (trust me, I tried!), I personally prefer that other, slightly less-famous quote of his: "No problem can be solved by the same consciousness that created it. We need to see the world anew."

The idea is the same, though: if you keep doing the same tests and using the same approach, you will inevitably get a similar outcome. And when your manager invades your office on a Friday night with that surprise "simple request," you know that this will not suffice. You know that you will have to try something new, different, and take it to that next level. If you use a technique, process, or system that nobody ever used before, chances are that you will be the one uncovering things for the first time.

When reading about audit analytics, you will find a lot of literature focusing on the proverbial "low-hanging fruit." Don't get me wrong; you should absolutely work on these quick wins when you can. Senior management loves nothing more than the sweet taste of fresh-squeezed, low-hanging fruit, and many consultants will be happy to help you find those yummy treats.

But that approach will only get you so far and won't solve every problem, especially not complex ones. Those require more creativity and potentially more work, what I would call "sniffing out the truffles." It is about rolling up your sleeves, plunging your hands in the dirt, and digging, in the hopes of pulling out something that is a lot more valuable.

From 'Standard' to 'Unconventional' Analytics
While there is no strict definition as to what "Unconventional Analytics" means (and the definition would probably differ from one person to another), there are some patterns of behavior that typically increase the chances that the tests you do will yield new findings or lead to new insights.

I would categorize "Unconventional Audit Analytics" tests into three main types (or any combination of the three):
A. Using standard functions on unusual fields,
B. Reconciling datasets that never "talk" to one another
C. Using new and unusual data sources

Here's an example: nobody ever bothers matching a transaction log date with a Windows user logon. Why would they? Well, "why not?" should always be the first question in your mind. Remember, this is all about going where no auditor has gone before.

The actual transaction vs. logon example mentioned above once showed evidence of trades that were performed while the trader was not even logged onto the network. It turned out to be an indicator of password theft to conduct fraudulent transactions. No one had found this before because no one had ever thought about conducting such a test.

Some of my most successful results happened when I tried to do some tests that did not seem to make sense at the time and would make my colleagues ask, "Why?" Although it was difficult to justify at the time, I would just sit back quietly and think, "why not?" I happen to have the data available, I have the tools, and it would just take a few minutes (maybe an hour max).

Chances are that when you use this method, you could be the first to ever look at a certain dataset, or become the first to connect two previously unconnected universes. By doing so, you will naturally increase your odds of finding something that no one ever noticed before.

Esoteric and Unstructured Datasets
Now these are mostly standard data points. So what other slightly more esoteric datasets exist in your company? As you can imagine, the amount of data, structured or unstructured, that is available nowadays is almost overwhelming. Usually, only a very small subset of it ever makes it into an audit test.
Have you ever looked at ID badge access logs? Phone logs? Remote access logs?

In fact, if you expand your scope to include unstructured data and add the latest techniques and newest technologies that are progressively becoming available to analysts, such as text-mining, geo-tagging, or facial and voice recognition, it is truly impressive to see what can be done in a company nowadays. There is a goldmine of data available in every organization that never gets used by auditors, about 80 percent of which is unstructured. If you are the first one to explore that data, you will likely see things that nobody did before you.

The Next Level
Going outside of your comfort zone is never easy, especially when you have to deal with budget constraints, but hopefully you will find inspiration to rise to the challenge. If you feel ready to take it on, keep one thing in mind: "Start small, then go big!"

Here are some other points to remember:
• Don't try to boil the ocean
• Start with the basic idea, then add to it gradually
• Break down big problems into smaller parts and address them separately
• When you start getting some traction, expand your tests by asking the three basic questions:
        1. Why not try something new?
        2. What other data exists and could be used?
        3. What else should I or could I know, and is there a way to do get there?

Just remember that simply by trying new ways of testing, asking the right questions, and getting outside of your comfort zone, you can really take your audits to the next level.


Yves Froude, CIA, CISA is Data Analytics Officer at The World Bank, Washington DC. The views expressed here are his own and are not intended to reflect those of any particular organization.