Preliminary COE Reading
We’ve already established the importance of COEs, along with how starting with a data and analytics Center of Excellence (COE) helps CMOs in the boardroom. So, assuming you took JT’s advice and created allies in your COE creation (or even if you didn’t), what are the next steps? This is your step-by-step guide to building a data and analytics COE.
Before we start, let’s first focus on where reporting and analytics should live in your organization.
Data and Analytics COE Organizational Design
You’re a smart marketer. The rest of your marketing team is smart too. Because you’re all so smart, each individual is perfectly capable of doing your own reporting on your individual job responsibilities, right? Sure. But, just because you can compile your own reporting doesn’t necessarily mean you should. Why?
Why You Shouldn’t Align the Responsibility to Individual Contributors
1. Positivity Bias
Who enjoys sharing poor performance, especially among competitive peers and people who can fire them?
Right. There’s an inherent and understandable conflict of interest placing sole responsibility for reporting in the hands of those who own the initiative. Of course, it’s important they understand and engage in the creation of the reports.
2. Analytics is a Skill
The ability to analyze reports is a skill set very different than even the ability to create reports. It’s also a skill set in high demand. If you give the exact same report to three people, there’s a high probability you’ll get three different interpretations of what the data means. And the real takeaway(s) will be somewhere in between those three interpretations.
Beyond consistency in cadence, process and look and feel, the real consistency issue lies in the focus toward organizational goals. As JT spoke about in his article, about CMOs in the boardroom, if the reporting the marketing team rolls up to the CMO doesn’t have an organizational focus on metrics that matter, CMOs will continue to struggle maintaining the respect they deserve in the boardroom. In addition, you want to ensure reporting remains consistent even as team members come and go from the organization. Unfortunately, this is a huge concern in marketing departments, which have the highest turnover of any other function.
Creating Marketing Analyst Roles
Instead of the above, think about creating a role in your marketing department dedicated to reporting. Sometimes this role falls within the marketing operations team. Sometimes it falls completely outside of that. Either way, the person, or people, responsible for data and analytics needs to be disconnected from campaign strategy and execution. This enables them to have an unbiased view of campaign measurement.
In addition to that, this enables this small team to create and maintain consistent cadences, processes and methodologies for collecting and analyzing data. This small team will be able to look at one data set and come to the same conclusion(s) about what the story the data tells, because they’ve standardized a process for evaluating that data. They will be intimately familiar with the sources of data and how data is collected, which plays an important role in data evaluation.
In addition to that, they can catalog key events to pinpoint misleading data. For example, one of my clients had a large (5x their normal rate) influx of Contact Us form fills that were essentially spam in a very short period of time. This was over a year ago and it still impacts how we evaluate their current data. Having that historical context when we evaluate their current data can be the difference between thinking we’re down year-over-year and realizing that we’re actually up year-over-year when adjusting for those spam records.
Lastly, having this small team will also give you a holistic view of all data, which helps you understand if there are any gaps or inconsistencies. If, for example, someone on your team starts using UTM parameters inconsistent to your standard practice, this team can identify that immediately and correct the behavior. They act as the guardians of data and protect your data’s integrity.
What is a Data and Analytics COE?
A data and analytics COE is not a dashboard. It’s also not a report or even a series of reports. Instead, it’s a series of answers to pertinent questions that we ask on a daily, weekly, monthly, quarterly and annual basis.
Your data and analytics COE also doesn’t answer every question you can possibly ask of data. It can’t predict those questions. But it does help identify those second and third level questions that you’re going to ask.
Building Your Data and Analytics COE
Where to start might seem like a daunting task, especially if you’ve recently invested a good deal of time convincing your organization of the value of COEs. However, you can leverage our standard process, which will take you a good portion of the way through implementation.
One caveat: This process is system-dependent. You’ll want to follow it for each system you’re building and analyzing reports from. If you have a mature reporting tool, such as Tableau or Domo, you can probably build your entire COE in one tool. If not, you’ll need to spread your COE across Salesforce, Marketo, and any other systems in play.
Let’s get started…
Step 1: List Top-Level Questions
Start by creating a list of the top-level questions you want to answer with reports. Remember, this isn’t a list of every question you’re ever going to want to answer. But it should be the most consistent and most important questions. You know, the kinds of questions which drive real business impact. Example of these questions are things like…
- “Which of my marketing channels created leads last quarter, and how many?”
- “How much pipeline did my Paid Search efforts drive last month, and through which campaigns?”
- “How much revenue did webinars drive in 2018?”
These are general, top-level questions that we’re going to ask on a consistent basis. These are the types of questions that should be answered by our COE.
Step 2: Evaluation of Existing Data and Systems
After you outline your goals, it’s time to assess how you can achieve reports that answer these questions today. Highlight the questions you can answer now, and through which system. Highlight those that you can’t answer now also. We’re going to create a roadmap to get these questions answered eventually.
Once you know which questions you can answer and through which system, you’ll want to organize them into the system(s) you’re going to use to answer them. If you find multiple systems can answer the question, identify which one does it best. For example, answering questions about funnel stage conversions is possible in both Marketo and Salesforce. However, Marketo does this much better than Salesforce does. So, we’ll want to use Marketo for that purpose.
Step 3: Plan
Assuming you’re at an organization where systems ownership is spread between multiple people in various departments, you won’t be able to build everything on your own. This will require you to involve various other people in your plan. If you have a PMO, getting in a sprint or aligning resources should be fairly easy. If you don’t, you may want to check out how to get started with a PMO here.
Step 4: Build
This is the tedious part, but very straight-forward… Build the reports that answer the questions. But luckily, you have a plan.
There’s one key part of this that most people don’t do. When you save your reports, don’t just give them a regular name. Explicitly list the question(s) that each report answers. This is so important. It not only helps you check them off your build list, but it also helps the folks that will use the reports.
In addition to the key above, there are two big pitfalls here.
Reporting Pitfall 1: Too many questions
Don’t try to answer too many questions in one report. Keep it simple. If you try to answer too many questions in one report, your report will become very confusing and difficult to interpret.
Reporting Pitfall 2: Reports for reporting sake
Do not build reports that don’t answer specific questions. If your question starts with, “It would be nice to know…”, it’s not a valid question. If your report doesn’t tell a story, or answer a question, it’s likely too broad. You know the type of report I’m talking about. They’re the reports that you look at and go, “I have no idea what I’m supposed to take away from this report.”
Step 5: Quality Assurance
Having a data and analytics COE doesn’t matter if the data is inaccurate. As a matter of fact, it’s worse than not having one at all. So, make sure you QA your reports.
This process is going to be completely different for each report you build, so I can’t really give you too many tips here… other than just to do your due diligence here. It’s well worth it. And, of course, if you have an analyst, per my previous suggestion, they will be well-positioned to troubleshoot and assess any discrepancies.
Step 6: Dashboards
Once you have a suite of reports built, it’s time to combine those into dashboards. Just like reports, dashboards need to have a specific purpose and answer specific questions. If they’re too broad, you won’t know what to take away from the dashboard.
I like to group key topics and areas together. So, I’ll create a dashboard for Paid Search that consists of a suite of reports all answering the various questions I might ask about Paid Search. I’ll create another dashboard for Social Media that consists of a suite of reports focused on answering questions about Social Media. This keeps my dashboards focused and prevents me from trying to do too much with one dashboard.
Step 7: Add Context
Reports are great and help us answer a whole host of questions about how we’re performing. But, without context, reports can lead us to conclusions that aren’t exactly… accurate. Let’s say you have a report that tells you that Paid Search influenced the creation of 500 leads in Q1. Is that good? Sounds like it could be. But what if I told you that in Q4-18 Paid Search influenced the creation of 600 leads. Is that 500 still good? Not anymore. And what if I told you that in Q1-18 Paid Search influenced the creation of 300 leads. Is that 500 good? Now, it looks good again. Context changed all of that.
As we progress, we need to create versions of our reports and dashboards that add that context. Context means both quarter-over-quarter and year-over-year. Year-over-year is super important because it helps filter out any potential seasonality that may exist in your business… whether that seasonality is known or unknown.
Step 8: Create a Template
Reports and dashboards provide you with data. Data is useless if it isn’t analyzed. A template provides us with guardrails that show us exactly how we’re going to present not only the data, but the key takeaways from that data.
When you enter the boardroom to present to your peers in the C-suite, you’re not going to show them raw data. You’re going to show them a series of conclusions, that are supported by data. So, create the PPT template that has placeholders for your conclusions or takeaways, along with placeholders for the raw data that supports these conclusions.
This template is then used repeatedly on a monthly/quarterly/annual basis… whatever your reporting cadence is.
Step 9: Be Prepared to Say, “I don’t have that answer… but I can get it.”
I know, this sounds horrible. But, a data and analytics COE can’t possibly be created that is going to answer all questions. That’s why the COE is designed to rule “most of them”.
See, we can predict what the top-level questions are going to be each week/month/quarter/year. But we can’t predict the questions that are going to be generated by the answers to those top-level questions. These questions are things like, “Why did Paid Search influence fewer opportunities this year over last year?”, or, “Why did our marketing stop influencing pipeline for Business Unit X in Q1?” Those “why” questions are the second and third level questions that our COE can’t predict. And quite frankly, those “why” questions can take three, four, or 15 different reports to answer. So, leave those out. We’ll create them when we need them.
The Final Product
The final product of this work is that you now have a standard suite of reports and dashboards in each system that represent the consistent reports that you are going to use on a periodic basis to answer the top-level questions in your organization. You also have a PPT template that you use to present to your C-suite peers.
These things, all combined together, add up to a data and analytics COE. It’s a repeatable process that can be followed as needed, on demand, and handed off to anyone in the organization to execute against… just like Justin and JT mentioned in their articles.
You also now have the keys to a high-performance data engine that will help make sure your marketing department is truly a “data-driven” marketing team. It won’t just be a catchphrase.
Meet Drew Smith
Drew brings more than a decade of marketing experience to LeadMD. He has led strategy in many different disciplines of marketing such as event marketing, product marketing, sponsorship, email, digital and website, marketing analytics and database management. With experience in both B2B and B2C environments, Drew is a well-rounded marketer with a tool for most applications. Drew thrives when he’s using his analytical skills to troubleshoot complex problems or identify areas of improvement. His favorite business quote comes from the movie “Tommy Boy”: “If you aren’t growing, you’re dying. There’s no third direction." Drew joined the LeadMD team in September 2015. When he isn’t working, you can find him captaining his kickball team, playing flag football, participating in obstacle course races and fun runs, or hiking the mountains of Phoenix, AZ with his dog, Sparky.