Creating and Managing the Analytical Business Culture

Creating an RFP: Overview

Web analytics products are classified as software applications. As such, and irrespective of the implementation method, each product has specific technical features and requirements. As part of selecting a Web Analytics product or vendor, it is critical to understand these technical aspects and identify which ones are applicable or feasible for your organization.

To better understand the various technologies in a product, the web analytics product is seen as consisting of three different, logical layers: 1) reporting and visualization, 2) processing analysis and integration, and 3) clickstream data collection.

Over the past 8 years, web analytics vendors have offered their products in multiple delivery models and through different pricing options. This module describes the pros and cons of each model and having a clear understanding of the pricing models is essential in identifying the optimum model for your organization.

Failure to understand these models may result in selecting a solution that costs significantly more than what was budgeted for or, more significantly, the complete inability to implement the selected solution.

As the web channel continues to increase in value to the organization, the demand for integrating it with the rest of the organization will increase. Even for pure dot com companies, other systems exist within the organization that could benefit from integration with the web analytics application.

Many systems exist within the modern enterprise. These systems manage the various operational aspects of the enterprise. Most common examples include:

• Finance systems
• Sales Systems
• Human Resources Systems

This module helps to determine which systems need to be integrated as part of the requirements gathering.

Creating RFPs: Objectives / Outcomes

Understand these concepts:

• The technical features and requirements in a web analytics tool
• Clickstream data collection categories
• Service delivery models for web analytics
• Various enterprise data integration scenarios
• The typical sales cycle for web analytics products
• Issues to look out for during the vendor selection

Be able to do these things:

• Identify a list of vendors and/or products that meet your organization’s technical requirements
• Create the technology subset of RFP items
• Identify the service model for your company’s requirements
• Identify enterprise data integration requirements
• Build a timeline and identify required resources during the various stages of the sales process
• Avoid common mistakes in selecting a vendor

---

Critical Implementation Strategies and Tactics: Overview

This module helps you to understand what data is required for the reporting process. A solid data collection plan will enable you to provide business users and stakeholders with the required reporting. Data is not limited to traditional clickstream data. It includes data from surveys, A/B testing, usability studies and similar activities that aim at understanding, analyzing and enhancing the performance of the web site.

The process outlined below represents a detailed overview of the steps that can be part of the sales cycle. While at first sight they may seem lengthy and involved, careful inspection would show that we may go through them in one fashion or another. Depending on the size of your organization, you may be able to skip some of these steps or combine them.

• Selecting the team
• Preliminary internal requirements gathering
• Independent market review – Building the long list of vendors
• Building of the vendor’s short list
• Building the RFI
• Selecting 2 finalists
• Build the RFP
• Conducting a proof of concept
• Contract award

While web analytics is relatively new, reporting is a well established activity in any business. Parallel to the growth of web analytics, significant growth also occurred in the Business Intelligence (BI) sector. The BI industry has not completely embraced the web analytics area. While some cases do exist where traditional BI players have invested in web analytics, due to the unstructured nature and complexity of the web channel and due to market fluctuations, web analytics generally existed outside the BI tools.

This has given rise to an interesting situation in the organization; the existence of multiple reporting platforms, one for the web and one or more for everything else.

This module includes a quick overview of the various data sources with a focus on the traditional clickstream data collection methods. Topics also covered include quantitative vs. qualitative data, panel-based vs. site data, and client side vs. server-side data collection. Later sections in this module discuss key reporting features and data export and import capabilities.

Implementation Strategies and Tactics: Objectives / Outcomes

Understand these concepts:

• Various models for data collection including quantitative and qualitative data
• The key differences in how vendors offer reporting and data export

Be able to do these things:

• Identify when to use each data collection method and issues associated with each collection method
• Investigate different reporting requirements, including those that involve non-clickstream data sources
• Assess reporting capabilities and how they relate to the business requirements

---

Organizational Structure and Behavior: Overview

This module examines the core description of a Web Analytics Manager and possible placements for that position in an organization. The first section concludes with the proposed best placement for this position.

Looking through the multitude of job descriptions for the Web Analytics Manager position, you’ll notice commonalities in the activities, scope, work product, tools, qualities, and pedigree. This is important to the advisory network in the company across various organizations in order to maintain a look ahead at potential hurdles as well as integrating the manager’s function into colleagues’ consideration and plans.

A later section discusses how analytics brings accountability to people’s work. Often for the first time, people are being held directly accountable for their decisions. But accountability breeds fear, because some people will feel that their jobs are threatened. There is a proper way to handle these emotions and this should be done up front, before producing reports that might be viewed as a “threat”.

Assuming you are successful at addressing this primary and very important “fear of failure” topic, and especially if you are a manager, you then have to keep building and enhancing the analytical culture in the business area. One thing to watch out for is becoming too “numbers focused” at the expense of creativity; make sure you keep “both brains” at the table. Don’t let the newly empowered left (analytical) brain smother the right (creative) brain. You want to encourage new ideas.

This module includes a discussion assignment called the Analyst's Enigma.

Organizational Structure and Behavior: Objectives / Outcomes

Understand these concepts:

• The role and the responsibilities of the Web Analytics Manager
• The impact of web analytics on the corporate culture
• How the presentation of new facts and data is critical to your success as an analytical manager

Be able to do these things:

• Analyze the interactions of the Web Analytics Manager in a cross-functional environment within an organization
• Manage the corporate culture issues and encourage an environment where analytics are embraced
• Define the importance of planning for presentations and preparing presentations based on the audience

---

Role Definition: Overview

This module helps you to determine who will make a good web analyst. It is more a question of mindset and personality than it is of rote skills or past experience. Of course, hard skills and past experience make a difference, but you’ll quickly find that experienced web analysts are in high demand, and are demanding increasingly higher salaries. As a result, the most efficient way to build your organization is to hire one solid analyst, if you can find one, then focus on building a team of people who, based on mindset, personality, and key non-analyst skills, can be quickly groomed to be solid web analysts. To do this, you have to look at candidates not for what they’ve accomplished but for what they’re capable of.

Web analysts are in huge demand these days as more and more companies are realizing the importance of this role in driving their online businesses. While this is requiring that the position and responsibilities of the Web Analyst be well defined, the number of individuals who can fill these requirements is literally dwarfed by the demand. It is important that companies become as creative as possible in trying to fill their web analytics resource needs.

This module also discusses where to find web analysts and source pros and cons.

Role Definition: Objectives / Outcomes

Understand these concepts:

• The personalities, backgrounds, and skill sets likely to lead to a talented web analyst
• The different avenues for finding a great web analyst

Be able to do these things:

• Recognize a "good fit" even if a candidate has never been a web analyst
• Find the best web analyst for your company

---

Stakeholder Education: Overview

This module discusses how to educate stakeholders on using web data to establish a program of continuous testing, thus ensuring future growth of the website.

A section in this module describes the process by which education must take place. It is not enough to provide the end-user with a canned report and show them how to run and print it for themselves. Training must be delivered in a give and take process; the web analyst must PICK the plan.

• Pinpoint the goal
• Identify the audience
• Choose appropriate metrics
• Kick off the final product

In addition to mastering the PICK process, it is critical that the Web Analyst also be able to provide high-value training regularly, in an efficient and consistent manner.

In this module, you also discuss how to manage reporting requests, and explore the three key activities in maintaining your queue of requests. Then you can introduce a process for implementing these activities.

More so than any other function, web analytics needs cross-functional support. As organizations become more data-driven, resistance to this change is inevitable. Perception is reality and no one wants their job to become more complicated. The key to overcoming these obstacles is, in fact, to make cross-functional team members’ jobs easier and make your peers look good in the process.

You’ll also discuss internal case studies that the address challenges above by celebrating the successes that are the result of using analytics to drive the business.

Stakeholder Education: Objectives / Outcomes

Understand these concepts:

• How to educate a stakeholder
• The analysis request management process
• How to create a structured case study

Be able to do these things:

• Develop a tailored education plan, identify audiences, and write testing goals
• List data points needed in a report request
• Explain why it’s important to communicate project status to stakeholders
• Conduct a case study program

---

Focus on Business Objectives: Overview

This module contains information about the most important tasks for a web analytics manager. Without a clearly defined set of business objectives, it will be nearly impossible to create and manage an analytical business culture as these are the building blocks for executing a successful web analytics strategy.

There is one central question that must be answered in order to achieve the above: “Why does the website exist?” There are generally two buckets. There are internal reasons why the website exists (i.e. objectives focused on driving the business) and external reasons why the website exists (i.e., objectives focused on addressing customer needs).

The cross-functional nature of the web analytics manager’s role means that he or she must have the ability to navigate an organization from both a functional and hierarchical perspective when creating deliverables. Very few items that the web analytics manager is responsible for can be created in a vacuum. Likewise, deliverables are often reviewed by a number of individuals across the organization. You can see that being able to interact and maintain positive relationships with influencers and stakeholders has a direct correlation to your success in creating and managing an analytical business culture and as a web analytics manager in general.

Discussion in this module includes how to manage expectations, pay attention to your timing and always have a “help me help you” attitude. You must have the approval of the influencers and stakeholders of your organization. Be sure to tap into their expertise to assist you while you are still developing a given strategy or deliverable. Involving these people early in the planning process is important. People may feel patronized by a last ditch request for feedback at a point where you are unable to act upon it anyway.

Focus on Business Objectives: Objectives / Outcomes

Understand these concepts:

• The rational for the company website and how it contributes to the organization
• Internal and external objectives of a website
• The process of interviewing influencers and stakeholders and what should be considered in the course of the process

Be able to do these things:

• Plan and execute the process of defining a set of internal objectives for your company
• Obtain necessary input from those who can help you to win acceptance and support from stakeholders

---

Functional Ownership for Web Analytics Data: Overview

This module delves further into the skill required of the Analytics Manager. Some skills center around understanding the context of the data that is recorded (i.e., the position within the buying cycle), interpreting the data into meaningful concepts, patterns, correlations or causes when appropriate; and communicating these understandings back to the business units, owners or organization.

You’ll learn how role of the Analytics Manager now spans the understanding of marketing, layout, merchandising, to some degree logistics or inventory as well as sales analysis. The AM has the critical role of deciding how the tags should be used to track marketing activity. As analytic packages have matured over the years, most have developed multi-part tagging capability. This provides a wonderful array of possibilities for the AM. However, it also adds complexity that needs to be addressed before confident tracking can occur.

One section discusses why most merchants have a difficult time understanding how analytics can help them beyond the tools they already have.

Imagine if the local department store knew how customers flowed through their store. What if they knew which displays they looked at and for how long? What could they do different or more effectively if they new the specific sequence in which customers shop? Wouldn’t they like to know everyone who picked up the new shirt, but put it back after looking at the price? These kinds of things and more are possible to know in the ecommerce realm. The Analytics Manager has the role and responsibility to track this behavior effectively and be able to translate it into something meaningful.

Functional Ownership for Data: Objectives / Outcomes

Understand these concepts:

• The role of the web analytics manager as it relates to cross-functional responsibility and accountability
• The responsibilities of the marketing function and their differences from those of the web analytics managers
• The responsibilities of the merchandising function and their differences from those of the web analytics managers
• The responsibilities of the usability function and their differences from those of the web analytics managers

Be able to do these things:

• Work with other functions to drive adoption of analytical principles in your organization
• Describe specific tasks involved in helping the marketing department achieve accountable objectives
• Describe specific tasks involved in helping the merchandising department achieve accountable objectives

---

Incorporating Web Analytics Elements into Existing Processes: Overview

Your ability to report on web site activity is directly related to and dependent on the data you collect. Collecting the data depends on understanding the process through which the web site is built and or changed.

This module discusses how web analysts needs to conduct their own testing and QA. Once the site is deployed, web analysts need to ensure that they are aware of any changes made to the site that can impact the reports being generated.

How business processes need to evolve in order to incorporate web analytics elements is a frequently discussed topic these days. Organizations are quickly realizing that on a relative basis, implementing analytics tools and generating reports is a piece of cake compared to finding experienced resources and training their employees to truly internalize and operationalize web analytics.

The website release process in particular needs to evolve, and the web analytics manager can facilitate this transformation. In this section, similar concepts will be applied to the campaign management process. Together, these two process transformations will allow organizations to make significant strides towards adopting an analytical business culture. This is not only due to the new processes themselves, but also the result of having actually gone through the steps necessary for completing the transformation.

Incorporating Web Analytics: Objectives / Outcomes

Understand these concepts:

• Key components of a standard Website release process
• Business Requirements Document, Functional Requirements Document, Technical Design Document
• How analytics brings new cultural challenges to the business setting that must be met in the development process

Be able to do these things:

• Apply the website release framework and modify it for your business
• Create your website release process to accommodate web analytics requirements
• Work through cultural issues when you plan the process and create an environment where analytics are embraced

---

Product Roadmap for KPI Dashboard: Overview

This module discusses how key metrics need to be customized to the goals and function of your company's website and the company's management objectives.

During the initial analytics implementation process there were most likely measurement objectives defined by a core committee supporting the vendor selection process. Regardless of which path you choose in managing your time and efforts focused on KPI reporting, you should always separate macro-level and micro-level reporting. Success metrics displayed in the form of KPIs should provide organizations with a roadmap to those micro-metrics that provide insight into how specific actions contribute to that success.

For e-commerce sites, the first step should be mapping how the purchase process is supported by the web channel. For other types of sites, such as content or information sites, the process to be mapped is one of customer engagement. The ideal dashboard will focus on the handful of key metrics that track acquisition, retention and loyalty behaviors.

A section of this module describes the importance of key management executives, who are stakeholders in the website operations, to see what metrics they want to track.

Streamlining all reporting vehicles for internal data distribution can be a challenging task and involves a multifaceted process. Web analysts will have to consider all levels of company staff from support positions to top level executives, and, moreover, stakeholders with varying ranges of web knowledge.

One size does not fit all in web analytics; dashboards can and should take on different formats. The practices in this section will help you create and modify reporting processes allowing users to benefit from increased efficiency and consistency.

Functional dashboards can provide users with relevant and actionable insights that empower them to make effective decisions in an efficient manor. Many different members within an organization can benefit from insightful analytics, provided they are delivered relevant data. Relevancy in this context is that the information disseminated throughout the organization is distributed based on the position and level that users hold within the organization.

Product Roadmap for KPI Dashboard: Objectives / Outcomes

Understand these concepts:

• The Dashboard and how to assess it for meeting the company's needs in charting the critical milestones that lead to desired customer behavior
• Data distribution, standard naming conventions, standard formats
• Hierarchical model for stakeholder reports by organizational role

Be able to do these things:

• Define and defend behavioral and attitudinal metrics in the dashboard, including industry benchmarks and advanced concepts
• Demonstrate how to implement a structured environment for internal data distribution
• Map which KPIs should be disseminated to each stakeholder

---

Determining Where to Focus Efforts: Overview

This module deals with how to prioritize areas in which the website can improve. You’ll learn how to focus on your site's most important elements and get your team to agree on the priorities. You can start by creating a list of initiatives and opportunities, then monetizing the potential effect of each one. This requires assessing the upside of each site change, assigning a dollar amount to each, and comparing them to one another.

Discussion in this module includes understanding how analytics brings new cultural challenges to the business setting that must be met. You’ll also learn how to work through cultures issues and create an environment where analytics are embraced. This means that you have to understand how to get people into an "accountability status" for what they are supposed to do.

Determining Where to Focus Efforts: Objectives / Outcomes

Understand these concepts:

• How analytics brings new cultural challenges to the business setting
• Dynamic Prioritization process

Be able to do these things:

• Assign a value to the desired site behavior
• Work through cultural issues that are impacted by web analytics in order to create an environment where analytics are embraced

---

How Reporting Drives Investigative Analysis: Overview

This module addresses how important it is to resist the urge to explain trends at face value, and instead strive to understand the underlying dynamics of the trends. Many web analytics managers acknowledge that they’re drowning in data. It can take so much time to pull reports, review them and focus on the most important trends. Adding the step of questioning the trends is something you may not feel you have time for, but it’s essential to channeling the data to guide you instead of trip you up.

There’s a lot of talk about cross-functional teams in the management of the analytical culture, and root cause is the reason for this, because root cause is often cross-functional in nature. Having a cross-functional mentality in place ensures that these complex root cause issues are addressed without finger-pointing. In a good cross-functional team, there is no blame, only learning and continuous improvement. If you don't chase down the root cause of your issues, the problem likely will continue, creating pain for all members of the team.

Reporting Drives Investigative Analysis: Objectives / Outcomes

Understand these concepts:

• The importance of finding the underlying dynamics of the trends rather than explaining trends at face value
• The Six Sigma approach to problem solving

Be able to do these things:

• Employ additional tools and solicit input from your organization to identify the cause-and-effect dynamics effecting trends
• Assess the true causes of a problem

---

Managing Downward and Upward: Overview

This module addresses how important it is to be proactive about driving the analytical mission – as opposed to reacting to it. This means communicating well – and a lot of this may be accomplished via email – up and down the organizational ladder.

By managing those above you, you'll be able to guide your company's decisions in the right direction and, hopefully, to create a fulfilling job for yourself. Although your efforts may not bear fruit immediately, you must keep in mind that companies often change their policies. When that time comes, your ideas will be on the table.

Managing Downward and Upward: Objectives / Outcomes

Understand these concepts:

• The kinds of staff communication that may occur before an analytical project
• What kinds of approaches work best when dealing with senior levels of the company

Be able to do these things:

• Empathize with people moving to a more accountable business model
• Generate excitement and interest in senior managers for the analytics effort

---

Emphasizing Planning and Communication: Overview

This module deals with the importance of improving site efficiency. To ensure maximum site and campaign performance, you must develop performance goals and measure your progress against these goals. However, these goals should not be set without a firm understanding of current performance. This is where the practice of establishing baselines begins. This module helps you to understand why baselines are important and how to use them when performing impact analyses.

In order to achieve maximum results, a testing strategy should be developed. Creating a testing strategy will enable you to test your site and your site content, and will allow you to determine which design has the greatest impact on your site visitors. This type of insight into visitor behavior allows you to continually improve the overall effectiveness of your site navigation, site content, and site promotions thereby increasing the overall performance of your site. Testing should be considered an essential part of your overall site optimization strategy and a key initiative of any successful online business.

Data can be dangerous when not fully understood by its recipients who may ultimately make bad decisions based on their incorrect interpretation. Therefore, this module stresses the extra care that you need to take in preparing, distributing, and following up with data sent across the enterprise.

Planning and Communication: Objectives / Outcomes

Understand these concepts:

• The importance of testing in the site optimization process
• Limitations and benefits of various data sources for establishing baseline metrics
• The most optimal ways to prepare an organization for data sharing
• Effective formats in presenting data to disparate groups

Be able to do these things:

• Create and define baseline metrics for impact analysis
• Develop a testing strategy and implement test results
• Prepare an effective and actionable scorecard
• Communicate data effectively across the enterprise