KMCategory Archive

As a consultant, I find it rare to spend fifteen minutes with a capable executive before she asks me to assess how they’re doing compared with other companies.  It’s a fair question: how can you know if you’re best in class unless you know how well other organizations are doing?  And if you’re not yet best in class, how can you know how to get there without data?

Benchmarking works best on truly objective outcome goals: revenue per headcount or per square foot of retail space.  Gross margin.  Hotel occupancy.  Share of wallet.

Unfortunately, support organizations own few of these.  Arguably, for some organizations, maintenance renewal rates are a clear outcome, although even then, this depends on many cross-functional groups.

No, support leaders tend to want to benchmark other, more operational metrics: first contact resolution (FCR), customer satisfaction (CSAT), website “did this help” ratings, contact avoidance, and the like.  And here’s where cross-company comparisons can confuse as much as enlighten.

First, on a very practical level, everyone measures these things differently.  You might think that there would be an industry-standard way of measuring FCR…but you’d be wrong.  (I was brutally disabused of this notion during a—I am not making this up—two hour conversation among ten service desk managers comparing and contrasting the details of their FCR measurement techniques.  These were among the longest, yet most enlightening, two hours of my life.)

CSAT, in particular, sounds like it should be straightforward to compare between companies, and yet it isn’t.  Survey results are notoriously sensitive to all kinds of details: how the question is worded and what scale is used for the answer can both substantially change the results.  More subtle forms of bias can creep in, too.  In our business, the most common is non-response bias—in other words, the small percentage of people who answer our surveys aren’t necessarily representative of all the people we want to reach.  Also, some companies exclude certain cases or customers from surveys, while others mix up transactional (post-case) surveys with relationship (periodic, overall experience) surveys. Comparing your “4.3 CSAT” against my “4.1 CSAT” really doesn’t necessarily teach me anything unless I really, really understand the details of how you measure CSAT.  If I’m simply using data from an industry benchmarking database, I have absolutely no idea how that particular brand of sausage was made.

If you have a deep and trusted relationship with a peer company, and you really understand how their measurement techniques match to yours, you might learn something from benchmarking.  Unfortunately, ethical access to the fine details of peer company operations is rare.

More strategically, measures reflect your philosophy of business. Zappos is well known for not measuring handle time, because they believe personal relationships in customer service fuel their corporate engine of growth.  So, what point is there in measuring your handle time against Zappos’s?

That’s an extreme example, but different business decisions drive different metrics, and obscure insights that might be gained by comparison.  I know some support leaders are fixated on time to first technical contact, others on contact rate, others on service levels, and still others on NPS.  None of these is wrong, but they will affect the measures.

So, benchmarking against other companies is fraught with difficulties.  Still, we need aspirational goals.  What to do?

First, sometimes benchmarks can be good to cause a deep rethinking of process.  For example, I worked once with a company that was very proud that they had engaged additional outsourced staff in order to move their time to publication for knowledgebase articles from 23 to 14 days.  When they heard that some companies do it in minutes, that data point spurred useful conversation.  (Clearly, this requires more than hiring some new reviewers.)

Secondly, the best benchmarks are self-benchmarks.  When I get on the scale, I don’t compare myself with the average male my height; I’m interested in where I am relative to where I was the week before.  Similarly, if I’ve improved my ability to capture, reuse, and improve knowledge, I’d like to see if my contact rate is going down, or my ability to close cases faster with reused knowledge is increasing.  Any metric I report on is a candidate for self-benchmarking, especially if I’ve been taking actions to change something.  Not every change is meaningful, but if you look at your own trends over time, in measures that are strategic to your business, you’ll have the best possible, least-gameable, most informative metrics of all.

Many of us “self-benchmarked” on doorways growing up.

(A quick note to start: our February KCS Foundations Workshop is filling up.  If you or a colleague are interested, now would be a good time to register.)

It’s a cliché that you get what you measure.  So before we start picking from the standard list of reports provided by your KM, CRM, or BI system, let me ask you: what do you want to get?

If what you want is the same thing as everyone else, then the standard measures should do just fine.  But if you have ambitions to stand out…if you have a new insight about how to do support…if you’re seeking competitive advantage, I’d say, not so fast.

When Southwest Airlines emerged into prominence, there were many standard airline metrics to choose from: occupancy, revenue per available seat mile, and the like.  But Southwest’s then-CEO, Herb Kelleher, recognized that their chief limit to growth was the number of extremely expensive airplanes they could afford to own.  Or, to be more precise, their revenue is bounded by the number of airplanes in the air.  This led to a simple insight: an airplane on the ground isn’t making money, so let’s minimize the time spent on the ground.

To put this insight into action, Southwest started measuring and improving their turnaround time, or the length of time their planes spend idle at the gate.  Much of what’s unique about Southwest, from flight attendants who clean the planes as passengers are leaving, to boarding by groups and unassigned seating, are the result of Southwest’s obsession with turnaround time.  The result is sustained profitability in a generally unprofitable industry, and the equivalent passenger capacity of 35 additional multi-million dollar 737s.  “Strategic” trumped “standard” for Southwest.

Other industry leaders have taken the same path.  Dell focused on its Cash Conversion Cycle (CCC), the length of time between paying vendors for components and receiving money from customers.  Dell’s remarkable build-on-demand infrastructure, leverage with suppliers, and streamlined operations resulted in a negative CCC—that is, Dell gets paid by customers several weeks before it has to pay its suppliers.  This cash float is a real engine for growth, and it’s there because Dell focused on its own, slightly non-standard, metric.

We’ve seen this in the support industry, too.  When innovative support and customer experience leader Brad Smith was at the helm at Openwave Systems, he had his team focus on margin dollars per case.  He reasoned that he wanted more margin (through efficiency and customer retention), and fewer cases (through product improvements, self-service, and proactive support).  The more margin, and the fewer cases, the better their margin per case.  I don’t know if any other organization has measured margin per case—and it certainly isn’t in any standard Support dashboard.  But Brad and his team were completely focused on higher renewals, more efficient case handling, and helping customers avoid cases…all great things for Openwave and its customers.

“Transformational measures can help organizations focus on what is most important today and for the future,” says Dr. Dean Spitzer in his outstanding book Transforming Performance Measurement.  “When we change our ways of measurement, the fundamental ‘lens’ used to view things changes.  Organizational transformation is what happens when people begin to see their organization through the new lens.”

If you’re not sure what your strategy should be, or how you’re going to transform your organization for the better, then by all means, use the same measures as everyone else.  But if you have a vision for where you want to go—and I hope that you do—then pick what Spitzer calls transformational measures to track and motivate your progress on that journey.

Because, you know, that’s what support does.  Close cases.  Quickly.  In a way that satisfies customers.  Mission accomplished; let’s grab a beer!

Not so fast.

This myth begs the question of why the Support organization exists.  Because most of the activity, headcount, and budget are tied up in closing cases, it’s easy to assume that the business of Support is closing cases, full stop.  But that’s far too limited a view of our business.

Support is in the customer success business—as a colleague likes to say, helping customers receive and perceive value from the product.  Closing cases can be part of that—a means to an end—but it’s not the most important part, and it’s certainly not the most valuable.

First, let me back up my statement about why Support exists.  When I ask a group of Support executives why they do what they do, they often start out with (true!) statements like, “the product breaks.”  Using a gentle version of the five whys, if I keep asking why that’s important, we always get to some version of, “if the customer doesn’t get value from the product, they won’t buy more, and they won’t tell their friends to buy, either.”  Bingo.  There’s no intrinsic value to the business in closing cases; it’s all about customer retention, loyalty, lifetime value, and referenceability.  Support feeds the company’s engines of growth and sustainability.  The Technology Services Industry Association has made this topic a focus of their research; their findings are summarized in Consumption Economics.

But even if we believe that Support’s mission is to make customers successful, it’s still easy to fall back on our old operational measures: cases closed, service levels, customer satisfaction (CSAT), and the like.  What’s wrong with that?

The problem is that it dramatically overstates the significance of the assisted case-closing channel as a way of delivering customer value.  Cases are both more rare, and deliver less value, than most support organizations think.

If customers have a problem with our products, or are struggling to use them, opening a case is surprisingly low on their list of next steps.  If you consider your own experience with technology, other approaches tend to happen first.  Think about it a minute…what did you do last time you had a problem?  If you’re like most people, you Googled it.  Or you asked a friend or colleague.  (I should know.  I’m often that friend or colleague.  And then I generally Google it!)

Sometimes I go to the vendor’s site and seek out an answer there.  Sometimes I’ll start a new thread in a community, although I’ll do so only after looking for a while, because I don’t want to be That Guy who re-posts an FAQ.  I do open support cases, but not usually.

Now you might say, as a consumer and small business owner, I’m not entitled to great assisted support, so I tough it out, making do with free resources.  But you know what?  I’d really rather figure it out on my own—even if the vendor is there to help me, I’d rather not jump through their hoops and explain it all to someone else if I can be self-sufficient.  Many people feel the same way.

At least, that’s what the numbers show.  As we look at our clients’ data, enterprise customers open a case only a small single-digit percentage of the time they want help.  (The number is much, much smaller for B2C support.)  The rest of the time, they’re using Google, the self-service site, in-product help, and social networks of various kinds, including of the sneakernet variety.  Sure, they tend to open cases for higher severity issues, but when it comes to customer success, every moment of need is important.

Do a perfect job on 100% of the cases you receive, and you’re only making a difference 5% of the time.  Or less.

So, when do customers open a case?  Generally when things have hit the proverbial fan.  When there has already been significant value erosion.  The system is down; something needs to be replaced; it’s throwing an error and not completing a task; it isn’t doing something it is supposed to be able to do; or, it’s doing something it shouldn’t be doing.  By the time the customer picks up the phone or logs in to our support portal, we have a problem, Houston, and we’re in recovery mode.

Closing a case quickly and well simply makes the problem go away.  It gets customers back to where they expected to be.  It doesn’t make things better than expected.  Certainly, herculean recovery efforts can engender a sense of personal gratitude, which is nice, but it’s doing a job well that never should have had to have been done in the first place.

Of course, once a case has been opened, closing it quickly and well is the best outcome.  But that doesn’t always happen.  Then things get really ugly, as you know from having been on both sides of this experience.

So, closing a case can be important, but only in so far as it gets us back to where we started.  What else can support organizations do?

• Provide feedback to Development to help them improve the product and the customer experience, avoiding problems in the first place
• Expand into a Customer Success Management role that proactively advises customers to help them get the most value from their products
• Build and implement configuration health checks and other utilities that help customers avoid problems in the first place
• Create knowledgebase articles, video tutorials, and automated fixes to empower customers with the expertise they need to be successful themselves, and to get themselves back to value more quickly when needed

All of these tasks  help customers receive and perceive value more than simply closing a case.  What else do they have in common?  Most support organizations are too busy to do them.  Why?  They’re closing too many cases.  Hmm.

Yes, but.  This is one of these partial truths where getting the details right really matters.

Followers of popular business books will recall the BHAG—the Big Hairy Audacious Goal—proposed by James Collins and Jerry Porras in Built to Last.  These goals are long-term, viewed as a long shot by outsiders, but considered possible by insiders.  They act as a unifying vision for the company, much as Lencioni’s “Rallying Cry” did in the last myth.

This kind of overarching, companywide, long-term goal is great, although, given how “goal” is commonly used at work, I’d prefer to call it a “measurable vision.”  Great executives set these, communicate them, and stick with them.

But most of the goals we have in companies aren’t like that.  They tend to be more tactical, more operational, and shorter term.  You can’t have a BHAG on service level compliance—BHAGs are a brass ring for the company overall, which service levels most decidedly are not.  So executives should stay away from these operational goals.

Why? Because they’re a means to an end—a way of achieving the BHAG.  And it’s the people actually doing the work who are in the best position to know how to get there, and how to hold themselves accountable to the work.

Those of you who have been to a KCS Foundations or Design Workshop may remember the Process and Change exercise we do with a ball.  While I’m not going to spoil it for future participants, I’ll say that the team estimates its own ability to improve its processes, and, especially after a few iterations, it’s remarkably good at setting its own goals.  The team is better that I as the facilitator would be, and I’ve led the exercise tens of times.  But they have a secret ingredient that I don’t have: they know each other, and they have hands-on experience with the process they’re measuring.

Forget the director in the corner office.  It’s way better for the people doing the work to set their own goals.

Now a cynic might ask, won’t they sandbag?  Won’t they set easy goals for themselves?  My experience is, unless the organization has completely stamped initiative and self-respect out of a team (in which case you need a different blog, or maybe just a different job), teams will challenge themselves and each other.  Almost all the people we work with want to do well, they want to feel a sense of accomplishment, and they want to be part of a high-functioning team.

Not only will teams do a better job of setting goals than their managers will, they’ll have that special something that comes from owning a goal.  If you tell me I have to do something, I might just argue with you.  But if I tell you I’m going to do something, just try and stop me.

So, executives should set an ambitious, exciting, and plausible vision.  When it comes to setting the goals along the way, let the people doing the work take the lead.  Support them.  And get out of their way.

“Men at some time are masters of their fates.”
Cassius, to Brutus, Julius Caesar, Act I, Scene 2.

It’s almost axiomatic that we should sign up for goals only if we have control over achieving them.  In Julius Caesar, Cassius berates Brutus for allowing himself to be treated as an underling; so to, conventional wisdom says that executives shouldn’t allow themselves be held responsible for the performance of other organizations.

It’s a nice theory.  Unfortunately, it doesn’t work in the real world.

First, like it or not, Support can’t be measured separately from the rest of the company.  Perceptions of Support are deeply enmeshed with perceptions about the product, its quality, and its supportability; by company policy; and by margin requirements levied by the CFO and Wall Street.  Studies show that the single most significant factor determining Support customer satisfaction is product quality.  No organization is an island, and least of all Support.

Secondly, what’s the point in optimizing the performance of Support—or any other organizational silo—if has the side effect of sub-optimizing the performance of the business as a whole?  For example, Support can deliver great First Contact Resolution (FCR)…if they’re solving the same, well-known problems over and over again.  This is great for Support’s numbers, but terrible for customers and the company.

Patrick Lencioni is a careful student of corporate culture.  In his book Silos, Politics, and Turf Wars, he dramatizes a number of environments in which various organizations were at each others’ throats.  While industries and details differed, the common theme was that each organization was working to meet its own objectives independent of those of other organizations.  This put the various orgs at cross-purposes.  Predictable conflict ensued.

This is old news to those of us who have been working in high tech for any length of time.  Sales struggles to make their numbers, then blames product delays, feature deficiencies, and Marketing ineptitude.  Product implementations go off the rails; Implementation Services blames Development, while Development wonders who hired these wet-behind-the-ears implementation teams.  The Product Management team wonders why no one ever builds the impossible products it dreams up, and everyone knows what happens to Support.  It’s a mess.

In this environment, does it really make sense for us to isolate ourselves from each other’s metrics?  Because clearly, our successes can’t be independent from that of other organizations.  As a CEO once told me early in my career, “David, there’s no way your department can succeed if the company fails.”

Lencioni proposes a two-part solution: a rallying cry (or overarching vision) for the whole organization, and cross-functional metrics to ensure that we’re all working together and supporting each other in executing on that vision.

This is a reason I love the Net Promoter Score (NPS), at least when it’s used right.  (It isn’t always used right; we’ll have a Metrics Myth about that soon.) No one organization can own NPS, and everyone needs to work together to improve it.  Unlike our siloed metrics of FCR, service level adherence, or customer satisfaction, something like Net Promoter Score gives Support a seat at the table.  It gives a reason for Development to listen to us when we talk about customer experience improvements in the product, and it gives Marketing a reason to pay attention when we make the case for exposing known issues.

As with SMART metrics, keeping our heads down and focusing on Support metrics is more comfortable than living in a world where we’re dependent on others.  But, since we can’t be successful without the rest of the enterprise, let’s pick cross-functional metrics that line us up in the same direction instead.

Metrics always seem to be a favorite topic of conversation.  At a recent CSI program team meeting, I was comparing notes with the Consortium’s Melissa Burch about some of the ways that metrics are misused in our industry.  Earlier in the year, my Third Tuesday collaborator, Francoise Tourniaire, and I led a talk on how much of the conventional wisdom in support is dead wrong.  Unsurprisingly, much of that “wisdom” had to do with metrics.  So, I decided it might be worth gathering up some of those threads and exposing our favorite metrics myths and half-truths.

Following the Spinal Tap motif that has now become conventional in this space, our myths will go to eleven.  We’ll break them in to bite-sized posts, one per myth.

If you’d like to see this in a webinar, rather than a series of blog posts, have a look here: http://www.youtube.com/watch?v=ayczz8W8kqw (warning: the audio quality is iffy.)

Metric Myth One: Goals Should be SMART

That is, individual performance goals should be Specific, Measurable, Attainable, Relevant, and Time-Bound…SMART.  Doesn’t that sound SMART?  Don’t I have to be a little DUMB to argue against SMART goals?

Well, not so fast.  We’re not on the assembly line any more.

F.W. Taylor, stopwatch and notebook in hand, invented the practice of scientific management.  He did painstaking research on the best way for men to shovel material (really!) and extrapolated general principles for making workers more efficient from these time and motion studies. He advocated paying workers by the piece, rather than a fixed salary.  He believed that there is One Best Way of doing any task, and that management’s job is to make sure that One Best Way is exactly how workers do it.

F.W. Taylor would have absolutely loved SMART goals.

The nature of work has changed in the last hundred years since Taylor did his groundbreaking (given his fascination with shovels, literally groundbreaking) analyses.  There’s now far less correlation between the units of work accomplished and the value that work creates.  Would you measure a programmer by how many lines of code she creates?  Or an advertising copywriter by the word?  I expect you wouldn’t like what you got.

KCS practitioners know that, when it comes to activity, more is not necessarily better.  As the KCS Practices Guide says,

“During the KCS adoption process, we have seen organizations put goals on KCS article creation (everyone should create five KCS articles a week) or KCS article reuse (analysts will be measured on how often they reuse KCS articles). The goals for these leading indicators may have been met, but the quality of the knowledge base has been seriously compromised. Invalid and duplicate KCS articles are created, because the focus is on the activity, not the outcome. Worse, emphasis can shift to gaming the system rather than generating real value. Inevitably, quality and morale suffer, management looks less competent, and the value of the knowledge is diminished.”

Unfortunately, it’s these activity measures that are just the kind of thing that are Specific and Measurable.  We believe there is no single golden metric for the value created in the knowledgebase.  Collaboration, which is notoriously hard to measure, is an important piece of employees’ contribution. We believe managers must do the hard work of providing feedback and assessing value by considering activities, outcomes, and quality measures in the context of the job and the environment—a judgment that’s driven by data, but is ultimately a human judgment, made in an environment of uncertainty.  Anyone who tells you there’s a simpler way is selling you something.

SMART goals are comfortable.  They’re reassuring.  You make them or you don’t, and it’s all up to you.

If you want comfortable and reassuring work, it’s probably time to pick up a shovel.  Knowledge work has many advantages, but being easy to measure isn’t one.  For support delivery staff developing knowledge, it’s time to bury SMART goals.