|
|
|
www.ASPonline.com |
ASP Forum
Benchmarking Knowledgebases
"We'd like to compare the success rate of our online knowledgebase
against other companies, but we've found that there isn't a standard
industry definition of 'success rate.' How can we benchmark our
results?"
—Gretchen from Great Neck
Gretchen—
David Kay's suggestions are right on the mark (see feedback below).
Self-benchmarking may be the best and the only option available today.
It is almost like the game of Golf, where you are trying to improve
your game and continuously. I would like to add a couple of things for
you to consider.
1) A typical search entry in a Knowledgebase often fetches too many
items to read and it can be overwhelming for someone who needs a quick
diagnosis and resolution. Our own research focuses on the total time
to resolution from start of a user query and comparing it to live
support with an expert. Ideally, the end user should want to go to
knowledgebase first before calling live support. We are finding that a
good old database of use cases and "frequently encountered problems"
related to those use-cases often help diagnose the issue rapidly rather
than free-form query into text files. Ninety percent of the time delays
in resolution seems to involve diagnosis.
2) Often, the cost of support and help-desk and maintenance fees pale
in comparison to the cost of downtime for client enterprises. If the
user is frustrated they will abandon. This abandonment rate is a good
measure to benchmark. This requires that you track the exact path taken
by a user using bread-crumbs to determine if they abandoned the
Knwledgebase. In addition, if the same person followed up with a call
for live support, then it is telling.
The ultimate goal is to reduce the stress levels of the user who is
under the gun to fix a problem. They are generally on intense
time-deadlines to get things back on track.
—Nat Kannan
CEO
Jeeva Portals
Managed Product Support Portals
nkannan@jeevaportals.com
You're right, there is no industry standard model for benchmarking
knowledgebase success, and in fact, there's a great deal of pernicious
nonsense propagated on the topic.. For example, you sometimes see
people assume:
– You can extrapolate the percentage of people who answered
"yes" to a "Did this knowledgebase article help you" question, even
though a tiny fraction of people provide feedback
– An opened document means the session was successful
– A successful self-service experience is the same thing as an
avoided call. n fact, depending on the business, only a small fraction
of successful self-service interactions are deflected calls. This
doesn't mean they're not important...they are! They satisfy demand
that would otherwise have gone unmet, increasing the value of your
product, loyalty, and—over time—they drive the top line.
So although companies do provide their data to a few benchmark sources,
you're right that they're calculating their percentages differently
and it's hard to compare.
I'd offer two suggestions that might get you part of the way to your
business goal:
1. Self-benchmark.
Even if you're not sure how others are doing, it can't hurt to
continually raise your bar on your own performance. Pick a measurement
technique—I like calling a few hundred customers who use your
web portal and asking them, "What was your objective in coming to the
site," "Were you successful in accomplishing that objective," (if yes)
"Would you have opened a case if you had not been," (if no) "Did you
open a case?" and finally "What words of advice do you have for us in
making the site better?"
It's a short conversation, but you get great data: what was their
interaction goal, were they successful, what's the escalation/deflection
rate, and other feedback. If you don't have time to do this yourself,
well, summer's coming and you can hire an intern, but doing this for a
half-hour a day is a great way to hear directly from your customers...
there's no substitute for the wisdom you'll get! Of course, make sure
you're coordinating with other people doing surveys and the other powers
that be, but don't let them put you off by offering to add a question to
a larger relationship survey. This is a very tactical, very focused
activity.
2. Rate comparable use cases with your competitors.
Assuming you have access to your competitors' customer support sites,
you can come up with a list of the top ten or twenty customer
interaction goals, try them on your site, try them on your competitors'
sites, and see how each one scores. Sure, this is just a sample, and
sure, you may know too much about how your own site works, but it's
still a source of great insight.
Finally, I can't leave this post without recommending benchmarking
yourself against the ASP Online Ten Best Judging Criteria—and,
of course, entering the 2010 competition. I recommend this to all of
my customers. From what I've seen, the Top Ten winners are doing an
increasingly good job at scoring high across the board, and if you're
not sure you are, you may have just answered your own question.
—David Kay
DB Kay & Associates
david@dbkay.com
www.dbkay.com
408-568-3551
[If you have any other advice on this question, please send an
email to membership director Jane Farber at jfarber@asponline.com,
and we'll post your feedback.]
|
|
|
