Menu
Choosing a mystery shopping provider can be tough.
Almost every mystery shopping company will tout their service and high quality of results. Cut through to find a provider when you choose something other than price.
Shopper management is one of the more critical aspects of mystery shopping and often distinguishes different providers. Choosing a mystery shopping provider is tough.
A variety of methods may be used to employ shoppers,
including cold calling, newspaper/magazine advertising, referrals, or even
bureaus with lists of available names.
At times, the customer’s own database can be used to recruit
shoppers.
As the client, it is important for you to understand these
sources. Be careful of shoppers being used from mystery shopping bureaus
because the detailed shopper information may not be available to the mystery
shopping company providing your service.
Ask for a list of all data fields captured for the shoppers.
In many cases, information such as height, weight, income group and sex can be
used to determine if the data is skewed to one demographic group. Alternately,
this information can be used to screen out inappropriate shoppers such as a man
trying women’s clothing.
Some mystery shopping companies use the employees (whether
full or part-time) while others use contractors. Most companies are now moving
towards the contractor route.
The most basic requirement is to ensure that shoppers can
read and write. Although software can detect spelling errors, they cannot
detect the incorrect use of a word. For example, if a shopper uses “their”
instead of “there.”
With strict auditing, many of these errors are detected, but
then some can slip through. To prevent problems for the exceptions, a mystery
shopping company needs to be sure the shoppers have good literary skills.
If a shopper cannot comprehend the shop guidelines or report
back accurate results, the shop can be made invalid, which then puts pressure
on the mystery shopping company to re-shop the store, thus putting pressure on
on-time delivery.
Testing shoppers with trial shops and writing/comprehension
tests have been shown to be unreliable. Anyone can get anyone else to help them
with an online test completed at home.
The more relevant qualification of a shopper’s ability is
their past performance.
Ask your provider how they vet their shoppers on
recruitment.
Ask whether the shopper demographics allow you to remove
shoppers with compromised interests. For example, as a clothing retailer, you
may not want a competitor’s employee mystery shopping your store. Not only can
your results be tainted, but the questionnaire could fall into the possession
of the competitor company itself.
This screening also allows you to remove shoppers who would
be unnatural in your store (usually a sign of a mystery shopper).
At times, it may be necessary to pre-screen shoppers before
conducting an assignment. You may have some very specific requirements for the
round of shops or require filtering beyond the initial questions asked by your
provider. For example, you may only want shoppers who have not been in your
store in the past month or who do not currently have your product.
If your filters are too restrictive, they may have an impact
on price. The more restrictive your requirements, the harder it will be for the
provider to find shoppers and may increase cost, or compromise the on-time
performance of shops.
You must decide whether you want shoppers who want to be in
your store or who are forced to go to your store. Your normal customers go to
your store or call centre because they have a want or need, usually not because
someone has forced them.
Allowing the shoppers to opt-in to the shop places means you
will have more normal people in their normal environment.
Ask how shoppers are given the opportunity to shop your
store. Are the shops assigned manually or automatically? This can impact you as
the client because efficiency in deployment of shops will increase the
likelihood of you receiving on-time results.
If the mystery shopping company allocates shops to shoppers,
it can create more consistent results, but they may be too consistent because
the shopper becomes an expert and doesn’t see the evaluation on its own merits
but in comparison to the best and worst they’ve seen. In addition, their
comments tend to rehash the same themes, whereas self-assigned filtered
shoppers provide more richness and diversity.
Shoppers who do not reside in the area of your stores can be
easy to detect, especially in small country towns. Store staff have an uncanny
ability to spot non-locals. I tried this myself when I started the company. We
didn’t have many shoppers in a rural area where we needed to find a shopper to
apply for an agricultural loan. I decided to make the drive and do it myself.
The bank manager asked me about my fictitious farm, why I needed the loan, and
then started talking about the market price of cattle that morning. I couldn’t
carry the conversation and left in a hurry.
Local shoppers are important even for phone shops to local
stores. People love to talk about the weather, and the shopper needs to be
close enough to carry that conversation.
Ask whether the shoppers are local or travel a distance to
the shop assignment.
Under what circumstances can your provider delete a shopper?
The answer to this question usually depends on the status of shoppers, whether
they are contractors or employees.
Shoppers come in various shapes and sizes. A typical mystery
shopping company will have thousands of shoppers. In any cross section of the
population, you will find people who are unreliable and incapable of conducting
a mystery shop. It is not easy to conduct a mystery shop; in fact, it is much
more difficult than it may otherwise seem.
The easier to delete a shopper, the less likelihood you will
have a rogue shopper conduct a shop on your premises.
Your provider will require at least ten shoppers for each
site and each evaluation. Whatever the site, some shoppers will not be
available for a number of reasons, such as illness or other commitments. Therefore,
more than one shopper is required per site.
The more frequently a location is shopped, the more shoppers
are required, because the same shopper should not shop the same store too
frequently.
When you ask the provider to match shoppers to your site
locations, ask for the breakdown showing number of shoppers within a 30 km (or
20 mile) radius of each site. There will be gaps, and the best company is not
necessarily the one with the least gaps. Those gaps need to be explained by the
recruiting processes.
Often mystery shopping companies keep the number of shoppers
as low as possible, pending the signing of new clients. The reason is that they
do not want idle shoppers in locations where there are no assignments.
Many mystery shopping companies quote ridiculously large
numbers of shoppers forming their shopper database. A common way to achieve
large numbers is to simply ask people to provide a name and email address in a
simple registration process. These are not shoppers but simply people
interested in one day becoming a shopper.
When you ask for shopper numbers, ask for the number of
shoppers that are fully engaged and active.
It is very difficult to cut through these statistics to find
the truth. Some companies deem that any person who has provided even a fleeting
interest is an available shopper. This may be true; however, you may then wish
to call into question what vetting and screening processes are adopted.
It is more relevant to ask how many shoppers have conducted
successful evaluations in the last year.
Depending on the industry, this is a critical factor in
managing a program. Shopper rotation refers to how often a shopper can return
to the same store in a cycle. If you have the same shopper visiting the same store
for every shop, you will receive the input of the same person. If that person
is restricted from revisiting the same store too frequently, it will allow
other shoppers (with another perspective) the opportunity to evaluate the
service.
Although mystery shopping questionnaires tend to be very
binary (yes/no answers) to remove subjectivity, the shoppers are often asked to
provide free form comments that often assist the store in understanding the background
to the results. Having the same result from the same shopper each month
provides no additional insights.
Having the same shopper visit has the shopper internally
comparing results between shops. This can be especially unfair if the shopper
begins to form an affinity with a staff member or the store itself. In other
words, they may start marking the store higher and higher as their affinity
grows.
In some cases, such as in the car industry, it would be
unnatural for the same shopper to enter the store each month to ask about a new
vehicle. In a large grocery store or fast food outlet, it would be less
problematic, but nonetheless one dimensional to receive the same shoppers
results each month.
Be satisfied your provider can deal with these rotations.
Shop frequency refers to how many shops can be conducted for
your brand over all. This should be restricted to remove the opportunity to
form an affiliation with the brand.
The other reason to restrict the number of shops per client
per month is to reduce the damage if the shopper is discovered.
If the shopper is discovered, the stores may talk among
themselves and compromise your shops in many locations.
Be satisfied that your provider has enough shoppers to cover
the rotation and frequency of shops.
The final restriction on shoppers is the number of shops per
month, across all clients. Professional shoppers tend to internally compare
clients and stores. Mystery shopping is about the singular experience at a
point in time, not comparisons.
The most alarming aspect of allowing too many shops per
month is inconsistency within a day or month. A shopper’s attitude on a fresh
Monday morning will be very different to a rushed shop assignment at the end of
the week on a rainy day.
Restricting the number of shops per month keeps rotation in
the shopper base. This restriction is not popular with shoppers but ensures you
retain randomness of shopper demographics, shop times and even moods of the
shoppers. From a client perspective, most importantly, you can be more assured
you have a fresh, keen shopper, with less likelihood of internal comparisons.
To keep the randomness of shoppers cycling through your
stores, ask the provider to show the demographic break-up of your shoppers in
terms of sex, age, income group and state.
When choosing a mystery shopping provider you clearly need to be concerned about quality. But quality in what? Here are some points to consider.
Despite software editing and auditing of results, there
remains the opportunity for error where the wrong word is used in a
questionnaire. As mentioned earlier, spell checkers will not detect the
incorrect use of a word such as “their” instead of “there.”
Reading each result will also pick up inconsistencies in the
questionnaire. At times, the free form comments can contradict the shop score
from yes/no questions. These inconsistencies can only be detected by the human
eye. For example, the mystery shop result may show a low score but the shopper
comments indicate they had a wonderful experience.
These inconsistencies can sometimes be explained, but they may
also indicate a problem with the results.
Another example is if the shopper indicated they waited five
minutes to be served, and then in the comments, they made a comment that they
were impressed by the quick service.
The most common form of verification is to simply read each
result.
Ask your provider to explain their verification process and
whether there is any additional charge for the service.
When possible, the provider should retain a receipt of the
shop to verify the shop took place. Receipt verification will also allow the
provider to check the shop took place during the times (restrictions) required
by the client and that the shopper went to the correct location. A time-stamped
receipt is more accurate than time stamping a photo of the store, because the
photo may be taken many minutes after the mystery shop took place.
Typically, although the shopper will enter the shop time and
date on the mystery shopping system and should be used to verify the time of
the shop.
Although the back operations are not your concern when choosing a mystery shopping provider, there are some elements you should be aware of.
Ask whether the system can cater for off-cycle shops. Although
your program may be monthly, occasionally you may want to add a wild card shop
(off-cycle). You may decide to conduct additional shops for whatever reason, but
once in a program, you will find there is an additional cost of implementation
or that the off-cycle shops are not possible.
Understand the process of briefing the shoppers. Are they
simply dumped with a questionnaire or are they pre-briefed on the requirements
and expectations?
Ask how quickly the shopper is required to submit their
results. A critical aspect of ensuring validity of results is to have results
input and returned quickly.
Typically, results should be input within 24 hours of the
shop taking place and returned to the client within three days.
Ask how the provider monitors how the shops are being
assigned and taken up by the shoppers. You can be confident the shops will be
completed on time if operationally there is a live update of each shop’s
status. Typically, the shop status includes classifications, such as:
Web reporting and reporting dashboards with automatic action reports is now the norm but their functionality is critical when choosing a mystery shopping provider.
Ask questions about where the data is stored and database
redundancies.
Typically, the individual receipts should be stored online
but are not necessarily made available to the store online. If there is a
query, you will want quick access to the receipt and be confident it can be
found, as opposed to being lost in a filing cabinet.
Providing receipts online can provide the opportunity for
the store to undertake investigations to identify the shopper, thus
compromising future shops.
In larger organisations it is important to have different
levels of access where each level can only see their own area of
responsibility. You may not want individual stores or state managers to see
each other’s results but still see their rankings.
At times during a pitch, the costs are simply stated as a cost per shop. The following costs may not be made clear until the decision made to appoint the successful company.
Choosing a mystery shopping provider, like most things, is not just about price.
Changes to questionnaires are not abnormal, especially early
in the program. Despite the best intentions by both parties to refine the
questionnaires, there is no substitute for seeing real data flow from the
program.
Understand the pricing of minor changes.
Understand if there are additional costs for manual reading
of the results.
Short-term contracts as short as one month are more and more
common and reasonable for your provider.
At times, especially with contracted shoppers, the provider
may introduce mileage reimbursements for shoppers. If these are necessary, ensure
there are minimum and maximum distances.
It would be unreasonable to have travel payments for short
distances five minutes from your site, and similarly unreasonable not to pay
mileage for over an hour of travel time.
Being part of affiliations may give the impression of being
an industry leader; however, more pertinent may be the ability for the provider
to cooperate with other worldwide offices.
When choosing a mystery shopping provider you may want to consider visiting the pre-eminent global association for mystery shopping companies is the Mystery Shopping Providers Association http://www.mspa-global.org.
As a client, it is reasonable to request not only the raw
data from completed shops but also shopper statistics such as demographics.
Regardless of the quality of questionnaires and provider,
there will be questions asked from the recipients of questionnaires. Ask you
provider about the mechanisms for feeding store questions.
Current best practice is to allow online queries directly linked
to the questionnaire results. This allows the feedback and responses to be
perpetually linked to the store result.
Meet the people, get the bios of critical people, and build
the human connection. Mystery shopping companies are run on computers, but
their success is driven by the people.
Mystery shoppers are paid cash, and wherever there is cash
being distributed, there is the opportunity for fraud. Here are four questions
to ask your provider:
Hope this helps you in your efforts choosing a mystery shopping provider.
© Copyright 2021 Service Integrity | Privacy Policy