Ok guys, stay with me here. I’m going to talk about Marketing Research and in particular Amazon Mechanical Turk (MTurk from now on) and some problems to watch out for.
So to a lot of us this stuff may sound super dry and you will be reading this like
Ugh why should we care about research and data?
But this stuff is actually pretty important. As a marketer, no matter how well you think you know your customers, chances are there is still a lot you can learn about them that can help with important decisions. The importance of prioritising market research is widely acknowledged within the marketing profession but it can be hard to convince others, especially considering effective market research can be a very costly exercise.
Three common reasons why businesses fail to effectively implement market research are:
- Senior management doesn’t understand the value and therefore doesn’t consider it a priority or use it correctly.
- Differing resource issues such as lack of time/motivation/money and technology
- Problems with the quality and quantity of the data
Thanks to the new amazing world of technology and the internet market research has evolved a bit more from mall-intercepts and mail-surveys. Now you can do surveys online for money!!! Ok so when I say money I’m talking like 50c a survey or some ridiculously low price but if you want to spend your days completing online forms then there is some cash dollar to be made.
MTurk is one of these places where you can complete a “Human-Intelligence-Task” (HIT)- say, a market research survey – and get paid real money (can confirm. Not Monopoly money). MTurk and other companies like it have skyrocketed in popularity among researchers (from marketing to social sciences and beyond) and for good reason. They are a LOOOTTTT cheaper than other full service market research companies (only a few hundred dollars for a survey versus thousands from other agencies); they have access to a massive number of participants from different demographics; and they are quick and easy (making it a perfect tool for students/university staff without a huge research budget).
However as with many good things, there are some problems.
The biggest issue people have with MTurk is the data sample. People are (very legitimately in my opinion) asking
Who it is that sits around and fills out a whole survey for 50c?
The implication being that they must be real weirdos and not a very accurate representation of the general public.
As well as them being ‘super odd ‘people who enjoy answering questions for very little money the diversity of demographics may not be as great as implied. Most participants so far come from “Western, educated, industrialised, rich and democratic cultures” who apparently have unusual psychological behaviour in comparison to the other 88% of the world.
There is also a selection bias to contend with where the MTurk survey participants pick your survey out from a long list of potential tasks. Why? Perhaps they have a particular interest or added knowledge in your study topic and this may skew the results.
And while it is a lot cheaper that full service research firms (as mentioned earlier), the costs still do add up. It’s the ever dreaded drip-pricing. It is not just the price for each survey paid to each participant that you organise and pay. Amazon then charges a 40-45% fee on top, (20% base, 20% for batches with more than 10 HITs and another 5% if you use the ‘Masters’ – more qualified participants). So in the end each batch of HITs costs 3 to 4 times as much as your original estimate!
These are just some of the problems that may affect the data from the researchers side, but we must also take a moment to consider the problems from the subjects side which can create ethical dilemmas for researchers. When researcher Utpal Dholakia signed up to MTurk for a week to see what it was like he was shocked. After completing around 300 surveys over 25 hours Utpal found that he had trouble figuring out his hourly wage on the site. Once he did the math himself he concluded he was making approximately $3-$3.25USD per hour of work! Now we know America has a lower minimum wage than we do here in Australia – but let me assure you it is still more than $3.25 an hour!
At this hourly rate and with all the other questionable aspects to sites like Amazon Mechanical Turk – is it ethically right to be getting data from this source or do the benefits outweigh the risks?