* is a multi-paradigm scripting language for the Java Platform
* easy to learn with Perl and Objective-C inspired syntax
* executes scripts fast with a small package size (~250KB)
* excels at data manipulation, component integration, and distributed communication
* seamlessly uses Java objects and 3rd party libraries
I think it's great he's quickly turned around a site that lowers the barrier to a specific problem. I'd just worry about the demographic makeup of the respondents.
While Turkers aren't all geeks getting good sensible feedback is a skill in itself. I would worry that most Turkers a) aren't you customers b) aren't demographically vetted.
That said getting independent opinions is surely great food for thought and that's a powerful tool.
I was worried about this too. I honestly had no idea what to expect when I first built the software. Then I ran it against my own sites and I started seeing very thoughtful feedback. This is when I felt I had something.
I also posed questions directly to the Turk community about what I could do to make the process better and get results quicker. They spoke, I did what I could, and I'm really proud of the results my customers are getting. I'm not just saying that either.
I'm a researcher by trade and have absolute sales call reluctance. I won't sell something if I don't feel it is worth the persons time to hear me or money to invest into the product.
While this little project may only go so far, I'm not afraid to put my name on it and say I'm giving people something of value for what I'm asking.
I'm a researcher by trade and have absolute sales call reluctance. I won't sell something if I don't feel it is worth the persons time to hear me or money to invest into the product.
Are you planning to introduce anything to help select the right Turkers to comment on a project? That would be worth having.
Heck it might even be worth having as a standalone "selecting applicants for usability testing" application.
He's using Mechanical Turk's API to create a "give me feedback on my site" task automatically.
That task on Mechanical Turk usually goes for $.02-.05 a task. And, if you're familiar with MechaTurk, it really only takes about 15 minutes to set that task up.
He's charging $7 because people don't want to mess with Mechanical Turk. It's brilliant, and I hope it takes off. I actually think that he should charge $20-30 and give people 40-50 user comments. It seems like a better value.
Typical tasks may go for $.02-$.05. I pay the workers much more than that. I experimented with different levels of payment prior to launch and had to offer more than I expected but my customers are receiving quality results, very quickly. To make this happen required paying up. My goal is to be a requestor the Turkers want to work for. This community outreach is a whole other side of what I'm doing that people don't see on the front end.
Have you posted to Turker Nation at all? It might be a good idea to start developing a "brand" for your questions so that you start to have people actively seek you out and prioritize your HITS. As it is, the Amazon search interface lists 1 hit available per website which is getting FeedbackArmied and it might not be obvious to the Turkers that requests for feedback on A, B, C, D, and E are linked so if you had good experience (had an enjoyable task and got paid promptly) on A and C then B, D, and E are right up your alley.
This service is a great idea! It's simple and seems to be well implemented. I liked how you ate your own dog food, by using Mechanical Turk to decide if it's worthwhile putting up the service.
Personally, I don't understand the need for this. If I want feedback on a website, I can simply ask friends, family or Hacker News. Friends and family usually provide honest answers while HN tends to be very critical, both useful feedback. The best part: it's all free.
If it's a product that already has a bit of traffic, I could just use a service such as UserVoice to take care of feedback/suggestions by using their little Feedback tab script. Posting about the site in a blog or a forum can also help in the search for getting feedback.
Basically what I'm getting at is.. why would I pay $7 for Feedback Army when there are so many free alternatives?
I can get feedback from my website from a group of programmers any time I want to. They pretty much invariably say "Bah, who in their right mind would pay for that? I could reproduce its output in Excel in, like, five minutes." Thirty thousand dollars later my answer is "Gah, my customers pay for it. Could you concentrate on the things which are actually important here? Mistakes, usability issues, things that will interrupt conversions, etc?" and then they say "Oh, well, the page doesn't validate. Maybe it could use an RSS feed."
sigh I spend time answering emails describing the difference between "web site" and "application". No, my customers would not benefit from an RSS feed, thank you.
Asking my mother has its own problems: "It's wonderful, dear. It would be even more wonderful in teal."
Its at that point where I think "If I were hiring myself as a consultant to read this feedback and search it for actionable insights I would bill myself $100 and report 'Sorry, there is nothing useful here'." Which makes paying $7 and giving a somewhat more diverse set of people micro-tasks pretty darn appealing. I can give them actual tasks, they will perform them because I am paying for completion, and I can analyze the results.
The initial customers of Feedback Army seem to be folks with niche websites. The kind of stuff HN would probably flag if it were posted here. Also some folks aren't as plugged into an online community and may be uncomfortable posting there. I know I'm this way. I don't blog publicly and HN is the only board I interact on. So I'd use something like Feedback Army over risking my standing on a forum such as this for something off topic. That said, the feedback is paid for and there is nothing stopping someone from going back for multiple rounds in a few days on the same site. I think the free feedback givers might burn out a little quicker.
I guess that makes sense.. I wasn't thinking from the perspective of a non-developer or someone not as well connected, in which case I could see them wanting a generic service that offers feedback.
After actually using the site myself, I must admit I am absolutely floored. It was well worth my $7 and I received great feedback within the first 10-15 minutes. I guess you really can't judge a book by it's cover.
Hi there, I address this on the site FAQ. I'm delivering value by making the process to post a request simple and making the results easy to get via an RSS feed. I also do this for a low margin. If someone wants to go to Amazon directly and pay workers what I pay, depending on how they value their time, they will be losing money.
Because Mechanical Turk only allows U.S. residents to set tasks (although the rest of the world can work on them). Feedback Army seems to get around that inconvenience.
This is really interesting. Is there a way to choose why type of users review the site? Say my website is aimed at 15-30 year olds, then is there a way to do that?
Not yet. You're not the first to ask for this feature. The demographics of the reviewers are slightly more women then men, mostly 20-30 year olds, and over half have college degrees.
The value proposition is ease of use. Feedback Army manages the whole process and makes results available via RSS. For someone who has never used Mechanical Turk, Feedback Army is a low risk way to try things out. I pay the Turkers pretty well to make sure users get good feedback quickly. The overall package is well worth the $7 asking price. -- Raphael, FeedbackArmy developer
http://sleep.dashnine.org/