top of page

The reality of remote task-based work: my experience


If you've been on the socials this past year, you've likely seen ads and testimonials from people proclaiming the glory of data annotation work. What this is, in many cases, is reviewing what AI generates in terms of search engine results, answers to user questions (users being anyone who uses the internet), or creating questions that tests the AI's skill in answering to the degree required by the parameters set. (They may want a chatty answer, a brief and pithy answer, etc.) Do I like AI? No. But it's now our reality, and I need to supplement my income. I decided that my principles had to give way to the need to pay my bills.


Early in 2024, I saw these ads and as someone who has a solid command of the English language most days, a university degree and 40+ years of working experience in a variety of media, I felt confident that I could at least give this a shot. Here's what happened.


 

Round 1: DataAnnotation.tech This place looked promising. Fill out an application, pass some tests, and then get work at $20/USD an hour that you can do whenever you have spare time. PERFECT FOR ME. As you can see above, I filled out the assessment, passed the first test, and then completed the following tests. The message below has not changed since January 2024. No feedback, just crickets. From reading social media and Reddit posts written by others who applied`, this is a common experience.




Lesson learned: The application process with this company and many others is fully automated. No human will ever contact you or, most likely, ever review your work. If it doesn't pass the bot's requirements, you're dead to them. Never made a cent.


 

Round 2: Outlier.ai

I got further with Outlier, and I messed this one up totally on my own.


The position was to create detailed and challenging prompts for AI to answer, or reviewing answers to other's prompts that AI generated. I got quite far in the evaluation process and though I found the work difficult at times (generating prompts in areas I am not skilled in, like anything related to mathematics, had my neurodivergent brain in knots), I did my best. However, I ran into a bug on Outlier's site where, despite inputting 2-3 long paragraphs to fulfill the query, after submitting it, it would fail because the website somehow dropped everything but the first 1 or 2 sentences. I reported it to my supervisor, and – instead of waiting to hear what to do next – I kept trying to submit my answers, which I would type in a text editor so I wouldn't lose them again. Of course this never worked. Why did I think it would?


ADHD is cruel sometimes in what it tells you you must do. The continued fails made human reviewers flag me as too stupid to continue (and I can't blame them), and I was kicked off the program. 100% my fault. And heartbreakingly, I was paid for my work till I was let go. More than $300usd.

I wrote the Supervisor, apologized for how I'd handled things, and asked if there was anything I could do. He told me that some assignments aren't a fit and just wait; there would likely be new assignments to take on in the future.


There haven't. I expect there never will be.


Lesson learned: Wait for the supervisor to tell you what to do if you encounter a bug in the system.


 

Round 3: Company name escapes me


This one was unspeakably cruel.


After my big self-induced fail with Outlier, I was determined to take my time, be smart and make it through to paid work. And I did great at first. I passed all the tests with a 80% score or higher (some at 100%) and was confidently sailing through the process. I found the easier than the Outlier assignment, and was really enjoying it! I took my time and was careful to do exactly what was asked of me.

I got to the last section of the test and it was an EASY question. Because the testing website prevented you from pasting text into the answer box (to avoid people using AI to answer the tests for them), it was very sensitive to keystrokes. I had plenty of time left to answer the question and was in the middle of the second sentence when I hit some key that the site registered as a SUBMIT rather than a line break, and my truncated answer was sent in. Of course I failed testing and was kicked out of the program. Of course the fact that I had more than enough time to answer the question didn't matter; I had failed and they don't allow you to retake the test. Even human support couldn't do anything for me. Never made it to a stage where payment would result.


Lesson learned: Technology is great except when it isn't.


 

Round 4: RWS


This was a long and ultimately painful process that wasted probably 30 hours of my time. Totally unpaid.


I was signed on to a program evaluating search results based on a 130+ page document that was only available to view in their proprietary portal that was completely unsearchable. This in itself is beyond stupid and makes the reviewer's job so much harder than it needs to be.


The initial brief told me to expect 10-15 hours of work to get past the testing stage, but it took me much longer.


I spent hours reviewing the long document, making notes, and preparing to do a great job. I thought I understood the criteria. I passed the first two phases of their testing with a 100% score. Yes! I is smrt.


Then came phase 3. With the guidelines document open to me, and only their subject index to help me find the relevant section, I tackled the 7 search query results – each timed to allow me no more than an hour to complete it – wrote detailed comments and rated everything in two categories according to my best understanding of their rules. I needed 80% to pass.


I failed and got informed of this by an email 5 minutes after finishing the test, which took me at least 6 hours to complete.


I was entitled to write and ask for someone (a human) to review my test and suggest things I could do to pass next time, because there was only one change to retake the test. I got their answers (very brief, but I heeded them) and retook it. And within 5 minutes of finishing my retake, which took me 5-6 hours to complete, I got the second YOU FAILED email, which reminded me to delete any materials I had and forget they ever existed.

Clearly an automated system reviewed my answers both times, so they never read my thoughtful and reasoned answers. I got below 80%, so bye bye. Did I get 79%? Did I get 30%. I'll never know.


Lesson learned: When computers are in charge, humans suffer.


 

I know there are people who are successfully working for these companies. My sibling is friends with someone who's still working for Outlier and making well above the $30/usd an hour that is the starting salary for many of their tasks. Some of them got in early; some of them are smarter than I am, obviously. Some of them are clearly better suited to the types of tasks that are assigned.


Am I going to stop trying to get work with companies like these? Not yet. My full-time job as editor of Knitty takes a lot of my time, and finding additional work that I'm good at and that can fit in my free nooks and crannies has proven to be fucking hard. You can't work at Starbucks with a "I'm free for 2-3 hours on 3 days this week, but I don't know which days and they could change, so you know, can you hire me?" But that is my reality. Remote task-based work is ideal for people like me. And moms. And students. And people with part-time jobs already.


What I do know is that these companies, except Outlier, who paid for me to test and train (oh, Outlier, please let me try again on a different assignment), are taking advantage of our need for such work by offering nothing to us in exchange for the promise of maybe-work, no matter how long it takes us to train and test. No feedback so we can improve and actually get the work. And only one chance to follow their extensive guidelines so that if we blow it, then that route is closed to us for good because computer says no. Our phone number is how they identify us, like a serial number, and once you apply and fail, that phone number is not welcome back.


If you're looking for work like this, I wish you better luck than I have had so far. And I hope that what I've shared lets you know what you might be in for so you are better prepared than I was.







89 views1 comment

Recent Posts

See All

1 Comment


debd94
Sep 27, 2024

Thank you for sharing your experience. I had wondered if these jobs were real and a possibility to supplement one's income. Seems like a big disappointment. I'm sorry that you had such a frustrating experience.

Like

© 2023 Amy Singer

bottom of page