How to use Connect to recruit participants for online studies
Summary
Connect is CloudResearch’s participant recruitment and crowdsourcing platform that offers high quality data for comparably low costs.
This tutorial covers how recruiting participants for online studies works and how to use the survey and HTML templates. For the HTML template practical examples on micro-tasks and how to integrate Pavlovia studies are provided.
The tutorial also offers some tips and tricks on how to deal with cheaters on Connect but also in general when running online studies.
What is Connect and why do you want to use it?
Connect is a participant recruitment platform based in the US with participants mainly from the US but also from other English speaking countries (UK, Ireland, New Zealand and Australia). Connect is by default easy to integrate into your Qualtrics, Gorilla, Alchemer or SurveyMonkey study but with a bit more effort you can also use it for other platforms, e.g., Pavlovia. Connect therefore is at least as good as MTurk used to be but more versatile and has more options to choose from for recruitment without paying extra. In contrast to Prolific, you can use an HTML template to upload your own tasks directly to Connect, i.e. it is very easy to run micro-tasks and you can use the same platform for recruiting participants and running studies. Connect is also much cheaper than Prolific:
| Connect | Prolific | |
| Participant payment | min. $7.5/h rec. $10/h | min. $8/h rec. $12/h |
| Service fees | 25 % | 33.3 % |
Connect is great if you want to recruit participants from the US, as they are already vetted and the data quality is therefore better than on other comparable platforms (Peer et al., 2022). You will still encounter cheaters but less often, and the majority of the data will be of good quality.
This tutorial will not cover how to design a good online studies however, there are great papers and resources for that out there that are worth taking a look at. I especially recommend this paper by Jennifer M. Rodd “Moving experimental psychology online: How to obtain high quality data when we can’t see our participants” (Rodd, 2024).
How can I use Connect to recruit participants for my study?
Connect offers three different templates for your projects: 1. survey template, 2. data labelling template and 3. HTML template.
Irrespective of which template you use, you can always add instructions that participants see before accepting the project, you need to specify the number of participants and how much you will pay them. You can also target participants based on demographic variables. You can target participants based on standard variables like gender or age but Connect also offers a wide variety of information about, for example, AI use, online dating or meditation practices – that all for free!
Survey template
If you have a link to your study, for example if you are using Gorilla or Qualtrics you can just put it in the survey template and specify how many participants you want and what you want to pay them. Connect will then show the study to eligible participants until enough participants have completed it.
show full image
URL parameters
When adding a link to your study you can chose if you want to pass up to three URL parameters. These variables are passed to the link that leads to your study and can therefore be saved in the results of your study if you set that up. This way you can save the participant ID with the corresponding results.
The values for these parameters are provided by Connect and identify the participant (Participant Id), the project (project Id) and an ID given to participants when they start the project that is different from their participant ID (Assignment Id). You can change the names of theses parameters which will affect how they appear in the URL, and you study results file. However, you cannot change the variables themselves.
Completion code
Connect will automatically generate a random completion code for every survey template. You can choose if you want participants to be redirected to Connect after the study or if they should enter the completion code themselves. If you redirect participants, you will need to set that up at the end of your study and copy the link that includes the completion code from the Connect template. Otherwise, you need to show participants the code at the end of your study, and they need to enter it manually on Connect. Once participants were redirected or entered the completion code they will show up as having completed the study successfully.
HTML template (and practical examples for Pavlovia studies and micro-tasks)
The custom HTML template works similarly to the survey template: you can specify instructions, set the number and payment of participants and target participants. The main difference if that you have to upload a csv file with data about your tasks where each row is a single task, and you need to add HTML/javascript code that gets your tasks to work. This makes the template very flexible but a little more difficult to use than the survey template.
show full image
Using the HTML template to run a Pavlovia/PsychoPy study with predefined conditions
This section explains how you can recruit participants on Connect for your study create with PsychoPy when you want to randomly assign predefined conditions to participants. However, this method will probably also work if your study is hosted somewhere else.
When using the survey template, the link to your study will be the same for all participants. Thus, if you have several different predefined conditions or trials that you want different participants to complete you would have to set that up in your study yourself. However, this can be quite tricky and can cause several issues. How do you keep track of which conditions were actually completed by participants? Or how do you make sure that a condition that was abandoned by one participant will be assigned again to a new participant?
To solve that issue you can use the custom HTML template and assign different links to each participant:
1. Create different trials files (trials01.xlsx, trials02.xlsx, trials03.xlsx, …) for each participant that includes all the information you need for each trial for the given participant (e.g. stimuli that will be shown in a given trial).
2. In your PsychoPy experiment add a “condition” field to the Experiment info box.
3. In the trials loop make sure that participants get assigned the trials file that corresponds to the value that was entered in the condition field. Now if you enter 01 in the condition field you will be assigned the file trials_01.xlsx.
4. Create a csv file with as many rows as you have trials files and a different link for each file. This file will be uploaded to Connect. You can pass the value for the condition field in the link to your study which means that it will be automatically entered into the field and the field will be skipped entirely. This also means that participants will be assigned a trials file based on the link they get forwarded to.
| URL |
| https://run.pavlovia.org/usernamer/your-study/?condition=01 |
| https://run.pavlovia.org/usernamer/your-study/?condition=02 |
| https://run.pavlovia.org/usernamer/your-study/?condition=03 |
| https://run.pavlovia.org/usernamer/your-study/?condition=04 |
5. Upload the csv file to Connect.
5. Now you need to make sure that participants that accept your project on Connect are forwarded to your study using a link from the csv file. This will happen in the HTML field. By default, Connect provides code that shows participants the instructions you add to the instructions box above and a submit button that participants click on to submit the study. But you can add any code you like or change the code entirely. Below I show an example of how I solved the problem, adding a custom link to the study and a completion code that enables the submit button to prevent participants from submitting without having done the study.
<html lang='en'>
<head>
<!-- Sets the character encoding of the document to UTF-8 -->
<meta charset='UTF-8'>
<!-- Includes some basic styling -->
<link href="dist/main.css" rel="stylesheet"/>
<!-- JavaScript code block starts -->
<script language="javascript" type="text/javascript">
// Defines a function named "check" that validates the input field
function check(){
// Retrieves the input element with the ID "cc" (= completion code)
const input = document.getElementById("cc");
// Stores the current value of the input field in a variable
const inputValue = input.value;
// Checks if the input value matches your completion code (here "123456789")
if (inputValue === "123456789") {
// If it matches, submits the form with the ID "ccForm"
document.getElementById("ccForm").submit();
}
}
// Defines a function to prevent form submission by pressing Enter
function prevent_enter(){
// Checks if the Enter key (key code 13) was pressed
if(event.keyCode === 13) {
// Prevents form submission
return false;
}
}
</script>
<!-- End of JavaScript code block -->
</head>
<body>
<div>
<!-- The form element where user input is collected -->
<!-- Assigns an ID "ccForm" to the form for easy reference -->
<form id="ccForm">
<!-- Heading for the instructions -->
<div class="py-3 px-4 font-bold cr-text-secondary cursor-pointer">
Instructions
</div>
<!-- Displays text given in the instructions field, with a fallback message if no text was entered -->
<div class="pb-3 px-4 ql-editor ql-display !min-h-0">
{{ task.instructions ?? 'No instructions provided.'}}
</div>
<!-- Additional instructions added manually, not in the instructions field -->
<div class="py-3 px-4 cr-text-secondary cursor-pointer">
To start the study please click on the link below. At the end of the study you will be shown the completion code. Please copy and paste the code in the field below and click submit.
Please do not close this tab while completing the study. Once you entered the completion code and clicked Submit you can close the tab.
</div>
<!-- Display text that leads to study if clicked on -->
<div class="py-3 px-4 font-bold cr-text-secondary cursor-pointer">
<!-- The link dynamically fetches a URL from "task.row_data", i.e. from the csv file you uploaded -->
<a style="border: 1px solid; padding: 5px" href="{{task.row_data['URL']}}" target="_blank">Click here to start the study</a>
</div>
<!-- Final instructions about entering the completion code -->
<div class="py-3 px-4 cr-text-secondary cursor-pointer">
<p>Once you completed the study please enter the completion code below and click on Submit. Please make sure there is no whitespace after the code.</p>
</div>
<!-- A styled input field for the completion code -->
<div style="margin-left: 18px; margin-bottom: 20px; ">
<!-- Input field with "completionCode" as its name and a prevention for the Enter key -->
<input style="border: 2px solid;" name="completionCode" type="text" id="cc" onkeydown="prevent_enter()"/>
</div>
<!-- A hidden input field that reads out another variable given in the csv file named conditionConnect that will be saved in the results -->
<p style="display: none;">
<input name="conditionConnect" type="text" value="{{task.row_data['conditionConnect']}}"/>
</p>
<!-- A styled button that calls the "check" function when clicked -->
<button class="py-3 px-4 mx-4 font-bold cr-text-secondary cursor-pointer border border-brand-primary-main dark:border-dark-100 rounded-md" type="submit" id="sbm" onclick="check(); return false;">
Submit
</button>
</form>
<!-- End of the form -->
</div>
</body>
</html>
<!-- End of the HTML document -->Double curly brackets {{ }}
Double curly brackets like this {{task.row_data['URL']}} are used to insert dynamic content into the HTML template. As each row in the csv file is a single task, the value for a specific column (URL) and the current row will be inserted. In this example the column URL in the csv file includes the different links to the study with different conditions. The first participant will be forwarded to the link in the first row of the column URL, the second to the link in the second row, …
Input fields <input>
Data is stored using <input> fields. These have many different types, e.g. text like in the example above. All input field will be saved under their respective name (here “completionCode”). In the example above “completionCode” will be the text that participants entered into the text field.
<input style="border: 2px solid;" name="completionCode" type="text" id="cc" onkeydown="prevent_enter()"/>The result looks like this
This method ensures that you will have one participant for each trial list in your csv file that successfully completed and submitted the study. If a participant starts and then abandons the task their trial list will still be assigned to a new participant once the first participant returns the study or times out. At the same time, each trial list will only be completed once.
Of course, you can add any code that you like to the HTML field, also integrating simple tasks directly.
How to run micro-tasks directly on Connect using the HTML template
You can also run micro-tasks directly on Connect using the HTML template. Since you can add any JavaScript or HTML code you can run all kinds of tasks fairly easy. Below I provide an example for a simple task where participants have to decide if the image they see is of an animal or not. The code displays a single image at a time. This image is determined by the uploaded csv file – in each row there is a link to one image. Below the image are two radio buttons, one for Yes and one for No. Participants can check only one of the boxes at a time and they have to check one in order to click on the submit button. For this example you only need a csv file with a single column called image_url and a row with a link for each image.
Even if you had thousands of images all you would need is a csv file with the links to all images and the code for the HTML template. Within the template you can specify how many tasks each participant is allowed to complete (from 1 to unlimited). Here you will pay participants for each task they complete.
<html lang="en">
<head>
<!-- Sets the character encoding of the document to UTF-8 for better compatibility -->
<meta charset="UTF-8">
<!-- Includes some basic styling but can be removed if you want to control all css -->
<link href="dist/main.css" rel="stylesheet"/>
<!-- Include AWS Crowd HTML scripts if using -->
<!-- <script src="https://assets.crowd.aws/crowd-html-elements.js"></script> -->
</head>
<body>
<div>
<!-- Replace <form> with <crowd-form> if using AWS Crowd HTML -->
<form>
<!-- Creates a drop down box for instructions -->
<details class="w-full border border-brand-primary-main dark:border-dark-100 rounded-md cr-bg-tertiary cr-text-primary">
<summary class="py-3 px-4 font-bold cr-text-secondary cursor-pointer">
Instructions
</summary>
<!-- Displays text given in the instructions filed, with a fallback message if no text was entered -->
<div class="pb-3 px-4 ql-editor ql-display !min-h-0">
{{ task.instructions ?? 'No instructions provided.'}}
</div>
</details>
<!-- Additional instructions added manually -->
<div class="py-3 px-4 cr-text-secondary text-center">
Is this an image of an animal?
</div>
<!-- Image added dynamically based on CSV file data {{ task.row_data['image_url'] }} -->
<div class="image-container my-4">
<!-- Image width is 1/5 of the container width, height is scaled accordingly -->
<img src="{{ task.row_data['image_url'] }}" alt="Task Image" class="w-1/5 max-w-lg mx-auto" />
</div>
<!-- Radio buttons for Yes/No options, saved under the name "response" -->
<!-- Only one of the buttons can be checked at a time -->
<div class="radio-buttons flex items-center justify-center space-x-4 my-4">
<label class="flex items-center">
<input type="radio" name="response" value="yes" class="mr-2" required />
Yes
</label>
<label class="flex items-center">
<input type="radio" name="response" value="no" class="mr-2" required />
No
</label>
</div>
<!-- Centered Submit Button -->
<!-- One of the Yes/No buttons have to be checked in order to be able to submit -->
<div class="flex justify-center my-4">
<button class="py-3 px-4 font-bold cr-text-secondary cursor-pointer border border-brand-primary-main dark:border-dark-100 rounded-md" type="submit" id="sbm">
Submit
</button>
</div>
</form>
</div>
</body>
</html>Double curly brackets {{ }}
Double curly brackets like this {{task.row_data['image_url']}} are used to insert dynamic content into the HTML template. As each row in the csv file corresponds to a single task, the value of a specific column (here: “image_url”) in this row will be inserted. The first participant will be assigned the first row in the file. If the image_url value in the first row is a link to an image of a donkey, they will see that. If the second row has a link to an image of a banana, the second participant will see a banana.
Input fields <input>
Data is stored using <input> fields. These have many different types, e.g. text like in the example above. All input field will be saved under their respective name (here “completionCode”). In the example above “completionCode” will be the text that participants entered into the text field.
<input type="radio" name="response" value="yes" class="mr-2" required />Displaying images
If you want to display images in your task you have to upload them, e.g. to your own website or a GitHub repository. These images can then be
accessed with a hyperlink and you can add the links to your csv file. Here the source (src) is a dynamically inserted hyperlink to an image (e.g. https://assets.malin-styrnal.de/img/donkey_10s.jpg):
<!-- Image added dynamically based on CSV file data {{ task.row_data['image_url'] }} -->
<div class="image-container my-4">
<!-- Image width is 1/5 of the container width, height is scaled accordingly -->
<img src="{{ task.row_data['image_url'] }}" alt="Task Image" class="w-1/5 max-w-lg mx-auto" />
</div>All input field will be saved under their respective name (here “response”). In the example above “response” will be Yes or No, depending on which radio button a participant checked.
This is what the task above looks like on Connect
Further reading
For more details on how to design tasks in HTML templates, take a look at Tim Bradys tutorial on MTurk (https://bradylab.ucsd.edu/ttt/), as this works similarly to the template from Connect.
How to deal with cheaters
Even though participants are checked by Connect and the data quality is overall really good you will still encounter cheaters on a regular basis. Connect allows you to reject participants and not pay them, to reject them and still pay them, and to flag them (meaning you can report them to Connect). You can also add participants to your universal block list, so they won’t be able to participate in any of your future studies (if you apply your universal block list). However, participants will not see if you flagged them or added them to your blocklist.
Different types of cheaters
During the studies I ran on Connect I noticed a few different ways in which participants try to cheat and tried different methods to detect cheaters.
1. Very fast response times, low accuracy or response patterns: Some participants obviously click through the study as fast as possible without actually doing the task. In my experience, when looking at task performance you will see many participants performing really well, some performing worse and then a few participants that perform really poorly. These participants have response times below 200 ms, accuracies at chance level, or always click on the same position, often even show a combination of these. This is the most common type of cheating and easy to detect by looking at the results. Note, that you have to look at the results of your task, not the data shown in Connect! Some participants do the task very fast but – often intentionally – take a long time to submit it, so on Connect it seems like they spend a long time doing the task.
2. Incorrect completion codes/reporting technical issues: I’ve also had participants messaging me directly and telling me that the completion code could not be submitted. This is of course a real possibility, and you might not be able to figure out if the person really could not submit. In these cases, it is a good idea to ask participants to send you the completion code directly and to tell you a few details about the task. Sometimes participants will send you the completion code directly when messaging you. Make sure to double check that this is the correct code, as some participants will send a random code to appear like they completed the study.
3. Submitting without a completion code: Participants have the possibility to submit the study without entering a completion code. This might also happen because they had technical issues with submitting the completion code but participants could also do that without completing the task. In this case, you can message them directly to ask what went wrong, tell them to send you the completion code and details of your task. You can also check if you are missing a results file.
4. Sharing completion codes: Sometimes participants seem to share completion codes with each other. This case is tricky to identify but might become apparent if participants submit the task faster than it can possibly be completed.
More tips and tricks
Participant groups
It is possible to create participant groups that you can include in or exclude from projects. If you are for example running multiple studies within the same project and want different participants for all studies, you can create a participant group where you add all participants from the studies you already ran. When creating a new project, you can then exclude the entire participant group from this project.
Scheduling projects
You can schedule projects to start at a specific date and time. You can either schedule the launch of the entire project or a pilot with a specified number of participants.
Piloting experiments
You do not have to launch the project for all participants at once, you can start with a pilot by launching the project only for a specified number of participants. Once enough participants took part you can launch the full project or add more pilot participants.
Managing your project
When clicking on your project name you get to a site where you can view all project details like the project status, activities, participants, etc.
Once your project is running you will be able to see how many participants are taking part at the moment, how many submitted, returned or timed out and if they left a comment or reported a technical issue. Here you will also be able to manage participants, message them and pay or reject them. When participants have a technical issue with the task or with submitting, they often return the project and either report an issue, which will be shown at the top of the participants list or they leave a comment, which will be visible next to their status. Thus, it makes sense to go through the participants list every once in a while and read the comments to see if there are any issues with the task.
Here, you will also be able to download a csv file with all standard demographic variables that Connect provides plus additional demographics that you can ask for when creating the project.