When newcomers first encounter the suggested edits module, their task types by default are set to "Easy", which includes "Copyedit" and "Add links between articles". Right now, everyone gets a copyedit task as the first one in the feed. This is causing many more users to do "copyedit" tasks than "add links" tasks. We want it to be random whether the user gets "copyedit" or "add links" first, so that we also generate a good amount of data on how users do with the "add links" task. This data is important for helping us plan the "add a link" structured task workflow.
Description
Description
Details
Details
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | MMiller_WMF | T227728 [EPIC] Growth: Newcomer tasks 1.0 | |||
| Resolved | kostajh | T248106 Newcomer tasks: randomize initial task type |
Event Timeline
Comment Actions
Change 581865 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/GrowthExperiments@master] [WIP] Randomize task type order
Comment Actions
I tried to do this by randomizing the order in which the task types are sent to the API, because the API appears to return results ordered exactly the same as the task type list it receives. However, that only works if no topic filter is set. With a topic filter, this order isn't respected, and in fact there seems to be no rhyme or reason to the results ordering as far as I can tell (neither in terms of task type nor topic).
Comment Actions
Returning this task back to the pool because I don't understand the search backend well enough, I'll need @Tgr to help me or tell me what to do here.
Comment Actions
@Tgr -- that's right, we would want roughly equal numbers of newcomers to be getting "copyedit" as their first card and "add links" as their first card. But this does not need to be scientifically equal or even particularly close. It's just that we want good amounts of each. It doesn't need to be like every other newcomer gets one or the other. It doesn't even need to be "randomized" -- like if we alternated by day or hour for all incoming newcomers, that would suffice.
Comment Actions
Change 582034 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Newcomer tasks: Randomize list of local search queries
Comment Actions
Change 582034 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Newcomer tasks: Shuffle list of search queries
Comment Actions
Moving to PM review - no issues found; please check my notes below. I've added the task to my list of post-deployment checking.
Checked in betalabs
- (the objective for this task) a new user, who hasn't selected any topics in the intro, will be presented with a randomly selected task from Easy filter - either "Add links between articles" or "Copyedit"
- if a user selects both task types in 'Easy" or 'Medium" difficulty - the tasks presented to a user would be equally shuffled - presenting the task type in equal amount (if possible).
- if a topic is selected, the tasks types would be randomly shuffled
Comment Actions
@Tgr -- we are about to use this change in some analyses, so I want to confirm which date it went into effect. Is it April 2, as you predicted?
Comment Actions
Sorry for missing the first ping. The patch went into wmf/1.35.0-wmf.26, which was indeed deployed to Wikipedias on April 2.
Comment Actions
Change 581865 abandoned by Kosta Harlan:
[mediawiki/extensions/GrowthExperiments@master] [WIP] Randomize task type order
Reason:
Cleaning up code review backlog for GrowthExperiments. If you still plan to work on this, then please restore the change.