Thank you for evaluating a crowdsourcing platform using these heuristics. Please make sure to have reserved at least 1 hour in your agenda to conduct the evaluation. Evaluations can take up to 3 hours depending on how familiar you are of the crowdsourcing platform itself.

In the following pages we ask you to reflect upon 6 main heuristics. The heuristics are meant as principles of good design practice. We ask you to carefully read the heuristic and after understanding its meaning look through the crowdsourcing platform for evidence that satisfy the heuristic.

The heuristics are categorized in two levels: Level 1: Each heuristic has one or more subcategories. For example, the heuristic: “Purpose” has three subcategories: clarity, visibility, and idealism. Level 2: Each subcategory has one or more items. For example, the subcategory "clarity" has one item: "Is the purpose of the platform clear?". Each item is then rated in a three-point scale: “No”: there is no evidence that the platform adheres to the heuristic; “Semi”: there is some evidence of an effort to adhere to the heuristic but this effort is not sufficient; and “Yes”: there is ample evidence that the platform fully adheres to the heuristic.

Some heuristics might not be applicable for the platform. That is why, you also have the option to rate the item with  "Not Applicable". Please explain in the evidence why the heuristics weren´t applicable. Your evaluation also contributes to how the heuristics can be interpreted, so a detailed explination helps us a lot to future polish the heuristics.

Should you have any questions feel free to contact us at: s.p.h.m.a.campo@student.tue.nl or v.j.khan@tue.nl.