complete solution
What does the SxS rating depend on?
depends on the difference in helpfulness between the two sides—the bigger the difference for the
user, the stronger the rating.
When should Much Better be used?
When the user would have a much easier time accomplishing a goal or finding info. Higher
placement of the best results, much better ordering of the results, or really good unique results
could be reasons for a much better rating. GIVE THE BETTER RATING TO THE SIDE THAT
IS MORE HELPFUL.
What should be included in SxS rating comments?
First describe the user intent for the given query.
Describe why you prefer one side over the other, or (optionally) why you think both sides are
about the same.
Describe any particular result(s) that led you to prefer either side.
What should i take into account when doing SxS ratings?
The overall ratings of both sides (does one side have higher ratings?), overall order (Is the order
of results on one side more user friendly?), number of results, resuly diversity.
What do you do if there are duplicates? (SxS)
Get rid of the duplicates and take a look at what's left.
True/ False, A rater who reads and writes Spanish is fully qualified to rate projects for all
task locations where Spanish is spoken
False
User Intent
refers to what the user was trying to accomplish by issuing the query
True / False : All queries have a locale
True
Result Block