complete solution
What does the SxS rating depend on?
depends on the difference in helpfulness between the two sides—the bigger the
difference for the user, the stronger the rating.
When should Much Better be used?
When the user would have a much easier time accomplishing a goal or finding info.
Higher placement of the best results, much better ordering of the results, or really good
unique results could be reasons for a much better rating. GIVE THE BETTER RATING
TO THE SIDE THAT IS MORE HELPFUL.
What should be included in SxS rating comments?
First describe the user intent for the given query.
Describe why you prefer one side over the other, or (optionally) why you think both
sides are about the same.
Describe any particular result(s) that led you to prefer either side.
What should i take into account when doing SxS ratings?
The overall ratings of both sides (does one side have higher ratings?), overall order (Is
the order of results on one side more user friendly?), number of results, resuly diversity.
What do you do if there are duplicates? (SxS)
Get rid of the duplicates and take a look at what's left.
True/ False, A rater who reads and writes Spanish is fully qualified to rate
projects for all task locations where Spanish is spoken
False
User Intent
refers to what the user was trying to accomplish by issuing the query
True / False : All queries have a locale
True
Result Block
This is an individual "block" that appears on the user's phone in response to the query.
The result block may display information in the block itself or contain links, or may do
both.
Landing Page
the page you see after you click a link in the result block.
Locale
task language and task location
Dominant Interpretation
what most users mean when they type the query. Not all queries have a dominant
interpretation.
Common Interpretation
what many or some users mean when they type a query.
Minor Interpretation
These are interpretations that few users have in mind.
User Location
This tells us where the user is located