Scope of Work

Evaluate and compare multiple AI-generated responses to technical prompts for clarity, correctness, tone, and usefulness.
Assign analytical quality scores and compose detailed rationales justifying each assessment.
Spot responses that are technically accurate but poorly communicated, or fluent but factually incorrect.
Thoroughly document observations and findings with precision and consistency.
Identify and articulate small errors, inconsistencies, or ambiguities in technical communication.
Collaborate asynchronously with project managers and reviewers to ensure alignment on evaluation standards.
Work independently to meet agreed-upon project requirements and deliverables within defined timeframes.

Preferred Qualifications

Native-level English fluency and exceptional written communication skills.
Professional experience as a software engineer in any modern technology stack.
Proven track record authoring or reviewing technical documentation, design docs, API references, or specs.
Experience conducting code review, design review, or providing feedback on technical writing.
Demonstrated ability to judge technical communication for audience appropriateness and instructional value.
Strong analytical skills and meticulous attention to detail.
Familiarity with AI model evaluation, annotation, or RLHF projects is a plus but not required.