Welo Data partners with technology companies to deliver datasets that are high‑quality, ethically sourced, relevant, diverse, and scalable to power their AI models. As part of the Welocalize family, WeloData draws on more than 25 years of experience working with leading global innovators and taps into a curated worldwide network of over 500,000 AI training specialists and domain experts. Our services include:

ANNOTATION & LABELLING: Transcription, summarization, and image/video classification and labeling.

LLM ENHANCEMENT: Prompt engineering, SFT, RLHF, red teaming, adversarial training, and model output ranking.

DATA COLLECTION & GENERATION: From institutional language resources to remote field audio collection.

RELEVANCE & INTENT: Culturally informed ranking, relevance assessment, and evaluations for search, ads, and LLM outputs.

Interested in joining the Welo Data team? We apply practical, hands-on AI expertise to every project, combining strong academic backgrounds with deep experience using advanced AI tools, frameworks, and best practices. Help us elevate our clients’ data at Welo Data.

Role Overview
We are looking for a Search Relevance Subject Matter Expert to join our team. In this position, you will support a major streaming platform by reviewing rater submissions in Label Studio, ensuring data quality, and recommending targeted training. You will identify recurring issues, deliver clear feedback, and help maintain consistently high performance across the team.

Key Responsibilities
• Weekly Data Analysis and Error Trend Identification
– Review rating submissions to identify repeated errors, patterns, and quality concerns.
– Prepare and deliver weekly trend reports for stakeholders.

• Quality Assurance and Guideline Compliance
– Assess and correct annotations to ensure accuracy and alignment with client guidelines.
– Work with team members to refine and clarify guidelines when needed.

• Expert Support and Feedback
– Act as the main resource for rater questions on best practices and rating methods.
– Provide actionable guidance and mentor raters to boost performance.

• Training Recommendations
– Create or suggest targeted training sessions based on error patterns and knowledge gaps.
– Partner with the internal Quality Team to develop strategies that address recurring issues.

• Collaboration and Reporting
– Communicate regularly with cross-functional teams to share performance updates, progress, and insights.
– Identify low performers and difficult cases to ensure timely support and resolution.

Qualifications
• Prior QA experience is strongly preferred.
• Strong analytical and communication skills, with the ability to interpret error trends and deliver clear, actionable insights. Able to mentor and guide larger teams both verbally and in writing.
• Localization experience and familiarity with cultural and linguistic nuances are a plus.
• Availability for 10–20 hours per week, with some overlapping time for coordination with global teams.
• Self-directed and detail-focused, capable of working independently, meeting deadlines, and maintaining high quality standards remotely.

Hours: Freelance, part-time, 10–20 hours per week
Location: Remote/Flexible