AI Quality Assurance Expert

2 days ago

Muscat, Muscat, Oman beBeeQuality Full time 90,000 - 120,000
AI Quality Assurance Engineer Position

We are seeking a highly skilled AI Quality Assurance Engineer to support our National Large Language Model (LLM) project, with a specific focus on Arabic language processing.

Key Responsibilities:
  • Establish comprehensive data quality frameworks for LLM training and evaluation.
  • Enforce quality gates across all phases: data preparation, model training, evaluation, and RAG implementation.
  • Define acceptance criteria and secure stakeholder sign-offs at each quality gate.
  • Develop annotation quality metrics targeting >90% inter-annotator agreement and >95% contextual/cultural accuracy.
Requirements:
  • Bachelor's or Master's degree in Computer Science, AI, Machine Learning, or related field.
  • 4+ years of experience in AI/ML quality assurance, with an emphasis on NLP and LLMs.
  • Proven experience with LLM evaluation frameworks and benchmarking techniques.
  • Expertise in setting quality gates and defining acceptance criteria for AI systems.
  • Strong understanding of NLP challenges and Semitic language intricacies.
  • Experience with multi-level annotation reviews and metrics-driven QA.
Preferred Qualifications:
  • Experience in government or enterprise-scale LLM evaluations.
  • Knowledge of NLP benchmarks and datasets.
  • Experience using platforms like Scale AI, Human loop, or similar tools.
What We Offer:
  • Opportunity to contribute to a nationally significant and socially impactful AI project.
  • Competitive compensation package.
  • Collaboration with leading researchers and engineers in LLM development.
  • A fast-paced, innovation-driven environment that values ownership and growth.

  • AI Quality Engineer

    3 days ago

    Muscat, Muscat, Oman Pranatree Full time

    Direct message the job poster from Prana TreeTalent Acquisition Specialist- Data Science AI,Next-GenAI, MLE,LLM, Quantum Computing ERP solutions Product development & other IT Services. HR…Job Title: AI Quality EngineerAbout the Prana Tree LLC :Prana Tree LLC is an innovative IT consulting firm specializing in building next-generation business...

  • AI Quality Engineer

    5 days ago

    Muscat, Muscat, Oman Elile Full time

    We are seeking a highly skilled AI Quality Engineer to join our National Large Language Model (LLM) Project. This key role will focus on establishing and implementing robust data quality frameworks, evaluation methodologies, and quality gates throughout the LLM development lifecycle. The ideal candidate will ensure our Arabic LLM meets the highest standards...

  • Arabic LLM Quality Assurance Specialist

    4 days ago

    Muscat, Muscat, Oman beBeeQuality Full time 60,000 - 90,000

    We are seeking a highly skilled AI Quality Engineer to join our project. This key role will focus on establishing and implementing robust data quality frameworks, evaluation methodologies, and quality gates throughout the LLM development lifecycle. The ideal candidate will ensure our Arabic LLM meets the highest standards of performance, reliability, and...

  • AI Expert and Consultant

    3 weeks ago

    Muscat, Muscat, Oman Elile Full time

    We are seeking an experienced AI Expert and Consultant to join our National Large Language Model (LLM) Project, replacing ChatGPT usage in the workplace. As a key technical advisor, you will provide expertise across the full LLM stack, from model training and fine-tuning to deployment and RAG implementation.Key Responsibilities:Provide strategic guidance and...

  • Expert Software Quality Assurance Specialist

    5 days ago

    Muscat, Muscat, Oman beBeeCompliance Full time 90,000 - 120,000

    Senior Test SpecialistDrive and oversee the testing process for projects initiated through formal proposals. The ideal candidate will have extensive experience in software testing within the banking sector, ensuring that all deliverables meet the highest quality standards and adhere to regulatory requirements.Key Responsibilities:Test Planning &...

  • Leading Quality Assurance Professional

    1 day ago

    Muscat, Muscat, Oman beBeeSenior Full time 60,000 - 80,000

    Job Title: Senior Quality Assurance ManagerThis is a unique opportunity to lead a team of quality assurance professionals in Oman. The successful candidate will be responsible for ensuring the highest standards of quality and safety across various projects.The ideal candidate will have extensive experience in quality management, with a proven track record of...

  • Enhance Hotel Performance as a Seasoned Training and Quality Assurance Expert

    5 days ago

    Muscat, Muscat, Oman beBeeProfessional Full time 42,765 - 54,977

    Training and Quality Assurance ManagerThe hotel's pre-opening team is seeking a seasoned professional to lead training and quality assurance efforts. As the Learning & Quality Manager, you will design, deliver, and monitor training programs across all departments, ensuring consistent service standards and operational excellence.Key Responsibilities:Develop...

  • Senior Quality Assurance Manager

    4 weeks ago

    Muscat, Muscat, Oman Mace Group Full time

    Position status: This opportunity is for a bid, so it is not yet a secured role. If you would like to be considered as we progress with this position, please click to apply.Middle East experience.Educated to degree level (or equivalent).At Mace, our purpose is to redefine the boundaries of ambition. Within our consult business we harness our unique...

  • Lead Software Quality Assurance Specialist

    2 weeks ago

    Muscat, Muscat, Oman beBeeSenior Full time 100,000 - 150,000

    Job Title: Senior Test EngineerThe Senior Test Engineer will oversee and coordinate the testing lifecycle of projects initiated through formal proposals within the banking sector.Key ResponsibilitiesTest Planning & Strategy:Develop comprehensive test plans and strategies to ensure alignment with project objectives and timelines.Collaborate with project...

  • Experienced Quality Assurance and Control Specialist Required

    5 days ago

    Muscat, Muscat, Oman beBeeQuality Full time 120,000 - 150,000

    Job Description:Muscat, OmanOur client is seeking an experienced Quality Assurance and Control Specialist to join their team in Muscat, Oman. The ideal candidate will be responsible for ensuring the quality of our construction projects meets the highest standards and adheres to all relevant regulations and guidelines.Key Responsibilities:Develop and...