Senior AI Quality Assurance Specialist

1 week ago

Muscat, Muscat, Oman beBeeQuality Full time
Job Description:

We are seeking a highly skilled AI Quality Engineer to support our National Large Language Model (LLM) Project, with a specific focus on Arabic language processing. This pivotal role involves establishing and enforcing comprehensive data quality frameworks, evaluation methodologies, and quality gates throughout the LLM development lifecycle.

You will ensure our Arabic LLM achieves world-class performance, reliability, and cultural relevance before deployment to 20,000 government employees.

  • Design and implement data quality frameworks for LLM training and evaluation.
  • Establish and enforce quality gates across all phases: data preparation, model training, evaluation, and RAG implementation.
  • Define acceptance criteria and secure stakeholder sign-offs at each quality gate.
  • Develop annotation quality metrics targeting >90% inter-annotator agreement and >95% contextual/cultural accuracy.
  • Implement QA processes for text normalization, diacritics standardization, and dialect mapping.
  • Optimize tokenization with >98% vocabulary coverage and >95% morphological accuracy.
  • Develop robust RAG (Retrieval-Augmented Generation) quality measurement frameworks.
  • Build real-time dashboards for automated quality monitoring and alerts.
  • Define and execute testing protocols for evaluation across diverse NLP tasks.
  • Establish regression testing frameworks to safeguard model quality during updates.
  • Design processes for bias detection and mitigation in training data and outputs.
  • Lead human evaluation campaigns to assess qualitative aspects of model responses.
  • Collaborate with annotation teams to ensure accurate and high-quality ground truth datasets.
  • Maintain thorough documentation on QA processes, metrics, guidelines, and acceptance criteria.
  • Participate in weekly quality governance meetings and RAG evaluation reviews.

  • AI Quality Engineer

    4 weeks ago

    Muscat, Muscat, Oman Pranatree Full time

    Direct message the job poster from Prana TreeTalent Acquisition Specialist- Data Science AI,Next-GenAI, MLE,LLM, Quantum Computing ERP solutions Product development & other IT Services. HR…Job Title: AI Quality EngineerAbout the Prana Tree LLC :Prana Tree LLC is an innovative IT consulting firm specializing in building next-generation business...

  • Quality Assurance Specialist

    4 days ago

    Muscat, Muscat, Oman beBeeSoftware Full time

    Job Title: Quality Assurance SpecialistWe are seeking a skilled Quality Assurance Specialist to join our team. The ideal candidate will be responsible for ensuring the quality of our products by performing application quality control and quality assurance tasks.Key Responsibilities:Develop and implement quality control procedures to ensure high-quality...

  • Quality Assurance Specialist

    2 days ago

    Muscat, Muscat, Oman beBeeQualityAssurance Full time

    Job Title: Quality Assurance SpecialistOur organization is seeking a qualified professional to serve as a Quality Assurance Specialist. The ideal candidate will possess excellent analytical and communication skills, with the ability to work effectively in a team environment.The primary responsibility of this role is to ensure that all software products meet...

  • Senior Quality Assurance Manager

    1 week ago

    Muscat, Muscat, Oman Mace Full time

    Join to apply for the Senior Quality Assurance Manager role at MaceJoin to apply for the Senior Quality Assurance Manager role at MaceGet AI-powered advice on this job and more exclusive features.At Mace, our purpose is to redefine the boundaries of ambition. Within our consult business we harness our unique combination of leading-edge practical construction...

  • AI Quality Engineer

    3 weeks ago

    Muscat, Muscat, Oman Elile Full time

    We are seeking a highly skilled AI Quality Engineer to join our National Large Language Model (LLM) Project. This key role will focus on establishing and implementing robust data quality frameworks, evaluation methodologies, and quality gates throughout the LLM development lifecycle. The ideal candidate will ensure our Arabic LLM meets the highest standards...

  • Quality Assurance Specialist

    1 week ago

    Muscat, Muscat, Oman beBeeQualityAssurance Full time

    Job Description:">We are seeking a skilled Quality Assurance Specialist to join our team. The successful candidate will be responsible for ensuring the quality of our projects and maintaining the highest standards of excellence.">About the Role:">This is an exciting opportunity for a professional with a strong background in quality assurance to make a real...

  • Quality Assurance Expert

    4 days ago

    Muscat, Muscat, Oman beBeeQuality Full time

    Quality Assurance Specialist">The Quality Assurance Specialist supports the implementation of quality assurance procedures to meet brand standards, customer satisfaction, and operational efficiency. The role focuses on supporting property-level quality initiatives and contributing to continuous improvement efforts under the guidance of senior...

  • Electronics Quality Assurance

    1 week ago

    Muscat, Muscat, Oman beBeeElectronic Full time

    Job Title: Electronic Quality Assurance SpecialistWe are seeking an Electronic Quality Assurance Specialist to ensure the highest quality standards in our electronic circuits, boards, chips, and other components used in lighting products and electrical items.The ideal candidate will be responsible for inspecting and analyzing electronic circuits, boards,...

  • Senior Quality Assurance Professional

    1 week ago

    Muscat, Muscat, Oman beBeeQuality Full time

    The RoleWe are seeking a Senior Quality Assurance Manager to join our team. As a key member of our organization, you will be responsible for leading the quality assurance process and ensuring that our projects meet the highest standards.Your primary responsibility will be to lead the Assurances team, comprising program and Mace employees. You will be...

  • Senior Software Quality Assurance Engineer

    1 week ago

    Muscat, Muscat, Oman beBeeQuality Full time

    Job Title: Senior Software Quality Assurance EngineerWe are seeking a skilled Sr. Software QA Engineer to join our team and play a key role in ensuring the highest quality of our products.The ideal candidate will have extensive experience with software testing, automation, and quality assurance methodologies. A strong background in Linux and open source...