ML-QA Engineer

vor 2 Wochen

Berlin AI Campus Munich, Deutschland Beroe x nnamu Vollzeit


Your tasks

We are looking for a ML-QA Engineer with strong automation experience testing intelligent/agentic applications to join our growing engineering team. You will play a critical role in ensuring the quality and reliability of our platform by developing automated test frameworks, writing robust test cases, and collaborating closely with developers, product owners, and DevOps.

The role goes beyond traditional QA: You will be working with AI-driven, agentic applications that can plan, act, and adapt. Your job will be to make sure they behave correctly, safely, and predictably under real-world conditions.

Key Responsibilities
  • Design, develop, and maintain automated test frameworks and test scripts using Python
  • Integrate automated tests into CI/CD pipelines to ensure continuous quality and fast feedback
  • Collaborate with developers and product managers to identify, reproduce, and resolve defects early in the lifecycle
  • Define and document test plans, strategies, and acceptance criteria for both deterministic and agentic features
  • Verify observability and explainability of agent behavior (logs, traces, intermediate actions) to ensure compliance, transparency, and auditability
  • Perform scenario-based, exploratory, and chaos testing to validate agent decision-making under uncertainty, failure, and edge conditions
  • Test human-in-the-loop workflows, ensuring proper escalation, overrides, and user safety
  • Conduct root cause analysis of production issues, including agent misbehavior, and contribute to long-term fixes
  • Contribute to the development and enforcement of QA standards and best practices


Your profile

Must-Have Qualifications
  • 3+ years of experience in QA Engineering with a focus on test automation for web and distributed systems
  • Proficient in Python for developing test scripts, validation tools, and data validation utilities
  • Hands-on experience with testing frameworks such as PyTest, Selenium, or Playwright for end-to-end and UI testing.
  • Strong understanding of software testing methodologies, including unit, integration, system, and end-to-end testing.
  • Familiarity with REST APIs and tools like Postman or Swagger for API testing
  • Strong problem solving skills with the ability to think beyond "happy path" testing.
  • Understanding of data validation pipeline and data pipeline testing, ensuring integrity and reproducibility across datasets
Nice-to-Have Qualifications
  • Experience testing agentic or AI-driven applications (e.g., autonomous decision-making, multi-step workflows)
  • Experience with scenario-based or chaos testing frameworks
  • Familiarity with observability and monitoring tools (e.g., Datadog, Grafana, AWS CloudWatch) to validate agent behavior
  • Experience testing microservices or serverless architectures on AWS
  • Familiarity with CI/CD tools like GitHub Actions, Jenkins, or GitLab CI.
  • Knowledge of security and guardrail testing (role-based access, safe action validation)
  • Experience writing performance and load tests for AI interface endpoints using Locust or k6.
Qualities we ​Value:
  • Systems thinker: You see how parts connect and anticipate failure points
  • Exploratory mindset: Comfortable testing for unknown unknowns
  • Domain curiosity: Willingness to deeply understand how the agent is supposed to act in the real world
  • Ethical awareness: You care about safety, fairness, and unintended consequences in AI systems
  • Collaborative: You work closely with engineers, product, and operations to build trust in our platform


Why us?

At Beroe X nnamu GmbH, we prioritize a balanced work-life experience.

 Here's what we offer you:
  • 4-Day Work Week & 30 Days Paid Vacation – More time to recharge
  • Competitive Compensation – Fair salary and comprehensive benefits.
  •  Monthly Benefits Allowance – €40/month in vouchers for fitness and other perks with Probonio
  • Flexible Work Arrangements – Work the way that suits you. Professional Development – Access to training and certifications. 
  • Team Events – Bi-annual company events and monthly lunch get-togethers. 
  • Work Abroad Flexibility – Remote work from the EU or selected non-EU countries for up to 8 weeks a year with travel insurance coverage
We operate on a hybrid model with offices in Berlin and Munich, offering a 32-hour, 4-day workweek. 
This means: 
  • In-Office Collaboration: Work from the office two days a week
  • Manage Your Own Hours: Flexibility to work around your needs as long as team goals are met.  
Our Culture   
We are committed to fostering a collaborative, innovative and inclusive work environment where everyone's ideas matter. We know that diverse teams lead to better outcomes and welcome applicants from all backgrounds.

About us

At Beroe x nnamu GmbH, we are committed to empowering procurement teams to make informed, strategic decisions that drive real impact. By integrating AI and game theory, our Software as a Service (SaaS) platform, nnamu.negotiations, delivers a groundbreaking approach to complex, yet autonomous negotiations.    
nnamu.negotiations delivers additional total-value-of-ownership savings efficiently and effectively for buyers with no prior knowledge of game theory required. By combining our world class proprietary AI with an unmatched, unique database of over EUR 400bn actual game theory project data we help clients uncover incremental value in their negotiations, driving outcomes that were previously out of reach.   
 
Beroe x nnamu GmbH is now a Beroe, Inc. group entity. Beroe is a global leader in procurement intelligence. Beroe has been on procurement's leading edge since the company's founding in 2006, bringing a world of insights forward. The unique combination of Beroe's expertise, AI tools, and vast amounts of reliable data enable organizations to make smarter, faster, better procurement decisions. Not tomorrow, not today, but now. Selected by ProcureTech as one of the "most pioneering Analytics, Data and Intelligence solutions in 2024", Beroe helps thousands of organizations sift through the data noise, mitigate risk, face fewer surprises, and ultimately gain a competitive edge.  
With nnamu's cutting-edge AI and Beroe's trusted market intelligence, we are unlocking game-changing possibilities for procurement teams worldwide  
 The Challenge We are Solving   
Despite the proven potential of game theory to enhance negotiation outcomes significantly – potentially unlocking an incremental USD 1 Trillion of value – its application in business negotiations remains rare, inconsistent, and hard to scale. This gap presents a vast opportunity but also a challenge that many companies have yet to overcome. Beroe x nnamu GmbH is pioneering the use of AI-powered game theory to make complex negotiation strategies accessible, scalable, and incredibly effective for organizations worldwide.

  • Senior QA Engineer

    vor 1 Woche

    Berlin, Deutschland Lumenaza GmbH Vollzeit

    Das sind wirWir sind ein Green-Tech Unternehmen und gestalten die Energiebranche nachhaltiger. Bei uns hast Du die Gelegenheit, in der Welt der dezentralisierten und erneuerbaren Energien aktiv mitzuwirken. Unsere Mission ist es, die Energiewende voranzutreiben – und dies tun wir mit Leidenschaft, Technologie und Innovation.Lumenaza lebt klare Werte:...

  • Senior QA Engineer

    Vor 7 Tagen

    Berlin, Deutschland Lumenaza GmbH Vollzeit

    Das sind wirWir sind ein Green-Tech Unternehmen und gestalten die Energiebranche nachhaltiger. Bei uns hast Du die Gelegenheit, in der Welt der dezentralisierten und erneuerbaren Energien aktiv mitzuwirken. Unsere Mission ist es, die Energiewende voranzutreiben – und dies tun wir mit Leidenschaft, Technologie und Innovation.Lumenaza lebt klare Werte:...

  • AI Engineer

    vor 2 Wochen

    Berlin AI Campus, Munich, Deutschland Beroe x nnamu Vollzeit

    Your tasks We are looking for an experienced AI Engineer to join our growing AI team at Beroe X nnamu. You'll play a key role in developing intelligent, agentic AI systems using cutting-edge large language models (LLMs), multi-agent orchestration, and retrieval-augmented generation (RAG). This is a hands-on role combining software engineering, ML/NLP...

  • QA / Test Automation Engineer

    vor 3 Stunden

    Berlin, Berlin, Deutschland Vattenfall Vollzeit

    Unternehmensbeschreibung Wir bei Vattenfall arbeiten an der Fossilfreiheit und brauchen hierfür Menschen, die sich mit ganzer Kraft für unsere Kund:innen und eine nachhaltige Gesellschaft engagieren – Menschen wie Sie. Wir sind ein führendes Energieunternehmen in Europa mit rund Mitarbeitenden. Seit mehr als 100 Jahren elektrifizieren wir Industrien,...

  • Senior QA Engineer

    Vor 4 Tagen

    Munich, Bayern, Deutschland reev Vollzeit

    Your Mission Are you ready to help us electrify the mobility sector? Apply now and become part of our team as a Senior QA Engineer (all genders).Our platform connects intelligent charging infrastructure with green energy. Our goal is to make 100% renewable electricity available at every semi-public parking space. With more than 7,000 B2B customers, we...

  • ML Platform Engineer

    vor 3 Stunden

    Munich, Bayern, Deutschland lytra GmbH Vollzeit

    We're looking for anML Platform Engineerwho helps us scale our AI infrastructure and optimize our data operations.As a key technical contributor, you'll own the analytics infrastructure that powers our decision-making, build robust data pipelines that feed our AI models, and dive deep into both customer and internal data to unlock optimization opportunities....

  • QA Engineer

    vor 3 Wochen

    Munich, Deutschland Bragi Vollzeit

    Bragi was founded in 2013 and launched the world's first Truly Wireless Earphones in 2015 (yes, a whole 12 months before the Airpods...) Since then, we have evolved from a promising startup to a market-moving creator of innovative technology, opening the new category of Truly Wireless Earphones. Bragi transformed into a software enabler for consumer brands,...

  • Quality Assurance

    vor 2 Wochen

    Berlin, Deutschland Arsipa Gruppe Vollzeit

    Wir suchen einen Quality Assurance (QA) Engineer (w/m/d) für unsere wachsende Unternehmensgruppe - Remote - deutschlandweit Als Quality Assurance Engineer (w/m/d) entwickelst du unsere Teststrategien, führst selbst sowohl manuelle als auch automatisierte Tests durch, analysierst Testergebnisse und machst Vorschläge zur Verbesserung der Testprozesse Die...

  • Senior QA Engineer

    vor 1 Woche

    Munich, Deutschland reev Vollzeit

    Your Mission Are you ready to help us electrify the mobility sector? Apply now and become part of our team as a Senior QA Engineer (all genders).Our platform connects intelligent charging infrastructure with green energy. Our goal is to make 100% renewable electricity available at every semi-public parking space. With more than 7,000 B2B customers, we are...

  • Senior QA Engineer

    vor 1 Woche

    Munich, Deutschland reev Vollzeit

    Your MissionAre you ready to help us electrify the mobility sector? Apply now and become part of our team as a Senior QA Engineer (all genders).Our platform connects intelligent charging infrastructure with green energy. Our goal is to make 100% renewable electricity available at every semi-public parking space. With more than 7,000 B2B customers, we are...