Data Scientist

vor 1 Woche

Frankfurt am Main, Hessen, Deutschland Mercor Vollzeit 70.000 € - 120.000 € pro Jahr

Job Description: AI Task Evaluation & Statistical Analysis Specialist

Role Overview

We're seeking a data-driven analyst to conduct comprehensive failure analysis on AI agent performance across finance-sector tasks. You'll identify patterns, root causes, and systemic issues in our evaluation framework by analyzing task performance across multiple dimensions (task types, file types, criteria, etc.).

Key Responsibilities

  • Statistical Failure Analysis: Identify patterns in AI agent failures across task components (prompts, rubrics, templates, file types, tags)

  • Root Cause Analysis: Determine whether failures stem from task design, rubric clarity, file complexity, or agent limitations

  • Dimension Analysis: Analyze performance variations across finance sub-domains, file types, and task categories

  • Reporting & Visualization: Create dashboards and reports highlighting failure clusters, edge cases, and improvement opportunities

  • Quality Framework: Recommend improvements to task design, rubric structure, and evaluation criteria based on statistical findings

  • Stakeholder Communication: Present insights to data labeling experts and technical teams

Required Qualifications

  • Statistical Expertise: Strong foundation in statistical analysis, hypothesis testing, and pattern recognition

  • Programming: Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analysis

  • Data Analysis: Experience with exploratory data analysis and creating actionable insights from complex datasets

  • AI/ML Familiarity: Understanding of LLM evaluation methods and quality metrics

  • Tools: Comfortable working with Excel, data visualization tools (Tableau/Looker), and SQL

Preferred Qualifications

  • Experience with AI/ML model evaluation or quality assurance

  • Background in finance or willingness to learn finance domain concepts

  • Experience with multi-dimensional failure analysis

  • Familiarity with benchmark datasets and evaluation frameworks

  • 2-4 years of relevant experience

  • Data Scientist

    Vor 6 Tagen

    Frankfurt am Main, Hessen, Deutschland IQVIA Vollzeit

    Data ScientistEMEA Data Science & Advanced AnalyticsFrankfurt am Main, Deutschland (Hybrid)Wir suchen eine hochmotivierte und neugierige Persönlichkeit als Data Scientist zur Verstärkung unseres Teams. In dieser Rolle entwickeln Sie neue Produkte, verbessern bestehende Lösungen und unterstützen innovative Projekte. Sie arbeiten mit...

  • Data Scientist

    vor 1 Woche

    Frankfurt am Main, Hessen, Deutschland Deutsche Börse Group Vollzeit 500.000 € - 1.000.000 € pro Jahr

    Your area of workEurex Clearing faces various types of risk and has comprehensive risk management policies and services in place to promote integrity, transparency, efficiency, and the safety of financial markets. The CCP risk management department is responsible to define these policies and effectively operate related procedures.The Risk Design team within...

  • Data Scientist

    vor 1 Woche

    Frankfurt am Main, Hessen, Deutschland Deutsche Börse Group Vollzeit 90.000 € - 120.000 € pro Jahr

    Your career at Deutsche Börse GroupYour area of workEurex Clearing faces various types of risk and has comprehensive risk management policies and services in place to promote integrity, transparency, efficiency, and the safety of financial markets. The CCP risk management department is responsible to define these policies and effectively operate related...

  • Senior Associate Data Scientist

    Vor 4 Tagen

    Frankfurt am Main, Hessen, Deutschland Publicis Sapient Vollzeit

    Job DescriptionSenior Associate Data Scientist L1As a Senior Associate Data Scientist, you will be responsible for developing and implementing AI & Machine Learning solutions to drive business insights and innovation. You will work on complex data sets, build predictive models, and collaborate with cross-functional teams to deliver impactful data-driven...

  • Senior Data Scientist

    Vor 2 Tagen

    Frankfurt am Main, Hessen, Deutschland DZ BANK AG Vollzeit

    Das erwartet SieAls integraler Bestandteil unseres Spitzenteams treiben Sie die strategische Weiterentwicklung und Skalierung unserer internen Plattform voran und gestalten maßgeblich die Bereitstellung innovativer UI-Applikationen für das Kapitalmarktumfeld mit.Neben der Mitarbeit beim Design sowie der Fullstack-Entwicklung von neuen Features...

  • Data Scientist

    vor 2 Wochen

    Frankfurt am Main, Hessen, Deutschland Deutsche Börse Vollzeit 90.000 € - 120.000 € pro Jahr

    Build the future of financial markets. Build yours.Ready to make a real impact in the financial industry? At Deutsche Börse Group, we'll empower you to grow your career in a supportive and inclusive environment. With our unique business model, driven by 15,000 colleagues around the globe, we actively shape the future of financial markets. Join our One...

  • Data Scientist

    vor 1 Woche

    Hofheim am Taunus, Hessen, Deutschland Cognite Vollzeit 60.000 € - 120.000 € pro Jahr

    About CogniteCognite is gearing up for an exciting year of growth in 2026. As we expand our global footprint and scale our impact across industries, we're building our team with talented, forward-thinking individuals who want to make a difference. Even if you don't see a role that matches your experience today, we encourage you to apply and join our talent...

  • Data Scientist German Speakers

    vor 1 Woche

    Frankfurt am Main, Hessen, Deutschland Global Market Solutions Germany Vollzeit 60.000 € - 120.000 € pro Jahr

    GMS is an operational consultancy with expertise in data and digital systems. Whether digitalisation, data & analytics or compliance - our consultants are the missing piece of our customer's digital puzzle. We empower their ongoing digital transformation and transform challenges into opportunities. We're part of the MoOngy Group, a more than 18 years old...

  • Data Scientist

    Vor 4 Tagen

    Frankfurt am Main, Hessen, Deutschland Honda Bank GmbH Vollzeit

    'There is only one future ahead of us, and that is the one that consists of our dreams – provided we have the courage to break with convention.' - Soichiro HondaOur philosophy is based on equality, respect and the initiative to implement creative ideas. We want to pass on the joy we get from creating and selling our products to our customers through our...

  • Data Scientist

    Vor 4 Tagen

    Frankfurt am Main, Hessen, Deutschland Honda Bank GmbH Vollzeit

    'There is only one future ahead of us, and that is the one that consists of our dreams – provided we have the courage to break with convention.' - Soichiro HondaOur philosophy is based on equality, respect and the initiative to implement creative ideas. We want to pass on the joy we get from creating and selling our products to our customers through our...