Data Scientist

Vor 2 Tagen

Berlin, Berlin, Deutschland MERCOR Vollzeit 60.000 € - 120.000 € pro Jahr

Job Description: AI Task Evaluation & Statistical Analysis Specialist

Role Overview

We're seeking a data-driven analyst to conduct comprehensive failure analysis on AI agent performance across finance-sector tasks. You'll identify patterns, root causes, and systemic issues in our evaluation framework by analyzing task performance across multiple dimensions (task types, file types, criteria, etc.).

Key Responsibilities

  • Statistical Failure Analysis: Identify patterns in AI agent failures across task components (prompts, rubrics, templates, file types, tags)
  • Root Cause Analysis: Determine whether failures stem from task design, rubric clarity, file complexity, or agent limitations
  • Dimension Analysis: Analyze performance variations across finance sub-domains, file types, and task categories
  • Reporting & Visualization: Create dashboards and reports highlighting failure clusters, edge cases, and improvement opportunities
  • Quality Framework: Recommend improvements to task design, rubric structure, and evaluation criteria based on statistical findings
  • Stakeholder Communication: Present insights to data labeling experts and technical teams

Required Qualifications

  • Statistical Expertise: Strong foundation in statistical analysis, hypothesis testing, and pattern recognition
  • Programming: Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analysis
  • Data Analysis: Experience with exploratory data analysis and creating actionable insights from complex datasets
  • AI/ML Familiarity: Understanding of LLM evaluation methods and quality metrics
  • Tools: Comfortable working with Excel, data visualization tools (Tableau/Looker), and SQL

Preferred Qualifications

  • Experience with AI/ML model evaluation or quality assurance
  • Background in finance or willingness to learn finance domain concepts
  • Experience with multi-dimensional failure analysis
  • Familiarity with benchmark datasets and evaluation frameworks
  • 2-4 years of relevant experience

  • Lead Data Scientist

    vor 1 Woche

    Berlin, Berlin, Deutschland Vividlyn Vollzeit 80.000 € - 120.000 € pro Jahr

    Job Title:Lead Data ScientistLocation:Germany (Hybrid / On-site)Employment Type:Full-timeOverviewWe are seeking an experiencedLead Data Scientistto guide our data and analytics initiatives in Ireland. The role involves leading a team of data professionals, designing advanced analytical models, and providing insights that drive strategic business...

  • Data Scientist

    Vor 2 Tagen

    Berlin, Berlin, Deutschland HelloFresh Vollzeit

    The roleAs a Data Scientist in our Global Consumer Alliance, you will be at the forefront of creating the right ecosystem at HelloFresh so that we are truly customer centric in everything we do. Our Global Customer Experience (GCX) team's mission is to understand our customers' journeys and empower market leads with insights and intelligent solutions that...

  • Data Scientist

    vor 1 Woche

    Berlin, Berlin, Deutschland Vodafone Vollzeit 40.000 € - 60.000 € pro Jahr

    Data Scientist (m/w/d)Stellen-ID: 263639Bei Vodafone arbeiten wir jeden Tag an einer besseren Zukunft. Für eine Welt, die besser vernetzt, inklusiver und nachhaltiger ist. Denn für uns ist Technologie nur so stark wie die Menschen, die sie nutzen. Sei dabei und lass uns gemeinsam die Welt von morgen gestalten.  Was Dich erwartet:Du entwickelst und...

  • Data Scientist

    Vor 2 Tagen

    Berlin, Berlin, Deutschland Vodafone Vollzeit 80.000 € - 120.000 € pro Jahr

    Data Scientist (m/w/d)Stellen-ID: 263639Bei Vodafone arbeiten wir jeden Tag an einer besseren Zukunft. Für eine Welt, die besser vernetzt, inklusiver und nachhaltiger ist. Denn für uns ist Technologie nur so stark wie die Menschen, die sie nutzen. Sei dabei und lass uns gemeinsam die Welt von morgen gestalten.  Was Dich erwartet:Du entwickelst und...

  • Data Scientist

    Vor 7 Tagen

    Berlin, Berlin, Deutschland Arsenault Vollzeit 60.000 € - 120.000 € pro Jahr

    The ideal candidate's favorite words are learning, data, scale, and agility. You will leverage your strong collaboration skills and ability to extract valuable insights from highly complex data sets to ask the right questions and find the right answers. Data Scientist Review academic and industry literature for applicable knowledge Educate others on...

  • Data Scientist

    vor 1 Woche

    Berlin, Berlin, Deutschland 1KOMMA5˚ Vollzeit 60.000 € - 120.000 € pro Jahr

    1KOMMA5°We are looking for you as an addition to our tech-team in Berlin, Munich or Hamburg. 1KOMMA5° is building Germany's largest one-stop-shop for sale, installation and services related to solar, heat pumps, electricity and charging infrastructure. And they are all connected Be a part of our mission Become a part of our mission and learn about our...

  • Staff Data Scientist

    vor 1 Woche

    Berlin, Berlin, Deutschland Delivery Hero Vollzeit 80.000 € - 120.000 € pro Jahr

    As the world's pioneering local delivery platform, our mission is to deliver an amazing experience, fast, easy, and to your door. We operate in over 70+ countries worldwide, powered by tech, designed by people. As one of Europe's largest tech platforms, headquartered in Berlin, Germany. Delivery Hero has been listed on the Frankfurt Stock Exchange since 2017...

  • Data Scientist

    vor 1 Woche

    Berlin, Berlin, Deutschland Prior Labs Vollzeit 60.000 € - 120.000 € pro Jahr

    Join Prior LabsWho We Are: Prior Labs is building breakthrough foundation models that understand spreadsheets and databases - the backbone of science and business. Foundation models have transformed text and images, but structured data has remained largely untouched. We're tackling this $100B+ opportunity to revolutionize how we approach scientific...

  • Data Scientist

    Vor 2 Tagen

    Berlin, Berlin, Deutschland Delivery Hero Vollzeit 80.000 € - 120.000 € pro Jahr

    As the world's pioneering local delivery platform, our mission is to deliver an amazing experience, fast, easy, and to your door. We operate in over 70+ countries worldwide, powered by tech, designed by people. As one of Europe's largest tech platforms, headquartered in Berlin, Germany. Delivery Hero has been listed on the Frankfurt Stock Exchange since 2017...

  • Senior Data Scientist

    Vor 2 Tagen

    Berlin, Berlin, Deutschland HeroSoftware GmbH Vollzeit 80.000 € - 120.000 € pro Jahr

    Bist du ein Ass in Datenarchitektur und Datenmodellierung? Dann suchen wir genau dich als Senior Data Scientist (m/w/d) (Vollzeit). Wir entwickeln Shopify Apps und brauchen dich, um daraus Strukturen, Insights und Entscheidungsgrundlagen zu schaffen.AufgabenDein Daily Business bei uns:Aufbau & Pflege einer skalierbaren Data-Warehouse-ArchitekturIntegration...