Senior Data Engineer

vor 4 Wochen

Berlin, Deutschland CloudFactory Vollzeit

As a global leader in combining people and technology to help companies realise the disruptive potential of AI, CloudFactory is seeking highly skilled and experienced Senior Data Engineers to join our dynamic team to help mature our data platform.

You will have the opportunity to work with cutting-edge technologies and collaborate with a diverse and talented team to scale our data processing capabilities, whilst creating economic opportunities for talented individuals in developing nations.

Whilst we offer remote work flexibility, we expect this role to be office-based two days a week.

Responsibilities

The successful candidate will translate user requirements into actionable insights by leveraging existing tools and technologies. Your responsibilities will include: 

  • Regular interaction with PMs/stakeholders to understand their requirements
  • Investigation to ascertain if the requirements can be met using existing technology and data resources
  • Data extraction using Python and other tools like Fivetran
  • Data modelling using SQL and the concept of Star and Snowflake schema
  • Assisting in cost and performance optimization choices within the data warehouse.  Communicating reasoning and logic behind opinions to help make optimal choices
  • Write data quality and other metadata collection systems within data pipelines to promote observability and recoverability of pipelines
  • Write unit and integration tests to bolster system reliability
  • Proactively identify the areas of improvement, propose solutions and manage technical debt
  • Mentoring, guiding and line management of three/four team members.
Tech Stack

Our tech stack:

  • Data extraction and orchestration tools - Fivetran, Prefect
  • Data modelling tool - DBT
  • Data Warehouse - Snowflake
  • Data quality cheeks - dbt, Elementary
  • Data visualisation - Tableau and Quicksight
  • Data catalogue and lineage - Atlan, dbt
  • Source code management - Github
  • Infrastructure as Code - Terraform, Cloudformation
  • Programming Language - Python
  • AWS services (ECS, S3)

Interview Process

Candidates will be expected to complete an initial coding and analytical thinking test, followed by an interview with Data Architect and Senior Product Manager where they will be expected to talk through previous projects, including approaches taken and results achieved. 

Successful candidates will then meet with the Engineering Director before a final interview with the CTO.

The process is expected to take 1-2 weeks.

Requirements

The ideal candidate will possess:

  • Proficiency in requirement capture and technical design
  • Mastery in Data pipeline development
    • Proficiency with extracting data from source systems using Python
    • Mastery in transforming data using SQL and data transformation tools
    • Mastery of data modelling concepts and practices, preferably with Star schema
  • Proficiency in writing data quality tests
  • Proficiency in writing end-to-end tests using test scenarios approach including test data creation and management
  • Proficiency in data pipeline observability techniques (e.g. monitoring, alerting, anomaly detection etc)
  • Some experience in building reports and dashboards
  • Data Warehouse administration
    • Proficiency with Data Warehouse administration (e.g. roles, permissions, compute, performance)
  • Data security and Governance
    • Proficiency in Data security and Governance (e.g. RBAC, encryption, masking, data classification and tagging, time travel and data recovery) practices
    • Some knowledge of using data catalogue and data lineage tools preferably Atlan and dbt documentation
  • Performance and cost optimization
    • Proficiency in performance and cost optimization of database queries using tools and techniques
  • Proficiency with line management to manage a team of Data Engineers and Data Analysts/Analytics Engineers
  • Excellent communication skills, able to translate complex data into understandable insights and narratives.

Benefits

Location dependent, but can include

  • Office snacks provided Monday-Friday, lunches
  • Home work and allowance
  • Medical
  • Amazing Company Mission and Culture
  • Growth Opportunities


Join us, and change the world for the better. If you are skilled and humble, with a commitment to lifelong learning, and you’re curious about the world and its people, you could be a good fit at CloudFactory. We welcome the unique contributions you can bring to help us build a diverse, inclusive workplace because we connect, learn, and grow stronger from our differences. We want you to bring your whole, authentic self to work.

We look forward to hearing from you

Still unsure? Read ‘5 Reasons You Should Work at CloudFactory’.

  • (Senior) Data Engineer Data

    vor 3 Wochen

    Berlin, Deutschland We are looking for a (Senior) DataCloud Engineer (fmd) Analytics & A Vollzeit

    What you are responsible for You team-up with our scientists, business and infrastructure experts to design data solutions that have global impact and scale to all E.ON countries You build and maintain efficient and scalable data pipelines that ensure data quality, accuracy, and reliability You stay up to date with emerging technologies and industry...

  • Senior Data Engineer

    vor 2 Monaten

    Berlin, Deutschland Digital Waffle Vollzeit

    Digital Waffle is proud to have partnered with an innovative tech company in Berlin who are on a mission to revolutionise the energy sector with data-driven insights maximising the efficiency and impact of renewable energy solutions About the Role:I am seeking a passionate Senior Data Engineer to play a pivotal role in our client's mission to combat climate...

  • Senior Data Engineer

    vor 3 Wochen

    Berlin, Deutschland Digital Waffle Vollzeit

    Digital Waffle is proud to have partnered with an innovative tech company in Berlin who are on a mission to revolutionise the energy sector with data-driven insights maximising the efficiency and impact of renewable energy solutions About the Role:I am seeking a passionate Senior Data Engineer to play a pivotal role in our client's mission to combat climate...

  • Senior Data Engineer

    vor 2 Stunden

    Berlin, Deutschland AI Futures Vollzeit

    AI Futures have been engaged by one of the most exciting AI companies in Europe in their search for a Senior Data Engineer. This is an opportunity for candidates to move into a tech-driven company, at the forefront of their field, and work to deliver cutting edge and robust data solutions with a real-world impact.You will work on various projects across the...

  • Senior Data Engineer

    vor 2 Monaten

    Berlin, Deutschland Materna Vollzeit

    Advanced Smart Services sind die Zukunft der digitalen Gesellschaft. Moderne Datenökonomien gelten als die wichtigste Basis, um großen Herausforderungen wie Gesundheitsvorsorge, Nachhaltigkeit im Sinne von Energie- und Ressourceneffizienz, Klimawandel sowie Umweltschutz gewachsen zu sein. Daher arbeiten unsere interdisziplinären Teams tagtäglich...

  • Cloud Data Engineer

    vor 1 Woche

    Berlin, Berlin, Deutschland flaschenpost SE Vollzeit

    Wir liefern den Wocheneinkauf innerhalb von 120 Minuten zu unseren Kund:innen. Gemeinsam mit über Kolleg:innen schaffen wir es, eine komplette Branche neu zu erfinden. Seit unserer Gründung im Jahr 2016 sind wir mittlerweile als Sofortlieferdienst für Getränke und Lebensmittel in nahezu allen Metropolregionen Deutschlands vertreten. Seit 2020 ist die...

  • Senior Data Center Engineer

    vor 1 Woche

    Berlin, Berlin, Deutschland Bundesdruckerei GmbH Vollzeit

    Senior Data Center Engineer (m/w/d)Zurück zu den Stellenanzeigen --> Senior Data Center Engineer (m/w/d) Unternehmen: Bundesdruckerei GmbH Standort: Berlin Vertragsart: Unbefristeter Vertrag Jetzt bewerben --> Gestalten Sie mit uns die digitale Zukunft Wir suchen Persönlichkeiten mit Teamgeist, die unsere Leidenschaft für den Schutz von...

  • Senior Data Engineer

    vor 1 Monat

    Berlin, Deutschland Sonova Vollzeit

    As a Senior Data Engineer, you will be crucial in ensuring we can bring together all marketing relevant data from various sources in a reliable and scalable way, by designing and developing a central marketing data platform, assuming stewardship for data quality, consistency and usability of data. Your responsibilities: Building the data pipelines on...

  • Senior Data Engineer

    vor 4 Wochen

    Berlin, Deutschland Sonova Vollzeit

    As a Senior Data Engineer, you will be crucial in ensuring we can bring together all marketing relevant data from various sources in a reliable and scalable way, by designing and developing a central marketing data platform, assuming stewardship for data quality, consistency and usability of data. Your responsibilities: Building the data pipelines on...

  • Senior Big Data Engineer

    vor 1 Monat

    Berlin, Deutschland Amadeus Vollzeit

    Job TitleSenior Big Data EngineerSummary of the roleThe Amadeus Media team is looking for a Data Engineer to join our team. You will be building a platform that models the universe of a traveler and provides seamless integrations with our partners. You will be developing the infrastructure to monitor the quality and availability of our data sources, track...

  • Senior Big Data Engineer

    vor 4 Wochen

    Berlin, Deutschland Amadeus Vollzeit

    Job TitleSenior Big Data EngineerSummary of the roleThe Amadeus Media team is looking for a Data Engineer to join our team. You will be building a platform that models the universe of a traveler and provides seamless integrations with our partners. You will be developing the infrastructure to monitor the quality and availability of our data sources, track...

  • Senior Data Engineer

    vor 2 Wochen

    Berlin, Deutschland SoundCloud Ltd Vollzeit

    SoundCloud empowers artists and fans to connect and share through music. Founded in 2007, SoundCloud is an artist-first platform empowering artists to build and grow their careers by providing them with the most progressive tools, services, and resources. With over 375 million tracks from 40 million artists, the future of music is SoundCloud. We are...

  • Senior Data Engineer

    vor 3 Wochen

    Berlin, Deutschland Novatec Consulting GmbH Vollzeit

    Senior Data Engineer (m/w/d) bei Novatec Consulting GmbH | softgarden View job here Senior Data Engineer (m/w/d) Voll- oder Teilzeit Berlin, Deutschland Hybrid Mit Berufserfahrung ...

  • Senior Data Engineer

    vor 2 Monaten

    Berlin, Deutschland Novatec Consulting GmbH Vollzeit

    Senior Data Engineer (m/w/d) bei Novatec Consulting GmbH | softgarden View job here Senior Data Engineer (m/w/d) Voll- oder Teilzeit Berlin, Deutschland Hybrid Mit Berufserfahrung ...

  • Senior Data Engineer

    vor 2 Wochen

    Berlin, Deutschland Novatec Consulting GmbH Vollzeit

    Senior Data Engineer (m/w/d) bei Novatec Consulting GmbH | softgarden View job here Senior Data Engineer (m/w/d) Voll- oder Teilzeit Berlin, Deutschland Hybrid Mit Berufserfahrung ...

  • Senior Data Engineer

    vor 1 Woche

    Berlin, Berlin, Deutschland Deutsche Bank Vollzeit

    DB Technology is a global team of tech specialists, spread across multiple trading hubs and tech centres. Our Berlin location is our most recent addition to our global network of tech centres and growing strongly. Cash Data In Cash Data domain we are working every day to help our corporate clients run their business smoothly. The clients that are using and...

  • Senior Engineer Data Infrastructure

    vor 1 Woche

    Berlin, Berlin, Deutschland ING Deutschland Vollzeit

    Senior Engineer Data Infrastructure (w/m/d) am Standort Berlin Warum zuschauen, wenn man aktiv mitgestalten kann? Dein Motto lautet "Besser geht's immer" und Dein Herz schlägt für innovative Technologien und gute Teamarbeit? Du übernimmst gerne Verantwortung und willst nicht nur Datenpipelines, sondern auch den Erfolg Deines agilen Teams...

  • Senior Data Engineer

    vor 1 Woche

    Berlin, Berlin, Deutschland Trade Republic Vollzeit

    DESTINATION FOR GREAT TALENT At Trade Republic, we are on a mission to democratize wealth. We set up millions of Europeans for wealth with fast, easy, and free access to capital markets. With over one million customers we are one of the largest savings platforms in Europe, with users holding over €6 billion on our platform. We are seen as the...

  • Senior Data Engineer

    Vor 2 Tagen

    Berlin, Deutschland DeepRec.ai Vollzeit

    Job Title: Freelance Principal Data Engineer (4-Month Contract)Location: RemoteContract Duration: 4 MonthsAbout the Role: My client are seeking a highly skilled and experienced Principal Data Engineer to join our team on a freelance basis for a 4-month contract, working 2 days a week. Join an extremely exciting scale-up that is experiencing extraordinary...

  • Senior Engineer Data Infrastructure

    vor 1 Monat

    Berlin, Deutschland ING Deutschland Vollzeit

    Senior Engineer Data Infrastructure (w/m/d) am Standort Berlin Warum zuschauen, wenn man aktiv mitgestalten kann? Dein Motto lautet „Besser geht's immer“ und Dein Herz schlägt für innovative Technologien und gute Teamarbeit? Du übernimmst gerne Verantwortung und willst nicht nur Datenpipelines, sondern auch den Erfolg Deines agilen...