Data Engineer

Vor 4 Tagen

Kaiserslautern, Rheinland-Pfalz, Deutschland Janson Vollzeit 60.000 € - 120.000 € pro Jahr


Full-time

Description

Job Description:

COMPANY is seeking a Data Engineer to join a cutting-edge initiative in support of the U.S. Department of Defense (DoD) in Germany. This role plays a vital part in delivering critical engineering and technical support services, with a strong emphasis on leveraging advanced data science and engineering solutions to drive mission success. 

Position: Data Engineer (Engineer IV)

Location: Kaiserslautern, Germany

Clearance: Secret

Responsibilities include, but are not limited to, the following:

Configure Data Pipelines:

Design ETL Processes:  Extract, transform, and load (ETL) data from various sources including but not limited to DoD Advana and Army Vantage, into data warehouses or data lakes.  Ensure data is cleaned, normalized, and enriched during transformation.

Build Data Pipelines:

  • Configure automated workflows to move data between systems using tools like Apache Airflow, AWS Glue, or Azure Data Factory. 
  • Optimize pipelines for scalability and reliability. 
  • Implement Real-Time Data Processing
  • Implement streaming data pipelines using tools like Apache Kafka, Apache Flink, or Spark Streaming.
  • Process and analyze data in real-time for applications like fraud detection or IoT analytics.

Manage Data Storage:

  • Design Databases 
  • Design and implement relational databases (e.g., MySQL, PostgreSQL) or NoSQL databases (e.g., MongoDB, Cassandra) based on project requirements.
  • Ensure efficient schema design and indexing for performance.

Warehouse Data:

  • Build and maintain data warehouses using platforms like Snowflake, Amazon Redshift, or Google BigQuery.
  • Optimize storage and query performance for analytical workloads.
  • Support the Management of Data Lakes/Integrate within existing organizational policies.
  • Understand and interpret policies to better support the command with regards to integrating products into existing data lakes to ensure long-term sustainability.

Integrate Data

  • Connect Data Sources
  • Integrate data from APIs, databases, flat files, and third-party systems.
  • Handle diverse data formats like JSON, XML, CSV, and Parquet.\
  • Migrate Data
  • Migrate data between systems during upgrades or transitions (e.g., moving from on-premises to cloud).
  • Configure APIs
  • Configure APIs to expose data for consumption by other systems or applications.
  • Manage Data Quality and Governance at the 21st TSC level in accordance with USAEUR-AF policies and guidance

Validate Data

  • Implement checks to ensure data accuracy, completeness, and consistency.
  • Identify and resolve data anomalies or errors.

Manage Metadata

Document data lineage, definitions, and transformations to ensure transparency and traceability.

Implement and manage Compliance and Security measures

Ensure data systems comply with regulations like GDPR, HIPAA, or CCPA.

Implement security measures such as encryption, access control, and auditing.

Work with internal teams within the Group to rectify or otherwise mitigate data quality issues in analyses and tool configuration being performed within the Group.

Work with PEOs and PMs of Systems of Record that provide feed the multiple data platforms to understand and correct data issues

Job Requirements:

Years of Experience combination of any of the following:

Eighteen (18) years of recent specialized experience

  • Major IT Certification AND fourteen (14) years of recent specialized experience 
  • Associate's degree in computer science, information management, or related discipline AND fourteen (14) years of recent specialized experience
  • Bachelor's degree in computer science, information management, or related discipline AND ten (10) years of recent specialized experience
  • Master's degree in computer science, information management, or related discipline AND seven (7) years of recent specialized experience
  • IA Technical Level II certification or higher:
  • CCNA-Security 
  • CompTIA CySA+ 
  • Global Industrial Cyber Security Professional (GICSP) 
  • GIAC Security Essentials (GSEC) 
  • CompTIA Security+ CE 
  • EC-Council Certified Network Defense (CND) 
  • (ISC)² Certified?System?Security?Practitioner (SSCP)

  • KI / Machine Learning Engineer (m/w/d)

    vor 2 Wochen

    Kaiserslautern, Rheinland-Pfalz, Deutschland matelso GmbH Vollzeit 80.000 € - 120.000 € pro Jahr

    Du willst in einem Software-Unternehmen arbeiten, das die Zukunft des Onlinemarketings und der Marketing-Technologie gestaltet? Bei matelso bekommst Du die Chance dazu – in Kaiserslautern Mit unserem interdisziplinären und internationalen Team entwickeln wir innovative Lösungen, die unseren Kunden helfen, Daten in Erfolgsgeschichten zu verwandeln –...

  • KI / Machine Learning Engineer (m/w/d)

    Vor 6 Tagen

    Kaiserslautern, Rheinland-Pfalz, Deutschland matelso GmbH Vollzeit 80.000 € - 120.000 € pro Jahr

    Du willst in einem Software-Unternehmen arbeiten, das die Zukunft des Onlinemarketings und der Marketing-Technologie gestaltet? Bei matelso bekommst Du die Chance dazu – in Kaiserslautern Mit unserem interdisziplinären und internationalen Team entwickeln wir innovative Lösungen, die unseren Kunden helfen, Daten in Erfolgsgeschichten zu verwandeln –...

  • Senior Site Process Engineer

    Vor 5 Tagen

    Kaiserslautern, Rheinland-Pfalz, Deutschland Amazon Vollzeit 90.000 € - 120.000 € pro Jahr

    DescriptionOperations is at the heart of Amazon business. We are known for our speed, accuracy and exceptional service. Our buildings deliver tens of thousands of products to hundreds of countries worldwide, every day. In these key roles, you'll come in to help us exceed productivity goals and deliver against our customer promises. Our scale is expansive and...

  • Senior Site Process Engineer

    Vor 4 Tagen

    Kaiserslautern, Rheinland-Pfalz, Deutschland Amazon Vollzeit 90.000 € - 120.000 € pro Jahr

    Operations is at the heart of Amazon business. We are known for our speed, accuracy and exceptional service. Our buildings deliver tens of thousands of products to hundreds of countries worldwide, every day. In these key roles, you'll come in to help us exceed productivity goals and deliver against our customer promises. Our scale is expansive and our...

  • Senior Site Process Engineer

    Vor 4 Tagen

    Kaiserslautern, Rheinland-Pfalz, Deutschland Amazon Vollzeit 60.000 € - 120.000 € pro Jahr

    DESCRIPTIONOperations is at the heart of Amazon business. We are known for our speed, accuracy and exceptional service. Our buildings deliver tens of thousands of products to hundreds of countries worldwide, every day. In these key roles, you'll come in to help us exceed productivity goals and deliver against our customer promises. Our scale is expansive and...

  • VMware Cloud Foundation Engineer

    Vor 6 Tagen

    Kaiserslautern, Rheinland-Pfalz, Deutschland SAIC Vollzeit 80.000 € - 120.000 € pro Jahr

    Are you an experienced Cloud Engineer with deep expertise in VMware Cloud Foundation (VCF)? Join SAIC in supporting the US Air Force Agency for Modeling and Simulation (AFAMS) at the US Air Forces Europe / Air Forces Africa (U-A) Warfare Center (UAWC). This an OCONUS position in Kaiserslautern, Germany.AFAMS is the leading organization responsible for...

  • VMware Cloud Foundation Engineer

    Vor 6 Tagen

    Kaiserslautern, Rheinland-Pfalz, Deutschland SAIC Vollzeit 99.900 € - 1.000.000 € pro Jahr

    SAIC accepts applications on an ongoing basis and there is no deadline.SAIC is a premier Fortune 500 mission integrator focused on advancing the power of technology and innovation to serve and protect our world. Our robust portfolio of offerings across the defense, space, civilian and intelligence markets includes secure high-end solutions in mission IT,...

  • VMware Cloud Foundation Engineer

    Vor 5 Tagen

    Kaiserslautern, Rheinland-Pfalz, Deutschland SAIC Vollzeit 80.000 € - 120.000 € pro Jahr

    Are you an experienced Cloud Engineer with deep expertise in VMware Cloud Foundation (VCF)? Join SAIC in supporting the US Air Force Agency for Modeling and Simulation (AFAMS) at the US Air Forces Europe / Air Forces Africa (U-A) Warfare Center (UAWC). This an OCONUS position in Kaiserslautern, Germany.AFAMS is the leading organization responsible for...

  • IT-Systemadministrator Linux

    vor 2 Wochen

    Kaiserslautern, Rheinland-Pfalz, Deutschland matelso GmbH Vollzeit 55.000 € - 75.000 € pro Jahr

    Du willst in einem Software-Unternehmen arbeiten, das die Zukunft des Onlinemarketings und der Marketing-Technologie gestaltet? Beimatelsobekommst Du die Chance dazu – in Kaiserslautern Mit unserem interdisziplinären und internationalen Team entwickeln wir innovative Lösungen, die unseren Kunden helfen, Daten in Erfolgsgeschichten zu verwandeln –...

  • Technical Sales Engineer

    Vor 6 Tagen

    Greater Kaiserslautern Area, Deutschland Irving Knight Group Vollzeit 1.000.000 € - 1.200.000 € pro Jahr

    Job Title:Technical Sales Engineer - Low Voltage Power DistributionSalary:€85,000 to €100,000Location:Greater Kaiserslautern Area (On-Site)Irving Knight are partnered with a leading manufacturer specialising in low-voltage electrical distribution products looking to add a Technical Sales Engineer to their data center business unit.Job Overview:Maintain...

  • VMware Cloud Foundation Engineer

    Vor 6 Tagen

    Kaiserslautern, Deutschland SAIC Vollzeit 90.000 € - 120.000 € pro Jahr

    Job ID: Location: KAISERSLAUTERN, RP, DEDate Posted: Category: Information TechnologySubcategory: Cloud Comp EngrSchedule: Full-timeShift: Day JobTravel: Yes, 10 % of the TimeMinimum Clearance Required: SecretClearance Level Must Be Able to Obtain: TS/SCIPotential for Remote Work: NoDescriptionAre you an experienced Cloud Engineer with deep expertise in...

  • Environmental Engineering I

    vor 2 Wochen

    Kaiserslautern, Deutschland AECOM Vollzeit

    At AECOM, we’re delivering a better world. We believe infrastructure creates opportunity for everyone. Whether it’s improving your commute, keeping the lights on, providing access to clean water or transforming skylines, our work helps people and communities thrive. Our clients trust us to bring together the best people, ideas, technical expertise and...

  • IT-Systemadministrator Linux

    vor 2 Wochen

    Kaiserslautern, Deutschland matelso GmbH Vollzeit

    Du willst in einem Software-Unternehmen arbeiten, das die Zukunft des Onlinemarketings und der Marketing-Technologie gestaltet? Bei matelso bekommst Du die Chance dazu – in Kaiserslautern! Mit unserem interdisziplinären und internationalen Team entwickeln wir innovative Lösungen, die unseren Kunden helfen, Daten in Erfolgsgeschichten zu verwandeln –...

  • IT-Systemadministrator Linux

    vor 3 Wochen

    Kaiserslautern, Deutschland matelso GmbH Vollzeit

    Du willst in einem Software-Unternehmen arbeiten, das die Zukunft des Onlinemarketings und der Marketing-Technologie gestaltet? Bei matelso bekommst Du die Chance dazu – in Kaiserslautern! Mit unserem interdisziplinären und internationalen Team entwickeln wir innovative Lösungen, die unseren Kunden helfen, Daten in Erfolgsgeschichten zu verwandeln –...

  • Master Thesis

    Vor 3 Tagen

    Kaiserslautern, Deutschland Fraunhofer-Gesellschaft Vollzeit

    The Fraunhofer-Gesellschaft , the resulting safety requirements to address critical failures are typically organized in requirements management tools like ReqSuite®. A related challenge is to keep the defined conceptual links between distinct artifacts like mínimal cut sets and the safety requirements consistent so that the resulting safety case is...