Skip to main content
D

Datamatics Technologies

Data Engineer

Chennai, Chennai, IndiaPosted today

Skills

SparkETLScalaSQLGitCI/CDPythonKafka

Job description

Job Description – Data Engineer

Employment Type: Full-Time

Work Location: KSA

Experience Required: Hands-on experience with Spark development, Informatica integration solutions, and enterprise data platforms.

Years of Experience: 5 to 10

Role Overview

We are looking for a Data Engineer to build, configure, and optimize data ingestion, transformation, and consumption workloads on a modern on-premises lakehouse platform. The ideal candidate will have strong hands-on experience with Spark-based development, enterprise data lake/lakehouse implementations, and Informatica-based integration solutions.

Key Responsibilities

Design and develop scalable data ingestion and transformation pipelines.

Build, optimize, and maintain Spark-based ETL/ELT workloads.

Develop and manage data models and datasets using Apache Iceberg tables.

Implement and support data integration workflows using Informatica.

Enable analytics and data consumption through Denodo, Starburst, and/or Dremio.

Develop secure, governed, and scalable data pipelines aligned with enterprise standards.

Support batch and near real-time data processing requirements.

Implement data quality, metadata management, and validation controls.

Collaborate with Platform Engineers on deployment, integration, and operationalization activities.

Support migration, testing, and production rollout initiatives.

Troubleshoot and optimize data pipeline performance, reliability, and scalability.

Must-Have Skills

Apache Spark (PySpark and/or Scala Spark)

Apache Iceberg

Informatica Data Integration

MinIO Object Storage

Denodo

SQL and Data Modeling

Data Warehousing Concepts

ETL/ELT Development

Git Version Control

Good-to-Have Skills

Starburst and/or Dremio

Data Governance and Metadata Management

Data Quality Frameworks

CI/CD for Data Pipelines

Python

Kafka or Streaming Technologies

Lakehouse Architecture Patterns

Experience with enterprise data lake and lakehouse implementations

Performance tuning and optimization of large-scale data workloads

Apply on Datamatics Technologies