Data Engineer

Job opportunity for Data Engineer at Bhumi Varta Technology in KAB. TANGERANG, BANTEN, Indonesia.

Qualification:

Responsibilities :

Design, develop, and maintain scalable data pipelines.

Build data ingestion processes using Change Data Capture and ETL / ELT jobs.

Build data pipeline to cleanse and transform raw data into clean data in the targeted format to be stored in Data lake.

Build and manage jobs to preprocess data into an optimized and highly structured data format into Data mart.

Write clean, efficient, and well-tested code for data processing and transformation.

Optimize data storage and retrieval processes to ensure high performance and scalability.

Collaborate with cross-functional teams to understand data requirements and deliver solutions.

Participate in code reviews and contribute to technical discussions.

Monitor and optimize data pipeline performance.

Document technical specifications and data pipeline documentation.

Troubleshoot and resolve complex technical issues related to data processing.

Preferred Qualifications:

Bachelor’s degree in Computer Science, Software Engineering, or equivalent practical experience.

3+ years of experience in data engineering.

Strong proficiency in building data pipelines using Debezium, Airflow, Spark, Hive Metastore, Trino (Presto).

Solid understanding of data lake and data mart architectures.

Experience with ETL processes and data transformation techniques.

Proficiency in SQL and experience with relational and NoSQL databases.

Strong knowledge of software design patterns and principles.

Experience with containerization (Docker) and version control (Git).

Proven track record of writing clean, maintainable code.

Proficiency in unit testing and test-driven development.

Excellent problem-solving, communication, and collaboration abilities.

Experience in event-driven architecture is a plus.