Data Engineer

We are looking for a skilled and motivated Data Engineer to join our team. As a Data Engineer, you will play a crucial role in designing, implementing, and maintaining data infrastructure and pipelines for our end clients. Your expertise in data engineering and strong technical skills will be vital in ensuring efficient data management and enabling data-driven decision-making.

Responsibilities:

1. Design and develop scalable, reliable, and efficient data pipelines, ensuring the smooth flow of data from various sources to our data storage and processing systems.
2. Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to understand data requirements and implement data solutions that meet business needs.
3. Build and maintain data warehouses, data lakes, and other data storage systems, ensuring data integrity, security, and availability.
4. Implement data transformation and cleansing processes to ensure data quality and consistency.
5. Optimize data pipelines and ETL processes for performance and scalability.
6. Develop and maintain data integration APIs and services to enable seamless data access and consumption by different applications and systems.
7. Work with stakeholders to define data architecture, data modeling, and database design to support analytical and reporting needs.
8. Implement and manage data governance processes, including data lineage, data cataloging, and data access controls.
9. Monitor and troubleshoot data pipelines and infrastructure to ensure smooth and uninterrupted data flow.
10. Stay updated with the latest trends and technologies in data engineering and recommend improvements.

Requirements:

1. Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
2. Proven experience as a Data Engineer or similar role, with a strong understanding of data engineering principles and best practices.
3. Proficiency in programming languages such as Python, SQL, or Scala for data manipulation and transformation.
4. Experience with data integration tools and technologies
5. Solid understanding of relational and non-relational databases, data modeling, and database design principles.
6. Familiarity with cloud platforms such as AWS, Azure, or Google Cloud, and experience with cloud-based data storage and processing services.
7. Strong knowledge of data warehousing concepts, ETL processes, and data pipeline architectures.
8. Experience with big data technologies such as Hadoop, Spark, or Hive.
9. Proficiency in data query and scripting languages such as SQL and Shell scripting.
10. Excellent problem-solving skills and the ability to work with large, complex datasets.