Data Engineer -3

We are searching for an accountable, multitalented data engineer to facilitate the operations of our data scientists. The data engineer will be responsible for employing machine learning techniques to create and sustain structures that allow for the analysis of data while remaining familiar with dominant programming and deployment strategies in the field. During various aspects of this process, you should collaborate with coworkers to ensure that your approach meets the needs of each project.

To ensure success as a data engineer, you should demonstrate flexibility, creativity, and the capacity to receive and utilize constructive criticism. A formidable data engineer will demonstrate unsatiated curiosity and outstanding interpersonal skills.

Responsibilities:

• Responsible for setting up a  scalable DataWarehouse and building data pipeline mechanisms  to integrate the data from various sources for all of Klub’s data.  
• Setup data as a service to expose the needed data as part of Apis. 
• Have a good understanding on how the finance data works.
• Standardise and optimise design thinking across the technology team.
• Collaborate with stakeholders across engineering teams to come up with short and long-term architecture decisions.
• Build robust data models that will help to support various reporting requirements for the business , ops and the leadership team. 
• Participate in peer reviews , provide code/design comments.
• Own the problem and deliver to success.

Requirements:

• Overall 5+ years of industry experience
• Prior experience on Backend and Data Engineering systems. 
• Should have at least 2+ years of working experience in distributed systems or 3+ years of experience in Data Engineering. 
• Deep understanding on python tech stack with the libraries like Flask, scipy, numpy, pytest frameworks.
• Good understanding of Apache Airflow or similar orchestration tools. 
• Good knowledge on data warehouse technologies like Apache Hive or similar. 
• Good knowledge on Apache PySpark or similar. 
• Good knowledge on how to build analytics services on the data for different reporting and BI needs. 
• Good knowledge on data pipeline/ETL tools Hevo data or similar. 
• Good knowledge on Trino / graphQL or similar query engine technologies. 
• Deep understanding of concepts on Dimensional Data Models. 
• Familiarity with RDBMS (MySQL/ PostgreSQL) , NoSQL (MongoDB/DynamoDB) databases & caching(redis or similar).
• Should be proficient in writing sql queries. 
• Good knowledge on kafka. 
• Be able to write clean, maintainable code.

Nice to have- 

• Built a Data Warehouse from the scratch and set up a scalable data infrastructure.
• Prior experience in fintech would be a plus.
• Prior experience on data modelling.