Role: Data Engineer
Total Experience: 5 to 8 Years
Job Location: Gurgaon
Budget -26 28 LPA
Job Purpose:
- Creating Project Technical Documentation
- Designing Solution architecture, and work on Data Ingestion, Preparation and Transformation.
- Debugging the production failures and identifying the solution.
- Developing efficient frameworks for development and testing using (Sqoop/Nifi/Kafka/Spark/Streaming/WebHDFS/Python) to enable seamless data ingestion process on to the Hadoop platform.
- Enabling Data Governance and Data Discovery on Hadoop Platform
- Building data processing framework using Talend, Spark, HQL
- Exposure of Security Framework with Kerberos, Ranger, Atlas
- Exposure of Data Pipeline Automation using DevOps tools
- Exposure of Job Monitoring framework along with validations automation
- Exposure to handling structured, Un Structured and Streaming data.
Skills:- Python, Tableau, ETL, NumPy, pandas, MongoDB, Shell Scripting, Bash and Amazon Web Services (AWS)