Big Data Engineer

We are looking for

• Develop and support ETL pipelines using Python & Pyspark with robust monitoring and alarming.
• Develop data models that are optimized for business understand-ability and usability.
• Develop and optimize divisional data lake using best practices for partitioning, compression, parallelization, etc.
• Develop and maintain metadata, data catalog, and documentation for internal business customers.
• Help internal business customers troubleshoot and optimize SQL and ETL solutions to solve reporting problems.
• 9+ years of Datawarehouse. Experience with Oracle/Teradata /Redshift.
• Extensive experience working with AWS cloud services like s3,Glue,RDS,Lambda,Redshift,DynamoDB,API gateway, SNS, SQL.
• Any prior experience on MAtillion Redshift ETL is good to have.
• Excellent communication must.
• Extensive hands on experience working on Apache Airflow.

• Big Data (Primary).
• AWS cloud ( s3, Glue, RDS, Lambda, Redshift, DynamoDB, API gateway, SNS, SQL ).
• ETL.
• Python.
• Pyspark.
• SQL, Oracle, Teradata, Redshift.
• Jira.
• DevOps.