Data Engineer

Coforge is an equal opportunities employer and welcomes applications from all sections of society and does not discriminate on grounds of race, religion or belief, ethnic or national origin, disability, age, citizenship, marital, domestic or civil partnership status, sexual orientation, or gender identity, or any other basis as protected by applicable law.

• He has hands on experience with Python, Java, Scala, PySpark, SQL, Hive, Pig, Hadoop Ecosystem on both on-premises and cloud (AWS) Infrastructure.
• In-depth understanding of Spark Architecture including Spark Core, PySpark, Spark SQL, Data Frames, Spark Streaming
• Worked on Dimensional Data modelling in Star and Snowflake schemas and Slowly Changing Dimensions (SCD).
• Experience on Azure Blob Storage and Cosmos Database for data transformation in Azure Databricks
• Experience in performing ETL on top of streaming data from IOT Machines using Spark Streaming.
• Worked in ETL, Provided technical consulting and guidance to development teams for the design and development of highly complex or critical ETL architecture.
• Experience on building the reporting subscription application in AWS cloud ecosystem using Amazon S3, EMR, Glue, Dynamo DB, Lambda and Redshift Database
• Hands on experience in importing and exporting data from databases like MySQL, MongoDB, Oracle, Teradata into HDFS and vice-versa using Sqoop.
• Experience in performing analytics on structured data in Hive with Hive queries, Views, Partitioning, and Bucketing using HiveQL
• Experience in writing the SQL and PL/SQL for creating the views and table for reporting and analytics.
• Experience in usage of Hadoop distribution like Cloudera5.3 (CDH5, CDH3) and Hortonworks distribution.