DATA LEAD with PYSPARK CONTRACT

28 Jun 2024

Vacancy expired!

Job title : DATA LEAD – PYSPARK

Location : Holmdel - NJ

Exp: 10+yrs

Required Skills ,Apache Spark, SparkSQL, PySpark

Nice to have skills - ,Scala, DB scripting

Job summary :
  • Develop PySpark jobs for ETL from source systems
  • Follow PySpark framework as outlined by the Enterprise Data Team
  • Perform code review with Enterprise Data Team
  • Create job scheduling via Control M with Scheduling Team
  • –-Unit test jobs and provide UAT support

Roles & Responsibilities
  • Develop PySpark jobs for ETL from source systems and perform enhancements
  • Perform sequential runs to validate the data
  • Support the PySpark Lead
  • Documentation of data flow
  • Understand the current landscape of Guardian applications
  • Review existing requirement documentations and understand the current state
  • Work with Business SMEs and understand the current source systems
  • Track and monitor progress on data offload and retro run
  • Update Status of progress with key Guardian stakeholders
  • Provide UAT support
ThanksKarthik | 240- 428- 4048karthikg(at)softpathtech(dot)com

  • ID: #43648503
  • State: New Jersey Holmdel 07733 Holmdel USA
  • City: Holmdel
  • Salary: Depends on Experience
  • Job type: Contract
  • Showed: 2022-06-28
  • Deadline: 2022-08-26
  • Category: Et cetera