Lead Data Engineer

29 Jul 2022

Job Title: Data Engineer Job Type: Contract/ Fulltime Job Location: Remote Required Skills: Python, Pyspark/Spark, SQL, Data Lake (ON Prem or cloud) AWS, Snowflake. Job Summary: The Data Engineer's role is to play a lead role in developing a high-performance data platform, integrating data from a variety of internal and external sources, in order to support data and analytics activities. This is a technical role that involves defining changes to the warehouse data model and building scalable and efficient processes to populate or modify warehouse data. The successful candidate will have hands-on data processing and data modeling experience in cloud and on-prem environments. Responsibilities:

  • Be a technical lead in the development of high-volume platforms to drive decisions and have a tangible beneficial impact on our clients and on business results.
  • Design and implement efficient data pipelines (ETLs) in order to integrate data from a variety of sources into Data Warehouse.
  • Design and implement data model changes that align with warehouse standards.
  • Design and implement backfill or other warehouse data management processes.
  • Develop and execute testing strategies to ensure high quality warehouse data.
  • Provide documentation, training, and consulting for data warehouse users.
  • Perform requirement and data analysis in order to support warehouse project definition.
  • Provide input and feedback to support continuous improvement in team processes.
  • Experience in working both, On-Prem and Cloud (AWS preferred).
  • Responsible for leading the team onsite and off shore with technical leadership and guidance.
  • 5+ years in a Data Engineering role
  • 7+ years hands on experience with SQL:
    • Ability to write/ interpret SQL and Complex Joins/ Queries
    • Execution plan and SQL optimization (Oracle SQL Profiler)
  • 3+ years coding experience (Python and/or PySpark).
  • 3+ years hands on experience with big data and cloud technologies (Snowflake, EMR, Redshift, or similar technologies) is highly preferred
    • Schema design and architecture on snowflake
    • Architecture and design experience with AWS cloud
    • AWS services expertise: S3, RDS, EC2, ETL services (Glue, Kinesis)
    • Consumption layer design experience for reporting and dashboard
  • Expert level understanding and experience of ETL fundamentals and building efficient data pipelines.
  • 3+ years at least - Enterprise GitHub branch, release, DevOps, CI/CD pipeline.
  • Team player, Strong communication and collaboration skills.
  • Experience with Agile methodologies.
  • Master's Degree (or a B.S. degree with relevant industry experience) in math, statistics, computer science, or equivalent technical field.
Satish Kumar Recruitment Manager , Paramount Software Solutions, Inc US: Email : Company Website : www.paramountsoft.net Product Website : www.farmtoplate.io Address : 4030, Old Milton Parkway, Alpharetta, GA 30005, USA Facebook LinkedIn Glassdoor Twitter

  • ID: #44466397
  • State: Arizona Chandler 85224 Chandler USA
  • City: Chandler
  • Salary: USD TBD TBD
  • Job type: Permanent
  • Showed: 2022-07-29
  • Deadline: 2022-09-26
  • Category: Et cetera