Pyspark Engineer

10 Nov 2024

Vacancy expired!

Role: Pyspark EngineerNo Of Positions: 5Location: Charlotte, NCDuration: 6-12 Months Contract Job Description:

  • Strong knowledge in distributed data systems and processing technologies Hadoop, Spark (Python)
  • Expertise in Python Programming and creating object oriented modular code
  • Knowledge of PySpark libraries
  • Expertise in ETL Talend Data Integration – Jobs Designing & Implementation, Joblets , Deployment.
  • Knowledge in creating pipelines for real time data ingestion using Talend
  • Expertise in Talend Data Mapper – Utilizing TDM to read and create Mainframe files
  • Proficient in SQL and HQL
  • Experience with data orchestration tools - Airflow, Luigi or similar
  • Experience working in AWS cloud is strongly preferred, specifically S3, EMR, Athena, IAM, and Redshift
  • Good understanding of ML concepts and lifecycle
  • Experience in streaming solutions (preferred) - Kafka, spark-streaming, Kinesis, flink etc.
  • Knowledge of Unix shell Scripting
  • Knowledge on TSYS Payment solutions is good to have
  • Experience with Scrum/Agile development Methodologies
  • Familiarity with DevOps and CI/CD including Git, Jenkins, JFrog

  • ID: #22484060
  • State: North Carolina Charlotte 28201 Charlotte USA
  • City: Charlotte
  • Salary: Depends on Experience
  • Job type: Contract
  • Showed: 2021-11-10
  • Deadline: 2021-12-24
  • Category: Et cetera