Senior Data Engineer

31 May 2024

Vacancy expired!

PETADATA is looking for a

Senior

Data engineer to work with one of our clients.

Job Location: Phoenix, AZ

Experience: 10+ years

Work Authorization: W2, Full-time

Visa:

H1B Visa, TN Visa,Citizen.

Job description:

Roles & Responsibilities:
  • Build big data pipelines and structures the large-scale banking systems
  • Implement and manage large scale ETL jobs on Hadoop/Spark clusters in Cloudera, Hortonworks platforms
  • Interface with internal teams and data consumer teams to understand the data needs
  • Own data quality throughout all stages of acquisition and processing, including data collection, ETL/wrangling, and normalization

Required Skills:
  • 4+ years of experience working with large data sets using open-source technologies such as Spark, Hadoop, Kafka on one of the major big data stack Cloudera, Hortonworks, and any other cloud systems like EMR
  • Strong SQL (, Hive, MySQL, etc) and No-SQL (HBase, etc.) skills, including writing complex queries and performance tuning
  • Must have a good command of Python, Spark, and big data techniques (Hive/Pig, MapReduce, Hadoop streaming, Kafka)
  • Excellent communication, relationship skills, and a strong team player. Preferred Qualifications
  • Experience developing and productizing real-world large scale data pipelines
  • Experience with Apache Airflow, Apache NiFi, Kafka, Apache Atlas, Schema Registry
  • Experience with DevOps, like GitLab, GitHub, Ansible, and automation
  • Expertise in Python scripting, Java is a big plus

Preferred Qualifications:
  • Bachelor's/ Master’s degree in Computer Science, Engineering, or a related field.

If you are interested and meet the above job requirements, please submit your resume to

After carefully reviewing your experience and skills, one of our Hiring team members will contact you on the next steps.

  • ID: #41983798
  • State: Arizona Phoenix 94536 Phoenix USA
  • City: Phoenix
  • Salary: Depends on Experience
  • Job type: Permanent
  • Showed: 2022-05-31
  • Deadline: 2022-07-16
  • Category: Et cetera