Big Data Specialist - East Coast

29 Jul 2022
Apply

Role: Big Data Developer/Specialist

Work Location: East Coast

Primary responsibilities:
  • Designing Hive/HCatalog data model includes creating table definitions, file formats, compression techniques for Structured & Semi-structured data processing
  • Implementing Spark processing based ETL frameworks
  • Implementing Big data pipeline for Data Ingestion, Storage, Processing & Consumption
  • Modifying the Informatica-Teradata & Unix based data pipeline
  • Enhancing the Talend-Hive/Spark & Unix based data pipelines
  • Enhancing the Talend-Hive/Spark & Unix based data pipelines
  • Develop and Deploy Scala/Python based Spark Jobs for ETL processing
  • Strong SQL & DWH concepts

    Preferred background:
  • Function as integrator between business needs and technology solutions, helping to create technology solutions to meet clients’ business needs
  • Lead project efforts in defining scope, planning, executing, and reporting to stakeholders on strategic initiatives
  • Understanding of EDW system of business and creating High level design document and low level implementation document
  • Understanding of Big Data Lake system of business and creating High level design document and low level implementation document
  • Designing Big data pipeline for Data Ingestion, Storage, Processing & Consumption.
  • Good to have hands on coding experience.

  • ID: #44466726
  • State: New York New york city 10001 New york city USA
  • City: New york city
  • Salary: $120,000 - $180,000
  • Job type: Permanent
  • Showed: 2022-07-29
  • Deadline: 2022-09-16
  • Category: Et cetera
Apply