Vacancy expired!
Position:
Senior/Lead Big Data DeveloperEnd-Client Locations: NYC, NY/New Jersey (Hybrid Mode)Job Type: Full Time Employment with Prime Implementer (not with End-client)Salary: Best Lowest per year + Standard Benefits Package (Supplier will get Referral Fee)Candidates that are authorized to work for any employer in United States without Sponsorship are encouraged to apply. Primary responsibilities:- Designing Hive/HCatalog data model includes creating table definitions, file formats, compression techniques for Structured & Semi-structured data processing
- Implementing Spark processing based ETL frameworks
- Implementing Big data pipeline for Data Ingestion, Storage, Processing & Consumption
- Experienced AWS Developer, familiar with AWS services, S3, EC2, EMR / databricks, Lambda, Aws Ci/Cd
- Modifying the Informatica-Teradata & Unix based data pipeline
- Enhancing the Talend-Hive/Spark & Unix based data pipelines
- Develop and Deploy Scala/Python based Spark Jobs for ETL processing
- Strong SQL & DWH concepts
- Function as integrator between business needs and technology solutions, helping to create technology solutions to meet clients’ business needs
- Lead project efforts in defining scope, planning, executing, and reporting to stakeholders on strategic initiatives
- Understanding of EDW system of business and creating High level design document and low level implementation document
- Understanding of Big Data Lake system of business and creating High level design document and low level implementation document
- Designing Big data pipeline for Data Ingestion, Storage, Processing & Consumption
- ID: #49077363
- State: New York New york city 30301 New york city USA
- City: New york city
- Salary: Depends on Experience
- Job type: Permanent
- Showed: 2023-02-08
- Deadline: 2023-04-08
- Category: Et cetera