Big Data Specialist - East Coast job vacancy

Role: Big Data Developer/Specialist

Work Location: East Coast

Primary responsibilities:

Designing Hive/HCatalog data model includes creating table definitions, file formats, compression techniques for Structured & Semi-structured data processing
Implementing Spark processing based ETL frameworks
Implementing Big data pipeline for Data Ingestion, Storage, Processing & Consumption
Modifying the Informatica-Teradata & Unix based data pipeline
Enhancing the Talend-Hive/Spark & Unix based data pipelines
Enhancing the Talend-Hive/Spark & Unix based data pipelines
Develop and Deploy Scala/Python based Spark Jobs for ETL processing
Strong SQL & DWH concepts
Preferred background:
Function as integrator between business needs and technology solutions, helping to create technology solutions to meet clients’ business needs
Lead project efforts in defining scope, planning, executing, and reporting to stakeholders on strategic initiatives
Understanding of EDW system of business and creating High level design document and low level implementation document
Understanding of Big Data Lake system of business and creating High level design document and low level implementation document
Designing Big data pipeline for Data Ingestion, Storage, Processing & Consumption.
Good to have hands on coding experience.