Vacancy expired!
Resp & Qualifications
PURPOSE:The Senior Data Engineer is responsible for orchestrating, deploying, maintaining and scaling cloud OR on-premise infrastructure targeting to implement the on-going expansion of our Enterprise Data platform as well as moving DB2, Cloudera and Ab Initio from on-premises to AWS cloud including data access APIs with emphasis on reliability, automation and performance. This role will focus on maintaining, developing solutions and helping transform the company's platforms deliver data-driven, meaningful insights and value to company.ESSENTIAL FUNCTIONS:- Administers and maintains infrastructure systems (e.g., data warehouses, data lakes) including data access APIs. Prepares and supports manipulating data using Hadoop or equivalent MapReduce platform. Secure Hadoop or similar platform along with securing data in transit and at rest by working closely with Security and by following Security teams guidelines. Troubleshoots incidents to recover services, and support the root cause analysis. Develops and follows standard operating procedures (SOPs) for common tasks to ensure quality of service.
- Manages customer and stakeholder needs, generates and develops requirements, and performs functional analysis. Fulfills business objectives by collaborating with network staff to ensure reliable software and systems. Enforces the implementation of best practices for data auditing, scalability, reliability, high availability and application performance. Develop and apply data extraction, transformation and loading techniques in order to connect large data sets from a variety of sources.
- Installs, deploy, tunes, upgrades, troubleshoots, and maintains all computer systems relevant to the supported applications including all necessary tasks to perform operating system administration, user account management, disaster recovery strategy and networking configuration.
- Creates data collection frameworks for structured and unstructured data.
- Applies data extraction, transformation and loading techniques in order to connect large data sets from a variety of sources.
- Applies and implements best practices for data auditing, scalability, reliability and application performance.
- Improves data delivery engineering job knowledge by attending educational workshops; reviewing professional publications; establishing personal networks; benchmarking state-of-the-art practices; participating in professional societies.
- AWS and/or Cloudera certifications are pluses.
- This role requires the ability and experience in plan, develop and lead systems engineering project and efforts, achieve milestones and objectives.
- 8+ years of experience as Big Data Administration – Cloudera including build, deploy and management of large-scale Hadoop based data Infrastructure.
- 8+ years of experience with My SQL, MS SQL Server, No SQL Databases.
- 3+ years of experience with integration between CDH 6.3 or higher/CDP 7.1.7 or higher Cluster with Mainframe, UDB DB2, MicroStrategy, Control-M, Ab Initio.
- Experience in troubleshooting errors in HBase Shell, Impala, Pig, Hive and MapReduce job failures.
- Experience with AWS.
- Experience with 24/7 on-call support and triage production issues.
- Knowledge and understanding of at least one programming language (i.e., SQL, NoSQL, Python).
- Knowledge and understanding of database design and implementation concepts.
- Knowledge and understanding of data exchange formats.
- Knowledge and understanding of data movement concepts.
- Strong technical and analytical and problem solving skills to troubleshoot to solve a variety of problems.
- Requires strong organizational and communication skills, written and verbal, with the ability to handle multiple priorities.