Data Analytics Engineer

Position: Data Analytics Engineer Location: Remote Duration: 12+ months Rate: Open Interview: Video Data Analytics engineer End-to-end implementation of Data pipelines: Lead and/or work with cross-disciplinary teams to understand, document & analyze customer needs. The demands could be new projects, enhancing or fixing existing ones.) Identify & present a range of potential solution options for any demand, informing stakeholders of advantages & disadvantages of each; assist them in arriving at an optimal solution strategy. Optimize flexibility, scalability, performance, reliability and future-proof capacity of IT services, at an optimal cost Implement chosen solutions, including infrastructure, scripts, database resources, permissions, source control Contribute to the wider enterprise architecture & roadmap Core Design & develop Glue ETL jobs that can accommodate diverse & complex data sources, highly complex transformations & merges. Leverage EMR and Hive to process change-data-capture records both in S3 & Redshift Design & develop Lambda & AWS Batch scripts in Python. Perform data replication with Qlik Replicate and maintain data marts with Qlik Compose Design & incorporate error handling & Data Quality processes into pipelines & processes Design, implement & analyze robust test plans & stress tests Planning Conduct research into, test & trial new technologies & approaches they could enhance our work. Educate and train yourself and others as you evangelize the merits of data and analytics Document own, or existing projects, in a clear yet comprehensive format for a wide range of audiences. Contribute to enhancing the team's own internal processes of communications, documentation, workload planning Work closely with management to prioritize business and information request backlogs Ensures data governance and data security procedures are followed JOB REQUIREMENTS 4-year College Degree required; Bachelor's Degree in Information Technology field or related technical discipline preferred 3+ years as a Python, PySpark, Scala, Java, Hive software developer; building scalable real-time streaming ETL applications and data warehouses. Experience in working with and processing large data sets in a time-sensitive environment while minimizing errors Hands-on experience working with big data technologies (Hadoop, Hive, Spark, Kafka) Proficient experience working within the AWS and AWS tools (S3, Glue, EMR, Athena, Redshift) Experienced in maintaining infrastructure as code using Terraform or cloud formation Advanced understanding of both SQL and NoSQL technologies such as MongoDB / DocumentDB Hands-on experience working with Qlik (Attunity) Replicate & Compose Solid understanding of data warehouse design patterns and best practices Ability to develop test plans and stress test platforms Experience with complex Job scheduling Demonstrated strength in process development, process adherence, and process improvement Effective analytical, conceptual, and problem-solving skills Must be organized, disciplined, and task/goal oriented Able to prioritize and coordinate work through interpretation of high-level goals and strategy Effective team player with a positive attitude Strong oral and written English language communications skills