Lead Data Engineer

24 Jun 2024

Vacancy expired!

We're looking for a full-time resource for our client and the first 90 days would be day 1 onsite and fairly open if they're open to working 4 days a week and completing a hybrid after 90 days. Pls, check from the exp range strong 9yrs-13-15yrs exp. Work location: Richardson, TX Position Overview A Technical Lead Data Engineer is proficient in leading a team in the development of all aspects of data processing including data warehouse architecture/modeling and ETL processing. This position focuses on research on the development and delivery of analytical solutions using various tools including Confluent Kafka, Kinesis, Glue, Lambda, Snowflake, and SQL Server. A Technical Lead Data Engineer must be highly technical and able to prioritize/align their team's work with company goals to deliver projects on time with scalable solutions. Position Responsibilities Responsible for project prioritization, resource allocation, and on-time delivery of solutions Require interaction with onshore/offshore counterparts to communicate business requirements in a technical design document Lead and foster data engineers in their careers to produce higher quality solutions at a faster velocity through optimization training and code review Multi-task across several ongoing projects and daily duties of varying priorities as required Partner with business stakeholders to gather requirements and translate them into technical specifications and process documentation for IT counterparts (on-prem and offshore) Highly proficient in the architecture and development of an event-driven data warehouse; streaming, batch, data modeling, and storage Advanced database knowledge; creating/optimizing SQL queries, stored procedures, functions, partitioning data, indexing, and reading execution plans Skilled experience in writing and troubleshooting Python/PySpark scripts to generate extracts, cleanse, conform and deliver data for consumption Expert level of understanding and implementation of ETL architecture; data profiling, process flow, metric logging, and error handling Support continuous improvement by investigating and presenting alternatives to processes and technologies to an architectural review board Develop and ensure adherence to published system architectural decisions and development standards The duties listed above are the essential functions of fundamental duties within the job classification. The essential functions of individual positions within the classification may differ. May assign reasonably related additional duties to individual employees consistent with standard departmental policy. Requirements 4+ years leading a team or projects 10+ years of development experience Highly proficient in Snowflake development Bachelor's degree in Computer Science, MIS, or related field (industry experience substitutable) Expert level in data warehouse design/architecture, dimensional data modeling, and ETL process development Advanced level development in SQL/NoSQL scripting and complex stored procedures (Snowflake, SQL Server, DynomoDB, NEO4J a plus) Extremely proficient in Python, PySpark, and Java AWS Expertise Kinesis, Glue (Spark), EMR, S3, Lambda, and Athena Streaming Services Confluent Kafka and Kinesis (or equivalent) Hands-on experience in designing and developing applications using Java Spring Framework (Spring Boot, Spring Cloud, Spring Data, etc)

  • ID: #43578933
  • State: Texas Richardson 75085 Richardson USA
  • City: Richardson
  • Salary: $160000
  • Job type: Permanent
  • Showed: 2022-06-24
  • Deadline: 2022-08-23
  • Category: Et cetera