Vacancy expired!
Lead Data Engineer
Piper Companies is seeking an Remote Lead Data Engineer for one of our customers. This position will play a a real role in the data engineering throughout our customer's environment. About the Role As a Lead Data Engineer, you will be working to implement and curate our big data strategy and to expand and optimize data pipelines and data movement across teams. You will be building and deploying solutions that enable the Organization to easily wrangle, prep, and leverage data in support of the ongoing needs of our clients. You will collaborate with our Data Science, Engineering and Product teams to understand requirements and to provide guidance and recommendations on solutions that meet their needs. Responsibilities:- Create and maintain efficient and scalable data pipeline architectures
- Define, implement, and evolve the necessary data models to support our big data lifecycle needs
- Build solutions to ingest, transform, and load data from a wide variety of data sources
- Work with all stakeholders including executive, product, and engineering teams to assist with data-related issues and needs
- Evaluate the existing big data ecosystem and recommend tools, processes, and methods that support the target strategy
- Establish analytics tools that enable data scientists and business analysts in building and optimizing models that support our business goals
- Refine the existing processes, guidelines, and tools for data management solutions covering data movement, data security, data privacy, and metadata management.
- Contribute to technical strategy, architectures, and risk consultation for Engineering and Product teams
- BA/BS degree in Computer Science, a related Software Engineering field or equivalent practical experience
- 5+ years of experience working in a Data Engineer role
- Familiarity with a wide variety of database technologies, both relational and non-relational.
- Advanced working knowledge of SQL and experience authoring, evaluating and optimizing queries
- Demonstrated knowledge of big data technologies and frameworks (i.e. Hadoop, Spark, Hive, Kafka, Snowflake etc.)
- Experience in the implementation of large scale data lake solutions and toolsets
- Hands on experience with ETL tools (Infosphere, Informatica, Talend etc.)
- Experience with big data solutions developed in large cloud computing infrastructures such as AWS or Azure is preferred
- Data Science and Analytics experience is a plus (Machine Learning, Scala, Python, R, etc.)
- Direct Hire
- Fully Remote
- $150-165K