Data Engineer

16 Jul 2024

Vacancy expired!

JD -

Data Engineering Lead/Solutions Architect As a data engineering lead and solutions architect, you will play a role of the key person on the team working with business analysts, data,, devops, test and release engineers to plan, estimate, architect the low level technical solution design and execute the project along with leading all the technical team on the project. Accountable that the team adheres to provided estimates and technical design, review code is appropriate to standards and best performance standards.

Requirements : • 10+ years of practical experience designing and building data solutions • Enterprise experience with Google Cloud Platform Big Data products including BigQuery, Pub/Sub, Cloud Composer, Dataflow, Dataproc, Dataprep, Data Studio, Bigtable, Cloud Storage, and more. • Understand different types of storage (filesystem, relational, NoSQL) and working with various kinds of data (structured, unstructured, metrics, log files, etc.) • Experience with data pipelines (ETL, ELT) and data wrangling procedures using Python and SQL. • Tune application and query performance using performance profiling tools and SQL • Experience with batch and stream processing (including Google Cloud Platform Dataflow/Kafka Streams/Spark) • Familiarity with agile software development practices and drive to ship quickly. • Research, analyze, and recommend technical approaches for solving difficult and challenging development and integration problems. • Bachelor's degree from four-year College or university in Computer Science or related field. • Responsible for maintenance and enhancement of data platform which involves adding various operators for carrying out tasks in Apache Airflow • Identify, design and implement internal process improvements by automating manual processes and optimizing data delivery

Nice-to-Have • Technical certifications such as Google Cloud Data Engineer or advanced certifications in data science a plus. • Experience with microservice patterns, API development, RESTful web services.

  • ID: #44129987
  • State: New Jersey Jerseycity 07097 Jerseycity USA
  • City: Jerseycity
  • Salary: Depends on Experience
  • Job type: Contract
  • Showed: 2022-07-16
  • Deadline: 2022-09-12
  • Category: Et cetera