Google Cloud Platform Data Engineer

26 Nov 2024

Vacancy expired!

Google Cloud Platform Data Engineer

The Google Cloud Platform Data Engineer is responsible for building and maintaining automated CI/CD pipelines using hosted cloud-based services in Google Cloud and AWS. This position is part of the Enterprise Information team and works with solution architects and management to ensure customer requirements are met in a timely and efficient manner. As we are a retail institution, security, data integrity, and system availability are essential.

Manager Notes: Developing integration program - data flow, in general the requirement have big data database and pull data into big query. Similar to Glue/EMR or snowflake - someone that is used to spark or hive prob wouldn' t work since it is different. Falls under the big data solution, Athena is what AWS has on top of its EMR, those are the similarities to what we have. Build streaming structures and build underlining tables. Doesn' t have to be truly Google Cloud Platform. Looking for how fast they have changed their mindset from traditional RDMS to solution of big data and give us a better understanding for learning and moving toward Google.

Job Duties:
  • Develop and implement data engineering solutions to support data and analytics capabilities
  • Work with solution architects to implement cloud applications and services based on customer requirements.
  • Act as an escalation for support administrators troubleshooting complex workflows.
  • Maintain secure environments in accordance with established guidelines and practices maintaining the model of least privileged access.
  • Develop batch, real-time data ingestion and stream-analytic solutions leveraging key technologies based on standardized architecture practices.
  • Implement and maintain application and pipeline performance/health monitoring for cloud infrastructure-based solutions.
  • Deep knowledge with google cloud SDK especially BQ and gsutil tools.
  • Implement and maintain standards for configuration management of cloud infrastructure services
  • Develop and maintain CI/CD solutions based on Gitlab, Maven etc Implement and maintain container-based application services using AWS Kubernetes Service (AKS) or Google Kubernetes Engine (GKE)
  • Implement and maintain container-based application services using AWS Kubernetes Service (AKS) or Google Kubernetes Engine (GKE).
  • Seek opportunities to improve or optimize processes through automation via cloud centric tools, Terraform, and other automation scripting tools.

Knowledge, Skills and Abilities:
  • Excellent background in BigQuery and Google Cloud Platform technologies.
  • Proficiency in Airflow, DAG, Data flow, cloud pub/sub
  • Google Cloud Platform certifications preferred
  • L4/L5 problem solving skills, including good understanding of cloud solutions provided by Google and AWS.
  • Experience with Object Oriented Programming using Python, Java and its design patterns.
  • Proficiency with Terraform, Chef, Drone / Jenkins, Git, shell scripting, python or equivalent scripting language.
  • Infrastructure as Code experience based on cloud automation using Terraform and cloud services such as AWS and Google Cloud Platform Devops.
  • Simple understanding of database SQL. Ability to write queries and verify datasets.
  • Strong communicator that is self-driven and willing to adapt quickly to the changing landscape of cloud
  • An understanding of the Agile development method and its implementation.
IND123

  • ID: #23516812
  • State: Texas Grapevine 76051 Grapevine USA
  • City: Grapevine
  • Salary: USD TBD TBD
  • Job type: Permanent
  • Showed: 2021-11-26
  • Deadline: 2022-01-24
  • Category: Architect/engineer/CAD