Azure Data Engineer

25 Nov 2024

Vacancy expired!

Cerebra Consulting Inc is a System Integrator and IT Services Solution provider with a focus on Big Data, Business Analytics, Cloud Solutions, Amazon Web Services, Salesforce, Oracle EBS, Peoplesoft, Hyperion, Oracle Configurator, Oracle CPQ, Oracle PLM and Custom Application Development. Utilizing solid business experience, industry-specific expertise, and proven methodologies, we consistently deliver measurable results for our customers. Cerebra has partnered with leading enterprise software companies and cloud providers such as Oracle, Salesforce, Amazon and able to leverage these partner relationships to deliver high-quality, end-to-end customer solutions that are targeted to the needs of each customer. Position Title: Azure Data Engineer Long Term, Atlanta, GA(Location: Atlanta (It is remote working due to Covid-19, later need to move) Job Description: We are looking for Azure Data Engineer to help build and support enterprise data and analytics enablement solutions in the cloud. Azure Data Engineer works with business analysts, product owners and data solution experts to implement purpose-built solutions for our stakeholders. The Azure Data Engineer promotes good data governance with an emphasis on industry best practices for maintaining security and privacy over our data as well as our customer s data. Responsibilities for Azure Data Engineer:

  • Development of big data pipelines using Azure Data Factory V2 and Azure Data Lakes
  • Transform data by using Apache Spark, Transact-SQL, Data Factory, Azure Synapse Pipelines and Stream Analytics
  • Interface with external data sources / enterprise systems such as Salesforce, Oracle (cloud-based as well as on-rem)
  • Develop SQL queries/scripts and similar artifacts to validate the completeness, integrity, and accuracy of data within an ETL process testing cycle.
  • Design strategies for Azure Synapse Analytics and in Azure Data Lake Storage Gen2.
  • Design Data Factories and Pipelines and integrate Jupyter/Python notebooks into Data Pipelines.
  • Develop batch processing solutions by using Data Factory, Data Lake, Spark, Azure Synapse Pipelines, and Azure Databricks
  • Build and consume APIs for data services, and work with JSON and Parquet files
  • Deploy with automation data platforms in a cloud environment with Continuous Integration/Continuous delivery (CI/CD) processes.
  • Create, document, and maintain logical and physical database models in compliance with enterprise standards and maintained corporate metadata definitions for enterprise datastores within a metadata repository.
  • Manage and schedule data pipelines in Data Factory/Synapse Pipelines; manage Spark jobs.
  • Design Azure role-based access control (Azure RBAC) and Access Control List (ACL) for Data Lake Storage Gen2; for row-level and column-level security
  • Manage and work with identities, keys, and secrets across different data platform technologies.
  • Work with other members of the senior team to ensure that all tasks are resolved in a timely and acceptable manner, escalating, and briefing effectively where necessary.
Other Requirements:
  • Working knowledge of programming languages such as Python to code transformations
  • Demonstrates knowledge of, and carefully follows all applicable data security and privacy requirements and regulations as applicable to industry catering to.
  • Expertise in Data modelling for Data Warehouse/Data Mart development, Data Analysis for OLTP and Data Warehousing (OLAP)/ Business Intelligence (BI) applications.
  • Practical understanding of the Data modelling (Dimensional & Relational) concepts like Star-Schema Modelling, Snowflake Schema Modelling, Fact and Dimension tables.
  • Experience in extracting, transforming, and loading (ETL) data from spreadsheets, database tables and other sources using Microsoft SSIS and Azure Data Factory (ADF).
  • Comprehensive knowledge and experience in process improvement, normalization/de-normalization, data extraction, data cleansing, data manipulation.
  • Resolve ETL, Database/Data warehouse applications and BI reporting issues to maintain system stability.
  • Gain hands-on experience with new data platforms and programming languages (such as Python, Hive, Spark)
  • Willingness to work during maintenance slots during the week ends, as necessary.
Chosen candidate will have the following:
  • Bachelor's degree in Computer Science with 10 years of relevant overall work experience which including Data Engineering or Data Warehousing
  • Microsoft Azure Data Engineer Certification is on in either the Azure Data Engineer and/or Azure Architect domains is preferred.
  • Hands-on at least 2 years of experience with leading commercial Cloud platforms, including Azure (preferred: ADF pipelines, Analysis services, Azure SQL Database), AWS and Google Cloud Platform
  • Practical and in-depth knowledge of BI tools to troubleshoot real-time platform issues.
  • 3 years of proven experience with ETL/ELT, data warehousing, data ingestion, data profiling and data visualization
  • Proficient in SQL and debugging complex queries.
  • Introduction and day-day operational familiarity to Cloud DevOps
  • Experience using version control software like GitHub, TFS, Azure DevOps
  • Strong aptitude for learning new technologies and analytics techniques.
  • Highly self-motivated and able to work independently as well as in a team environment.
  • Understanding of agile project approaches and methodologies
  • Proficient in a source code control system, such as Git
  • Proficient in the Linux shell, including utilities such as SSH.
  • Self-motivated, with positive attitude and work ethic.
  • Ability to work well alone or within a team setting.
  • Possesses the ability to successfully manage and prioritize concurrent tasks.
  • Ability to communicate well (verbal and writing) and work with client s consumers/customers.
Please send us the profiles at or call me at .

  • ID: #23457338
  • State: Georgia Atlanta 30301 Atlanta USA
  • City: Atlanta
  • Salary: USD TBD TBD
  • Job type: Contract
  • Showed: 2021-11-25
  • Deadline: 2022-01-24
  • Category: Et cetera