Big Data Architect

09 May 2024

Vacancy expired!

Internal Note:

  • How efficiently in the past 5 years has the candidate used ERWIN between (2016-2022) ?
  • Has the candidate done Data Architecture like Data Modelling with Big Data/Hadoop Platforms in the recent past(2+ years) ?
  • Job Description : BigData Architect is responsible for planning and designing next-generation big-data systems and managing large-scale development and deployment of Hadoop applications At least 6 years of hands-on experience with Hadoop applications and Google Cloud Platform Strong experience building data ingestion pipelines (simulating Extract, Transform, Load workload) in data warehouse and database architecture Hands-on development experience using open source big data components such as Hadoop, Scala, Hive, Pig, Spark, HBase, HDFS, YARN, Sqoop, NiFi, Storm, Impala, Hawk, Oozie, Mahout, Flume, Kafka, ZooKeeper, Sqoop etc. preferably with Cloudera / Hortonworks Experience in architecting and designing large scale ETL workload migration from traditional ETL (Ab Initio / Informatica) to Spark Experience in designing and leading Big Data projects, designing data models on Hive and HBase for high-performance and storage. Executed multiple Hadoop data lake projects (batch & streaming real time data) and led the developers’ team. Strong experience with data modeling, design patterns, building highly scalable Big Data Solutions and distributed applications Experience with storing, joining, filtering, and analyzing data using Spark, Hive and Map Reduce Deep understanding of business domain data and how it is used for metrics and analytical data designs Strong data design experience & knowledge that can be applied to big data architectures and cloud data lake environments, (i.e Very strong data analysis & design experience on Teradata and/or big data platforms for curation and analytics use cases) Business Analyst mindset/aptitude to understand domain data requirements for design deliverables Experienced in creating source to target mapping design documentation for data engineering Data Warehouse Data Modeling using Erwin - Data Flow, ER Diagram, Conceptual, Logical and Physical Deep understanding of data flows, data taxonomy and organization, data lineage Experience with architecting end to end data solutions for both batch and real time designs Metadata & documentation management required for Erwin modelling, data cataloging Creating high level and detailed data design documentation Experience working collaboratively with clients, developers, and architecture teams understanding requirements to design and implement data solutions Excellent verbal and written communication skills Strong data profiling skills and analysis skills (expert SQL skills, and strong data exploration and discovery skills)