Vacancy expired!
Job Responsibilities Include: Design and Develop methods to connect & collect data from different data sources Analyze and understand data sources & APIs Design and Develop methods to filter/cleanse the data Design and Develop SQL, Hive queries, APIs to extract data from the store Work closely with data Scientists to ensure the source data is aggregated and cleansed Work with product managers to understand the business objectives Work with cloud and data architects to define robust architecture in cloud setup pipelines and work flows Work with DevOps to build automated data pipelines Basic Qualifications: 10 plus years of overall IT experience 3 plus years of experience with Hadoop ecosystem and Big Data technologies 3 plus years of Hadoop (Cloudera) or Cloud Technologies building pipelines using Spark /Pyspark 3 plus years of experience in programming in Scala and Python Preferred Skills: Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Yarn, Hive, Pig, Impala, Spark, Kafka,) Good understanding for ETL tools like Ab-initio, TalenD Familiarity with HTTP and invoking web-APIs Exposure to machine learning engineering Exposure to NLP and text processing Experience working in SCRUM methodology Proven sense of high accountability and self-drive to take on and see-through big challenges Confident, takes ownership, willingness to get the job done Excellent verbal communications and cross group collaboration skills Experienced in managing work with distributed teams