Vacancy expired!
Client is embracing Big Data technologies to enable data-driven decisions. We’re looking to expand our Big Data Engineering team to keep pace. As a Sr. Big Data Engineer, you will work with a variety of talented Client teammates and be a driving force in technical initiatives that will accelerate analytics at Client. You will be working on projects that build data artifacts to answer questions about consumer behavior, commerce trends, consumer touchpoint preferences and more!
Here are some tasks that you could do day to day.- Design and implement distributed data processing pipelines using Spark, Hive, Python, and other tools and languages prevalent in the Hadoop ecosystem. You will be given the opportunity to own the design and implementation. You will collaborate with Product managers, Data Scientists, Engineering folks to accomplish your tasks.
- Publish RESTful API’s to enable real-time data consumption using OpenAPI specifications. This will enable many teams to consume the data that is being produced.
- Explore and build proof of concepts using open source NOSQL technologies such as HBase, DynamoDB, Cassandra and Distributed Stream Processing frameworks like ApacheSpark, Flink, Kafka stream.
- Take part in DevOps by building utilities, user defined functions and frameworks to better enable data flow patterns.
- Work with architecture/engineering leads and other teammates to ensure high quality solutions through code reviews, engineering best practices documentation.
- Experience in Business Rule management systems like Drools will also come in handy.
- MS/BS degree in a computer science or related discipline
- 3+ years’ experience in large-scale software development/Big Data technologies
- Programming skills in Java/Scala, Python, Shell scripting, and SQL
- Development skills around Spark, MapReduce, and Hive
- Strong skills around developing RESTful API’s