Software Engineer II – AWS EMR, AWS EMR

DescriptionJob summaryWant to change the world with Big Data and Analytics? Come join us on the Amazon EMR team in Amazon Web Services!Amazon EMR is a web service which enables customers to run massive clusters with open-source data processing frameworks like Apache Hive, Presto, Hadoop, Spark, HBase and more, with the ability to effortlessly scale up and down as needed. We run large number of customer clusters, enabling a variety of customer use cases including analytics and web-scale machine learning.We are doing innovative work on the open source data processing frameworks with the goal of making the EMR the best place to run big data workloads. We are looking for top engineers to design and build features and accelerate performance.You will work backwards from customer needs and you will get to do everything from designing and building large scale systems and cutting-edge features for the savviest customers in the business.You will have a chance to work with the open source community and contribute significant portions to open source projects including Hive, Tez, Presto, and Iceberg. You need to not only be a top software developer with excellent programming skills, an understanding of distributed systems and parallel data processing, and a stellar record of delivery but also excel at leadership and customer obsession and have a real passion for massive-scale computing. If you want to truly test your mettle against the hardest challenges in distributed systems to build solutions for large scale problems in a wide variety of domains, come join our group.Key job responsibilitiesDesign and develop complex, distributed platforms for storing data, scaling parallel algorithms, etc.

Work on internals of large-scale, highly parallel and fault-tolerant data processing engines like Hive, Tez and Presto.

Work on internals of file/table formats like Apache Iceberg.

Work on all components of EMR to provide a scalable and reliable Saas/Serverless platform for processing petabytes of data on hundreds of thousands of machines.

Solve challenging problems in query optimization, vectorized execution engine, resource management, transactions, security, governance, data access control, indexes, encoding, etc.

Contribute back to the community by open sourcing some of the developed features.

Work with customers to understand their needs and incorporate them as features into our product.

Inclusive Team CultureHere at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 16 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust.Work/Life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded professional and enable them to take on more complex tasks in the future.BASIC QUALIFICATIONS1+ years of experience contributing to the system design or architecture (architecture, design patterns, reliability and scaling) of new and current systems.

2+ years of non-internship professional software development experience

Programming experience with at least one software programming language.

Bachelors degree in Computer Science or Engineering required, Masters/Ph.D. a plus

Minimum 3 years of work experience

Strong CS fundamentals including problem solving, data structures, algorithms, and distributed systems

Systems programming skills including multi-threading, concurrency, etc.

Fluency in C, C, or Java preferred

PREFERRED QUALIFICATIONSMasters or PhD in Computer Science or Engineering with a minimum of 3 years of relevant software development experience

Experience in distributed systems design, implementation and testing

Experience in database internals such as query optimization or query execution

Hands-on experience in big data processing and infrastructure frameworks like Hadoop, Hive, Spark, Presto, Tez, HBase, HUDI, Iceberg, Docker, Kubernetes is a big plus

Experience with AWS or any other Cloud Platform is a plus.

Experience in delivering one or more version one products, ideally in a startup setting.

Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.