Hadoop Engineer / Hadoop Admin.

Our client, one of the largest banks in the US with wealth management, investment banking, and international business, is seeking a Hadoop Engineer / Hadoop Admin.

• Design, build, and maintain Big Data workflows/pipelines to process continuous stream of data with experience in end-to-end design and build process of Near-Real-Time and Batch Data Pipelines
• Responsible for capacity planning and estimating the requirements for lowering or increasing the capacity of the Hadoop cluster.
• Active development of ETL processes using Spark or other highly parallel technologies, and implementing ETL/data pipelines
• Troubleshooting application errors and ensuring that they do not occur again.
• Responsible for deciding the size of the Hadoop/Spark cluster based on the data to be stored in HDFS. Implement, manage, and administer the overall Hadoop infrastructure.
• A Hadoop/Spark administrator will have to work closely with the database team, network team, BI team, and application teams to make sure that all the big data applications are highly available and performing as expected.
• Demonstrated work experience in the following with Big Data and distributed programming models and technologies
• Strong understanding of core Big Data Cloudera Hadoop services such as HDFS, MapReduce, Kafka, Spark and Spark-Streaming, Hive, Impala, HBase, Kudu, Sqoop, and Oozie.
• Proficient operating systems (Linux), servers, and shell scripting (e.g., Bash, KSH, etc.)

• 1-2 years, creating, maintaining, and managing Hadoop and Spark clusters
• 3-5 years of experience with development to centered around big data applications
• Ensures that the Hadoop/Spark cluster is up and running all the time.
• Monitoring the cluster connectivity and performance.
• Manage and review Hadoop log files.
• Backup and recovery tasks
• Experience with Data technologies and Big Data tools, like Spark, Kafka, Hive
• Takes care of the day-to-day running of Hadoop/Spark clusters
• Knowledge of database structures, theories, principles, and practices (both SQL and NoSQL).

• Ability to have effective working relationships with all functional units of the organization
• Excellent written, verbal and presentation skills
• Excellent interpersonal skills
• Ability to work as part of a cross-cultural team
• Self-starter and Self-motivated
• Ability to work without lots of supervision
• Works under pressure and can manage competing priorities.