Hadoop Lead Systems Administrator

20 Jul 2024

Vacancy expired!

Job Description Role: Hadoop Lead Systems AdministratorLocation: st. Louis, MO (on site).Duration: Long Term Contract Your role: As an active member of the TELUS International, you will be in a core of team that focuses on new features, maintenance of existing application, delivering best user experience and high quality. The Lead Administrator will perform and oversee a variety of key functions to design, implement, monitor and maintain Hadoop solutions ensuring they are available, reliable, and stable in a highly secure environment. Will also serve as the technical lead to projects that are large and/or complex in nature and will work closely with Business Partners, Project Leads, IT architects and Data Scientists as well as provide mentorship to new team members and provide technical guidance to the rest of the team and management.

Job Responsibilities:
  • Setup all the configurations- Core-Site, HDFS-Site, YARN-Site and Map Red-Site. However, when working with popular hadoop distribution like Hortonworks, Cloudera or MapR the configuration files are setup on startup and the hadoop admin need not configure them manually.
  • Responsible for capacity planning and estimating the requirements for lowering or increasing the capacity of the hadoop cluster.
  • responsible for deciding the size of the hadoop cluster based on the data to be stored in HDFS.
  • Ensures that the hadoop cluster is up and running all the time.
  • Monitoring the cluster connectivity and performance.
  • Manage and review Hadoop log files.
  • Backup and recovery tasks
  • Resource and security management
  • Troubleshooting application errors and ensuring that they do not occur again.
  • Deploys Hadoos cluster, maintains clusters and manage to add or remove nodes using cluster monitoring (Cloudera Manager) configuring the NameNode high availability and keeping a track of all the running hadoop jobs.
  • Implement, manage and administeris the overall hadoop infrastructure.
  • Takes care of the day-to-day running of Hadoop clusters
  • A hadoop administrator will have to work closely with the database team, network team, BI team and application teams to make sure that all the big data applications are highly available and performing as expected. Required knowledge and experience:
  • 7+ years of overall experience in Information Technology and Systems (Sr. or Lead)
  • Excellent knowledge of Hadoop, Spark and Cloudera
  • 5+ years of Cloudera practical experience on enterprise platforms
  • Experience with multiple large scale Enterprise Hadoop environment builds and operations including design, capacity planning, cluster set up, security, performance tuning and monitoring.
  • Experience with the full Cloudera CDH distribution to install, configure and monitor all services in the CDH stack
  • Strong understanding of core Big Data Cloudera Hadoop services such as HDFS, MapReduce, Kafka, Spark and Spark-Streaming, Hive, Impala, HBASE, Kudu, Sqoop, and Oozie.
  • Expertise in typical system administration and programming skills such as storage capacity management, debugging, performance tuning.
  • Proficient operating systems (Linux), servers and shell scripting (e.g. Bash,ksh,etc.) Skills Needed:
  • Excellent leadership skills.
  • Must have initiative and be a self - starter.
  • Problem-Solving Skills and technical understanding to determine the causes of operating errors and be able to make decisions accordingly.
  • Excellent communication skills. Being assertive to convey information effectively.
  • Effective Time Management.

Studies:
  • Bachelor's degree in the field of Information Systems, Computer Science, Data Analytics, Physical Science, or related field.
  • Hadoop Certifications (Desirable)

  • ID: #44240571
  • State: Missouri Saintlouis 63101 Saintlouis USA
  • City: Saintlouis
  • Salary: Depends on Experience
  • Job type: Contract
  • Showed: 2022-07-20
  • Deadline: 2022-09-15
  • Category: Et cetera