What you’ll be doing...

  • Deploy hadoop cluster, maintain a hadoop cluster, add and remove nodes using cluster monitoring tools (Cloudera Manager), configure the NameNode high availability and keep track of all the running hadoop jobs.
  • Implement, manage and administer the overall hadoop infrastructure.
  • Take care of the day-to-day running of Hadoop clusters
  • Work closely with database team, network team, BI team and Application teams to make sure that all the big data applications are highly available and performing as expected.
  • Be responsible for capacity planning and estimating the requirements for lowering or increasing the capacity of the hadoop cluster.
  • Be responsible for deciding the size of the hadoop cluster based on the data to be stored in HDFS.
  • Ensure that the hadoop cluster is up and running all the time.
  • Monitoring the cluster connectivity and performance.
  • Manage and review Hadoop log files.
  • Backup and recovery tasks
  • Resource and security management
  • Troubleshoot application errors and ensure that they do not occur again.

What we’re looking for...

  • Experience in designing, implementing and administering highly available Hadoop clusters secured with Kerberos, preferably using the Cloudera Hadoop distribution
  • In-depth understanding of the Hadoop ecosystem (e.g. HDFS, MapReduce, HBase, Pig, Scoop, Spark, Hive).
  • Experience with SSL & Kerberos installation and configuration for Hadoop services
  • Experience setting up/configuring system-wide health and metric alerts.
  • Work with developers to evaluate their Hadoop use cases, provide feedback and design guidance
  • Day-to-day troubleshooting of problems and performance issues in our clusters Investigate and characterize non-trivial performance issues in various environments
  • Operational expertise such as troubleshooting skills, understanding of system's capacity, bottlenecks, basics of memory, CPU, OS, storage, and networks.
  • Experience with monitoring tools used in the Hadoop ecosystem such as Nagios, Cloudera Manager
  • knowledge of Linux Administration and Troubleshooting.
  • Experience with automation tools such as Ansible.
  • Hands on experience with administration of MySQL database platform preferably Percone MySQL.
  • Strong experience on Kafka administration and standing up Kafka clusters.
  • Design and implement Elasticsearch.
  • Provide administration and operations of the Kafka platform like provisioning, access lists Kerberos and SSL configurations and configuring
  • Kafka for performance.
  • Provide expertise in Kafka brokers, zookeepers, Kafka connect, schema registry, multi data-center replication and Kafka Control center.
  • Create topics, setup redundancy cluster, deploy monitoring tools, alerts and has good knowledge of best practices.
  • Ensure optimum performance, high availability and stability of solutions.
  • Scripting using Shell, Python etc will be extremely useful.
  • Java knowledge is a plus.
  • Extensive skills in Elasticsearch tuning/optimizing based on application needs.
  • Ability upgrade ELK stack, troubleshoot and ability apply OS patches.
  • Willing to be perform operational support (24X7) on-call duties in rotation.
  • Ability to work independently and willing to work with team members spread across the globe.

When you join Verizon...

You’ll be doing work that matters alongside other talented people, transforming the way people, businesses and things connect with each other. Beyond powering America’s fastest and most reliable network, we’re leading the way in broadband, cloud and security solutions, Internet of Things and innovating in areas such as, video entertainment. Of course, we will offer you great pay and benefits, but we’re about more than that. Verizon is a place where you can craft your own path to greatness. Whether you think in code, words, pictures or numbers, find your future at Verizon.