Apply on Employer Site

InfoCepts · 3 months ago

Sr. Hadoop Administrator

United States

Contract

Remote

Senior Level

InfoCepts is focused on harnessing the power of Big Data through their Big Data Operations team, which is building a NextGen Big Data platform on AWS. The Sr. Hadoop Administrator will design, build, and maintain infrastructure to support Big Data applications, ensuring efficiency, reliability, and security, while also automating operational tasks and participating in on-call rotations.

AppsArtificial Intelligence (AI)Big DataBusiness IntelligenceCloud Data ServicesMachine LearningMobile AppsNatural Language ProcessingPharmaceuticalRetail

H1B Sponsor Likely

Hiring Manager

Murtaza Patanwala

Responsibilities

In this job, you’ll design, build, scale and maintain the infrastructure in both datacenter and AWS to support Big Data applications

You will design, build, and own the end-to-end availability of Big Data platform in both AWS and datacenter

You will improve the efficiency, reliability, and security of our Big Data infrastructure, while making sure that our developers & analysts have a smooth experience with it

You will work on automation to build and maintain new platform on AWS

You will build custom tools to automate day-to-day operational tasks

You will be responsible for setting the standards for our production environment

You will take part in 24X7 on-call rotation with the rest of the team and respond to pages and alerts to investigate issues in our platform

Qualification

Hadoop ecosystemAmazon EMRHadoop securityInfrastructure automationPerformance tuningMonitoring solutionsBash & PythonJavaRDSJRE/JVM understandingRDBMS experienceSnowflake experienceQubole experienceAirflow experience

Required

Strong experience with Hadoop ecosystem like HDFS, Yarn, Hive, Spark, Oozie, Presto and Ranger

MUST have strong experience with Amazon EMR

Good working experience with RDS and good understanding on IaaS, PaaS

Strong foundation of Hadoop security including SSL/TSL, Kerberos, Role based authorization

Performance tuning experience of Hadoop clusters, ecosystem components and MR/Spark jobs

Experience with Infrastructure automation using Terraform, CI/CD pipelines (GIT, Jenkins etc), and configuration management tools like Ansible

Able to leverage technologies like Kubernetes/Docker (ELK) to help our Data Engineers/Developers scale their efforts in creating new and innovative products

Experience with providing and implementing monitoring solutions based on logs using CloudWatch, CloudTrail and Lambda etc

Ability to do post-mortem if something bad happens to your systems. Identify what went wrong and provide detailed RCA

Proficiency in Bash & Python or Java

Good understanding of all aspects of JRE/JVM and GC tuning

Hands-on experience with RDBMS (Oracle, MySQL) and basic SQL

Preferred

Hands-on experience with Snowflake is a plus

Hands-on experience with Qubole and Airflow is a plus

Company

InfoCepts

Glassdoor3.6

End-to-end Data & Analytics Solutions

Founded in 2004

Mclean, Virginia, USA

1001-5000 employees

http://www.infocepts.com

H1B Sponsorship

InfoCepts has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)

Distribution of Different Job Fields Receiving Sponsorship

Represents job field similar to this job

Trends of Total Sponsorships

2025 (26)

2024 (20)

2023 (22)

2022 (38)

2021 (45)

2020 (72)