Novia Infotech · 1 day ago
Hadoop Developer
United States
Contract
Remote
Mid LevelNovia Infotech is seeking a Hadoop Developer to design, develop, and maintain Big Data applications using the Hadoop ecosystem. The role involves building and optimizing ETL data pipelines and developing real-time and batch data processing solutions.
Information Technology & Services
Hiring Manager
Arpan Gupta
Responsibilities
Design, develop, and maintain Big Data applications using the Hadoop ecosystem
Work extensively with HDFS, Hive, Spark, MapReduce , and YARN for large-scale data processing
Build and optimize ETL data pipelines for structured and unstructured data
Develop real-time and batch data processing solutions using Apache Kafka and Spark Streaming
Perform data ingestion from multiple sources using tools like Sqoop, Flume, Kafka , and custom scripts
Optimize query performance in Hive, Impala , and Spark SQL
Collaborate with data analysts, data scientists, and application teams to deliver scalable data solutions
Ensure data quality, reliability, security, and governance across Hadoop clusters
Monitor, troubleshoot, and tune Hadoop jobs and cluster performance
Write clean, reusable, and well-documented code following best practices
Qualification
Required
Design, develop, and maintain Big Data applications using the Hadoop ecosystem
Work extensively with HDFS, Hive, Spark, MapReduce, and YARN for large-scale data processing
Build and optimize ETL data pipelines for structured and unstructured data
Develop real-time and batch data processing solutions using Apache Kafka and Spark Streaming
Perform data ingestion from multiple sources using tools like Sqoop, Flume, Kafka, and custom scripts
Optimize query performance in Hive, Impala, and Spark SQL
Collaborate with data analysts, data scientists, and application teams to deliver scalable data solutions
Ensure data quality, reliability, security, and governance across Hadoop clusters
Monitor, troubleshoot, and tune Hadoop jobs and cluster performance
Write clean, reusable, and well-documented code following best practices
Strong hands-on experience with Apache Hadoop ecosystem: HDFS, Hive, Spark, MapReduce
Cloudera / Hortonworks distributions
Impala, HBase
Experience with NoSQL databases such as MongoDB, Apache Cassandra
Proficiency in Java / Scala / Python for Big Data development
Experience with Apache Kafka for real-time data streaming
Strong knowledge of Linux/Unix environments and shell scripting
Familiarity with SQL and data warehousing concepts
Understanding of distributed systems, data partitioning, and fault tolerance
Experience with version control tools like Git
Company
Novia Infotech
At Novia Infotech, we’re more than just a service provider — we’re your partner in professional growth.
Founded in 2015
51-200 employees
Funding
Current Stage
Growth StageCompany data provided by crunchbase