This job has closed.

Tata Consultancy Services · 1 day ago

Pyspark Developer

Jersey City, NJ

Full-time

Onsite

Mid Level

$110K/yr - $130K/yr

Tata Consultancy Services is a leading IT services provider, and they are seeking a Pyspark Developer to design and maintain data pipelines. The role involves optimizing PySpark applications, ensuring data quality, and collaborating with cross-functional teams to support data-driven decision-making.

Business Information SystemsConsultingInformation TechnologyIT Management

H1B Sponsor Likely

Responsibilities

Design, develop, and maintain scalable data pipelines using PySpark for ETL processes, integrating multiple data sources, transforming datasets, and loading into target systems

Optimize PySpark applications and Spark jobs for efficiency, fine-tuning configurations, and ensuring high performance with large-scale datasets

Implement validation rules, error-handling mechanisms, and monitoring frameworks to ensure accuracy, consistency, and integrity of data throughout its lifecycle

Partner with data engineers, data scientists, and business analysts to translate business requirements into technical solutions that support data-driven decision-making

Write clean, efficient, and well-documented PySpark code, follow coding standards, and actively participate in peer code reviews

Monitor Spark jobs, identify and resolve issues, and provide production support for PySpark-based applications

Qualification

Apache SparkPySparkPythonETL processesSQLNoSQL databasesBig data ecosystemsCloud platformsProblem-solvingCommunication skillsCollaboration skills

Required

Strong hands-on experience with Apache Spark and PySpark for large-scale data processing

Proficiency in Python programming with focus on writing optimized, modular, and reusable code

Solid understanding of ETL processes, data pipeline design, and data integration techniques

Experience in performance tuning and optimization of Spark jobs in distributed computing environments

Good knowledge of SQL and working with relational as well as NoSQL databases

Familiarity with big data ecosystems (e.g., Hadoop, Hive, HDFS) and data warehouses (e.g., Snowflake, Redshift)

Understanding of data quality, validation, and error-handling frameworks

Strong problem-solving skills and ability to troubleshoot production issues

Good communication and collaboration skills to work with cross-functional teams

BACHELOR OF COMPUTER SCIENCE

Preferred

Exposure to cloud platforms (AWS, Azure, or GCP) and their data services is a plus

Benefits

Discretionary Annual Incentive.

Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.

Family Support: Maternal & Parental Leaves.

Insurance Options: Auto & Home Insurance, Identity Theft Protection.

Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.

Time Off: Vacation, Time Off, Sick Leave & Holidays.

Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Company

Tata Consultancy Services

Glassdoor3.8

Tata Consultancy Services is a business solutions company that specializes on information technology services and consulting.

Founded in 1968

Mumbai, Maharashtra, IND

10001+ employees

http://www.tcs.com

H1B Sponsorship

Tata Consultancy Services has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)

Distribution of Different Job Fields Receiving Sponsorship

Represents job field similar to this job

Trends of Total Sponsorships

2025 (7880)

2024 (9690)

2023 (8537)

2022 (11159)

2021 (9813)

2020 (11984)