Tata Consultancy Services · 13 hours ago
Pyspark Developer
Jersey City, NJ
Full-time
Onsite
Mid Level
$110K/yr - $130K/yrTata Consultancy Services is a leading IT services provider, and they are seeking a Pyspark Developer to design and maintain data pipelines. The role involves optimizing PySpark applications, ensuring data quality, and collaborating with cross-functional teams to support data-driven decision-making.
Business Information SystemsConsultingInformation TechnologyIT Management
Responsibilities
Design, develop, and maintain scalable data pipelines using PySpark for ETL processes, integrating multiple data sources, transforming datasets, and loading into target systems
Optimize PySpark applications and Spark jobs for efficiency, fine-tuning configurations, and ensuring high performance with large-scale datasets
Implement validation rules, error-handling mechanisms, and monitoring frameworks to ensure accuracy, consistency, and integrity of data throughout its lifecycle
Partner with data engineers, data scientists, and business analysts to translate business requirements into technical solutions that support data-driven decision-making
Write clean, efficient, and well-documented PySpark code, follow coding standards, and actively participate in peer code reviews
Monitor Spark jobs, identify and resolve issues, and provide production support for PySpark-based applications
Qualification
Required
Strong hands-on experience with Apache Spark and PySpark for large-scale data processing
Proficiency in Python programming with focus on writing optimized, modular, and reusable code
Solid understanding of ETL processes, data pipeline design, and data integration techniques
Experience in performance tuning and optimization of Spark jobs in distributed computing environments
Good knowledge of SQL and working with relational as well as NoSQL databases
Familiarity with big data ecosystems (e.g., Hadoop, Hive, HDFS) and data warehouses (e.g., Snowflake, Redshift)
Understanding of data quality, validation, and error-handling frameworks
Strong problem-solving skills and ability to troubleshoot production issues
Good communication and collaboration skills to work with cross-functional teams
BACHELOR OF COMPUTER SCIENCE
Preferred
Exposure to cloud platforms (AWS, Azure, or GCP) and their data services is a plus
Benefits
Discretionary Annual Incentive.
Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
Family Support: Maternal & Parental Leaves.
Insurance Options: Auto & Home Insurance, Identity Theft Protection.
Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
Time Off: Vacation, Time Off, Sick Leave & Holidays.
Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.
Company
Tata Consultancy Services
Tata Consultancy Services is a business solutions company that specializes on information technology services and consulting.
Founded in 1968
10001+ employees
H1B Sponsorship
Tata Consultancy Services has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (7880)
2024 (9690)
2023 (8537)
2022 (11159)
2021 (9813)
2020 (11984)
Funding
Current Stage
Public CompanyTotal Funding
unknown2004-08-25IPO
Leadership Team
K. Krithivasan
Chief Executive Officer & Managing Director
Aarthi Subramanian
President and Chief Operating Officer
Recent News
Hindu Business Line
2026-01-16
TradingView
2026-01-16
Company data provided by crunchbase