ECS · 11 hours ago
Data Pipeline Technical Lead
Fairfax, VA
Full-time
Hybrid
Lead/Staff
10+ years expECS is a leading mid-sized provider of technology services to the United States Federal Government, and they are seeking a highly skilled Data Pipeline Technical Lead. This role involves designing, building, and maintaining large-scale, real-time data pipelines while leading a team of engineers to ensure the product's reliability, scalability, and performance.
Artificial Intelligence (AI)Cloud InfrastructureComplianceConsultingCyber SecurityInformation TechnologyMachine LearningSecuritySoftware
No H1B
U.S. Citizen Only
Responsibilities
Accountable for all technical facets in the delivery of our data pipeline product
Lead the architecture, design, development, and delivery of our data pipeline product, which consists of a team of 20+ members
Build and maintain microservices and data connectors that ingest, process, and distribute data across multiple systems and APIs
Ensure reliability, scalability, and performance of data pipelines through best practices in resiliency, fault tolerance, and eventual consistency
Optimize data processing and enrichment leveraging caching layers like Redis, ElastiCache, or Valkey
Guide data modeling decisions and define and enforce data format standards (JSON, Avro, Parquet), while leveraging a schema registry
Define technical work necessary for cross-functional team members (DevSecOps, Site Reliability Engineering, Security, Testing, Infrastructure) to ensure successful delivery of the data pipeline product
Work within a SAFe Agile framework to plan work, decompose Epics, develop Stories, and task team members appropriately
Represent the data pipeline team on a broader level within the program, whether that is across internal teams, with our customer and other stakeholders, provide demos, deliver product status, or generally answering technical questions about the data pipeline
Provide mentorship to team members to strengthen their skills, knowledge, and capabilities
Qualification
Required
Must be a US Citizen and able to acquire DHS Public Trust Suitability
10+ years of hands-on technical experience as a software developer
6+ years experience leading engineering teams of at least 10 members, balancing technical direction with mentorship and skills development
6+ years experience focused specifically on data engineering or data pipeline solutions and concepts
6+ years developing microservice based applications utilizing components such as containers, Docker, Kubernetes, AWS ECR/ECS/EKS/Fargate, and container registries
6+ years leveraging git repositories (Github, Gitlab, Bitbucket), with an advanced understanding of branching strategies, pull requests, commits, and other key concepts
Expert level Python experience, to include reviewing code commits and coaching developers on how to write better code
Demonstrated solution architect experience in designing highly performant, reliable, and scalable data pipeline solutions that include batch and stream processing concepts
Experience with stream processing technologies such as Kafka, Kafka Streams, Flink, or Spark
Proficiency working with diverse data formats (JSON, Avro, Parquet), and how a schema registry, schema evolution, and semantic validation can all work together to provide a mature solution
Experience developing custom data connectors and integrations with REST APIs
Expertise implementing and tuning pipeline caching layers (Redis, ElastiCache, Valkey)
Experience designing and implementing data pipeline solutions that leverage cloud native services, particularly in AWS (Step Functions, EventBridge, Lambda, SNS, SQS, S3, etc.)
Understanding of resiliency, idempotency, and eventual consistency in distributed pipelines
Strong analytical and problem-solving skills, with attention to detail
Preferred
Experience with IaC (Infrastructure as Code) (Terraform, CloudFormation)
Exposure to CI/CD pipelines and data observability tools (Elasticsearch, Prometheus, Grafana, OpenTelemetry)
Bachelor's degree in Computer Science, Engineering, or a related field
Experience using ALM tools (Jira, Confluence, or similar)
Expertise in configuration management tools (Ansible, Puppet, Chef)
In-depth understanding of networking, security, and system administration of Linux operating systems
Experience supporting large Federal programs ($200M+)
Company
ECS
ECS is a fast-growing 4,000-person, $1.2B provider of advanced technology solutions for federal civilian, defense, intelligence, and commercial customers.
1001-5000 employees
Funding
Current Stage
Late StageTotal Funding
unknown2018-01-31Acquired
2015-04-10Private Equity
Leadership Team
Keith McCloskey
VP / Chief Technology Officer
Ryan Garner
Chief Financial Officer
Recent News
2025-11-19
2025-03-26
2025-02-14
Company data provided by crunchbase