ByteDance · 1 week ago
Software Engineer Intern (Inference Infrastructure) - 2026 Start (PHD)
United States
Internship
Remote
Intern
2+ years exp
Start in 2026ByteDance is a rapidly growing technology company known for its innovative products like TikTok. They are seeking a Software Engineer Intern to contribute to the Inference Infrastructure team, focusing on building large-scale, cloud-native systems for AI workloads and collaborating with world-class engineers.
ContentData MiningFoundational AIInternetSocial Media
Responsibilities
Design and build large-scale, container-based cluster management and orchestration systems with extreme performance, scalability, and resilience
Architect next-generation cloud-native GPU and AI accelerator infrastructure to deliver cost-efficient and secure ML platforms
Collaborate across teams to deliver world-class inference solutions using vLLM, SGLang, TensorRT-LLM, and other LLM engines
Stay current with the latest advances in open source (Kubernetes, Ray, etc.), AI/ML and LLM infrastructure, and systems research; integrate best practices into production systems
Write high-quality, production-ready code that is maintainable, testable, and scalable
Qualification
Required
B.S./M.S. in Computer Science, Computer Engineering, or related fields with 2+ years of relevant experience (Ph.D. with strong systems/ML publications also considered)
Strong understanding of large model inference, distributed and parallel systems, and/or high-performance networking systems
Hands-on experience building cloud or ML infrastructure in areas such as resource management, scheduling, request routing, monitoring, or orchestration
Solid knowledge of container and orchestration technologies (Docker, Kubernetes)
Proficiency in at least one major programming language (Go, Rust, Python, or C++)
Preferred
Experience contributing to or operating large-scale cluster management systems (e.g., Kubernetes, Ray)
Experience with workload scheduling, GPU orchestration, scaling, and isolation in production environments
Hands-on experience with GPU programming (CUDA) or inference engines (vLLM, SGLang, TensorRT-LLM)
Familiarity with public cloud providers (AWS, Azure, GCP) and their ML platforms (SageMaker, Azure ML, Vertex AI)
Strong knowledge of ML systems (Ray, DeepSpeed, PyTorch) and distributed training/inference platforms
Excellent communication skills and ability to collaborate across global, cross-functional teams
Passion for system efficiency, performance optimization, and open-source innovation
Benefits
Interns have day one access to health insurance
Life insurance
Wellbeing benefits and more
Interns also receive 10 paid holidays per year
Paid sick time (56 hours if hired in first half of year, 40 if hired in second half of year)
Interns who are not working 100% remote may also be eligible for housing allowance.
Company
ByteDance
ByteDance is a technology company that develops content creation platforms and services.
10001+ employees
H1B Sponsorship
ByteDance has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (1350)
2024 (1123)
2023 (775)
2022 (487)
2021 (417)
2020 (245)
Funding
Current Stage
Late StageTotal Funding
$9.8BKey Investors
Capital TodayG42Tiger Global Management
2025-11-20Secondary Market· $300M
2024-07-25Secondary Market
2023-03-14Secondary Market· $100M
Leadership Team
Jochen Bischoff
Head of Global Business Solutions - Africa
Matty Lin
General Manager, Global Business Solutions, KR
Recent News
2026-01-08
Company data provided by crunchbase