AMD · 1 week ago
Senior Software Development Engineer – LLM Inference Framework
Santa Clara, CA
Full-time
Onsite
Senior Level
$192K/yr - $288K/yrAMD is focused on building great products that accelerate next-generation computing experiences. The Senior Software Development Engineer will be responsible for building and optimizing production-grade inference runtimes for large language models on AMD GPUs, driving performance, scalability, and reliability.
AI InfrastructureArtificial Intelligence (AI)Cloud ComputingComputerEmbedded SystemsGPUHardwareSemiconductor
Responsibilities
Architect and optimize distributed LLM inference runtimes based on in-house LLM engines or open-source stacks such as vLLM, SGLang, and llm-d
Design and improve TP / PP / EP (MoE) hybrid execution, including KV-cache management, attention dispatch, and token scheduling
Implement and optimize multi-node inference pipelines using RCCL, RDMA, and collective-based execution
Drive throughput, latency, and memory efficiency across single-GPU and multi-GPU clusters
Optimize continuous batching, speculative decoding, KV-cache paging, prefix caching, and multi-turn serving
Work with AMD GPU libraries (AITER, HIPBLAS-LT, RCCL, ROCm runtime) to ensure inference frameworks efficiently use FP8 / FP4 GEMM and FlashAttention / MLA
Collaborate with compiler teams (Triton, LLVM, ROCm) to unblock framework-level performance
Upstream features and performance fixes into vLLM, SGLang, and llm-d
Enable customer PoCs and production deployments on AMD platforms
Build and maintain benchmark-grade inference pipelines
Qualification
Required
Hands-on understanding of vLLM, SGLang, or similar inference stacks
Experience with distributed inference scaling and a proven track record of contributing to upstream open-source projects
Strong experience integrating optimized GPU performance into machine-learning frameworks (e.g., PyTorch, TensorFlow) for high-throughput and scalable inference
Strong background in NVIDIA, AMD, or similar GPU architectures and kernel development
Expertise in Python and preferably experience in C/C++, including debugging, performance tuning, and test design for large-scale systems
Experience running large-scale workloads on heterogeneous GPU clusters, optimizing for efficiency and scalability
Understanding of compiler and runtime systems, including LLVM, ROCm, and GPU code generation
Master's or PhD in Computer Science, Computer Engineering, Electrical Engineering, or a related field
Benefits
AMD benefits at a glance.
Company
AMD
Advanced Micro Devices is a semiconductor company that designs and develops graphics units, processors, and media solutions.
Founded in 1969
10001+ employees
H1B Sponsorship
AMD has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (836)
2024 (770)
2023 (551)
2022 (739)
2021 (519)
2020 (547)
Funding
Current Stage
Public CompanyTotal Funding
unknownKey Investors
OpenAIDaniel Loeb
2025-10-06Post Ipo Equity
2023-03-02Post Ipo Equity
2021-06-29Post Ipo Equity
Leadership Team
Lisa Su
Chair & CEO
Mark Papermaster
CTO and EVP
Recent News
2026-01-19
2026-01-18
2026-01-17
Company data provided by crunchbase