Boson AI · 2 weeks ago
Member of Technical Staff - Agent Platform (Agent OS)
Santa Clara, CA
Full-time
Onsite
Mid Level
$150K/yr - $400K/yr
3+ years expBoson AI is pioneering the future of enterprise AI by developing cutting-edge AI solutions. The role involves engineering and evolving the core Agent OS, focusing on building high-performance systems that integrate complex agentic orchestration and dialog management functionalities.
Artificial Intelligence (AI)Information TechnologyMarket Research
Responsibilities
System Ownership: Take ownership of the core dialog & policy engine. Define and implement the state machine for agent state representation, the decision-making logic, and the mechanisms for enforcing complex safety policies and guardrails at the execution layer of a workflow
Distributed Context & Memory: Design, implement, and maintain the high-performance context and memory systems. Focus on low-latency, reliable access to conversational and user history, including the tight integration and optimization of RAG and vector retrieval pipelines for production use
Agentic Orchestration Frameworks: Define, architect, and deliver robust agentic orchestration patterns, including battle-tested planner–executor schemes, ReAct-style reasoning and acting loops, and resilient, multi-step workflows that programmatically combine tools, LLMs, and stateful memory
Internal SDK/Framework Development: Build and evolve the internal, production-grade equivalent of frameworks like LangChain/LlamaIndex. Design composable graphs and execution chains with clear APIs and type safety that product engineering teams and low-code builders can safely reuse, extend, and deploy at scale
Voice Runtime Infrastructure: Own and optimize the voice runtime components for streaming audio, low-latency barge-in detection, and reliable turn-taking protocols. This requires deep collaboration with Application and ML Platform teams to meet tight latency, jitter, and quality of service (QoS) constraints
Tooling & Integration Architecture: Architect a robust, secure tooling and integration framework (MCP/A2A). This includes building the underlying infrastructure for tool registration, handling complex authentication/authorization, implementing rate limiting/circuit breaking, managing retries, and ensuring typed, validated I/O between agents and external microservices
Platform Observability & Reliability: Define, instrument, and monitor rigorous SLIs/SLOs for the Agent Platform. Lead engineering efforts to continuously improve reliability, enhance system debuggability (rich, step-level traces and structured logging), and drive core performance optimizations over time
API & Abstraction Design: Ensure the platform's public-facing APIs and internal abstractions are clear, well-documented, and fundamentally sound, enabling junior and senior engineers alike to compose sophisticated agent behavior without introducing systemic invariants or breaking changes
Advanced Capabilities R&D: Explore and prototype future capabilities, focusing on the engineering challenges of on-device personalization, implementing privacy-preserving federated learning signals, or integrating novel policy adaptation techniques that influence agent behavior in production
Qualification
Required
3+ years of hands-on experience in backend engineering and distributed systems, with a track record of building and owning core platforms or frameworks used successfully by other engineering teams
Demonstrated, hands-on experience architecting, building, or operating production-grade agentic systems: orchestrating LLM calls, managing complex tool interactions, and defining stateful workflows—moving beyond simple single prompt/response API integrations
Strong working knowledge of engineering orchestration frameworks (e.g., LangChain, LlamaIndex, or internal equivalents) and a deep understanding of core design patterns like RAG, ReAct, and multi-step planning
Deep and practical understanding of distributed system design, concurrent programming, and building for reliability in multi-tenant cloud environments with strictly defined latency and cost envelopes
Proven experience designing, implementing, and rolling out successful frameworks or libraries that other internal engineering teams enthusiastically adopt and productively build upon
Comfort and prior experience working on security-sensitive systems, including implementing authz/authn schemes, isolation boundaries, data protection protocols, and integrating with centralized policy/safety infrastructure
Strong technical communication skills and the ability to lead complex, cross-functional technical initiatives, driving consensus and influencing architectural decisions across partner teams
Preferred
Experience developing and operating conversational AI platforms, agent frameworks, or high-throughput, complex workflow engines in a production setting
Engineering background in real-time media (audio/video) systems or low-level signaling protocols where extreme low-latency and jitter management are critical performance factors
Prior experience building high-stakes enterprise platforms (e.g., payments, identity, core data services) where correctness, auditability, and absolute reliability are non-negotiable requirements
Exposure to emerging systems and engineering techniques, such as integrating federated learning models, enabling on-device personalization, or implementing bandit-style adaptive policy systems
Benefits
401k plan
HSA
FSA
Free food
Company
Boson AI
Boson AI is an AI company that develops large language model tools.
11-50 employees
H1B Sponsorship
Boson AI has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (4)
2024 (7)
2023 (2)
Funding
Current Stage
Early StageLeadership Team
Alex Smola
CEO & Cofounder
Mu Li
Co-Founder & CTO
Company data provided by crunchbase