Apply on Employer Site

Kamet Consulting Group · 2 days ago

Agentic / Applied AI Engineer

San Francisco Bay Area

Full-time

Hybrid

Senior Level

5+ years exp

Kamet Consulting Group is looking for an Agentic / Applied AI Engineer to help build the core intelligence layer of their platform. The role involves designing and implementing agentic workflows, ensuring reliability and evaluation of AI systems, and collaborating with product and engineering teams to deliver features.

Management Consulting

Responsibilities

Design multi-step agentic workflows using patterns like Plan → Execute → Validate → Retry

Implement tool calling systems with strong contracts (typed inputs/outputs)

Build safety mechanisms: timeouts, retries, fallbacks, confidence checks, guardrails

Implement retrieval pipelines (vector + hybrid search) with chunking, metadata, filters, and ranking

Ensure outputs are traceable and explainable, with citations and provenance where needed

Build evaluation pipelines (offline + online) to measure: correctness, completeness, consistency, and failure modes

Grounding quality (is it supported by sources?)

Cost and latency tradeoffs

Create regression tests for agent behavior as prompts/models/tools evolve

Work closely with product + full-stack engineers to ship agentic features end-to-end

Optimize for cost, latency, and scalability (caching, batching, model routing)

Instrument agent traces, logs, and telemetry for debugging and iteration

Qualification

PythonLLM applicationsProduction systemsReliability engineeringFastAPIData pipelinesVector databasesHybrid searchSoft skills

Required

5+ years of software engineering experience (Python-heavy preferred)

Strong hands-on experience building production systems with: Python (FastAPI preferred), data pipelines, async workflows

LLM applications using OpenAI / Anthropic / Azure OpenAI / open-source models

structured outputs (JSON schema), validation, parsing, error recovery

Solid understanding of: retrieval, ranking, embeddings, vector databases

agent tool calling patterns + prompt orchestration

reliability engineering for non-deterministic systems

Preferred

Experience with LangGraph, LangChain, LlamaIndex, or custom orchestration frameworks

Experience implementing hybrid search (BM25 + vectors), rerankers

Vector DBs (Pinecone, Weaviate, FAISS, pgvector, Milvus)

Exposure to enterprise workflows or regulated domains (healthcare, pharma, fintech)

Comfort with 'systems thinking': state machines, DAG workflows, Temporal/Airflow-style patterns

Experience with eval tooling: Ragas, promptfoo, DeepEval, LangSmith, or custom test harnesses

Benefits

High ownership, strong growth, and direct product impact

Flexible work setup (hybrid preferred)

Company

Kamet Consulting Group

Kamet Consulting Group provides Strategic Advisory, Implementation, Capability Building, and Outsourced Services to Life Sciences, Food, Digital Health, and Consumer Health companies across the value chain.

San Francisco, California, US

11-50 employees

https://www.kametcg.com

Funding

Current Stage

Early Stage

Leadership Team

Megha Sinha

Principal and CEO

Company data provided by crunchbase