Pacific Northwest National Laboratory · 6 hours ago
Senior Azure Databricks Engineer
United States
Full-time
Remote
Senior Level
$161K/yr - $255K/yr
5+ years expPacific Northwest National Laboratory (PNNL) is a world-class research institution focused on scientific research and innovation. They are seeking a Senior Azure Databricks Engineer to lead the design and operation of their data lakehouse, enabling analytics and reporting across enterprise systems while ensuring data quality and security.
EnergyGovernmentRenewable EnergyBiomass EnergyProduct ResearchSecurity
No H1B
U.S. Citizen Only
Responsibilities
Lead the design, build, and operation of our data lakehouse that powers analytics and reporting across PNNL Enterprise Systems
Deliver governed, performant, and reliable data products—especially for ERP and other enterprise—and enable self-service analytics with Power BI and AI/ML
Design and evolve a Databricks‑based architecture that moves data with confidence from source systems to curated gold tables
Build robust pipelines that transform raw data into analytics‑ready assets for Power BI and AI/ML, balancing pragmatic MVP delivery with a future‑focused architecture
Lead modernization from legacy data warehouses and ETL tools into Azure Databricks—refactoring brittle jobs into scalable patterns
Shape CI/CD for Databricks (e.g., DAB, Azure DevOps, GitHub Actions) and standardize deployment practices across environments
Configure and operate workspaces, clusters, jobs, and workflows; tune for performance and reliability; and embed data quality, monitoring, and observability to keep critical pipelines healthy
Implement role‑based access controls, data masking, and fine‑grained models with Unity Catalog to enable secure, compliant data sharing and ensure proper classification, lineage, and auditability
Guide engineers and analysts in adopting lakehouse best practices and modern data engineering standards—coding, testing, version control, and documentation
Stay current with Azure and Databricks capabilities, recommending and piloting features like Delta Live Tables and Unity Catalog enhancements to build a secure, reliable, and future‑ready data platform that accelerates science and mission delivery
Qualification
Required
PhD and 3 years of relevant experience -OR-
MS/MA or higher and 5 years of relevant experience -OR-
BS/BA and 7 years of relevant experience -OR-
AA and 16 years of relevant experience -OR-
HS/GED and 18 years of relevant experience
Qualifying software development experience in designing, architecting, programming, deploying, and automating software solutions in support of scientific research or consumer digital product development may be counted
Preferred
7+ years of professional data engineering or platform engineering experience, with 3–5+ years focused on cloud data platforms
5+ years of experience operating production Azure Databricks, including Delta Lake, SQL, notebooks, Jobs/Workflows, and cluster management
Production experience (3-5+ years) designing and operating ingestion-to-gold pipelines (medallion architecture) for ERP or other complex transactional sources
Experience with Azure Data Factory and/or Fabric Data Pipelines for orchestration and integration
Familiarity with core Azure services: ADLS Gen2, Key Vault, Azure DevOps or GitHub
Strong proficiency in Python and SQL in a Spark/Databricks environment
Experience implementing Databricks Asset Bundles (DAB) or equivalent for CI/CD and standardizing deployment workflows
Experience using GenAI / LLM-based tools (e.g., GitHub Copilot, Azure OpenAI, Databricks Genie, or similar) to accelerate and automate engineering tasks such as code generation, test creation, documentation, and troubleshooting
Exposure to agentic AI / AI agents (e.g., orchestrating multi-step AI workflows for data quality checks, pipeline monitoring, or support automation) is a plus
Benefits
Medical insurance
Dental insurance
Vision insurance
Robust telehealth care options
Several mental health benefits
Free wellness coaching
Health savings account
Flexible spending accounts
Basic life insurance
Disability insurance
Employee assistance program
Business travel insurance
Tuition assistance
Relocation
Backup childcare
Legal benefits
Supplemental parental bonding leave
Surrogacy and adoption assistance
Fertility support
Company-funded pension plan
401 (k) savings plan with company match
Up to 120 vacation hours per year
Ten paid holidays per year
Company
Pacific Northwest National Laboratory
Pacific Northwest National Laboratory operates as a government research laboratory.
5001-10000 employees
Funding
Current Stage
Late StageTotal Funding
unknownKey Investors
US Department of Energy
2022-07-14Grant
2018-07-16Grant
Leadership Team
Allison Campbell
Associate Laboratory Director for Earth and Biological Sciences
Deborah Gracio
Laboratory Director
Recent News
Inside HPC & AI News | High-Performance Computing & Artificial Intelligence
2026-01-11
Pacific Northwest National Laboratory
2026-01-03
Company data provided by crunchbase