Commence ยท 8 hours ago
Lead Enterprise Data Engineer
Baltimore County, MD
Full-time
Onsite
Senior Level, Lead/Staff
7+ years expCommence is at the forefront of data-centric transformation in healthcare, combining quality data-driven solutions with clinical expertise. The Lead Enterprise Data Engineer will design, build, and operate the enterprise data platform, ensuring compliance and enabling analytics and AI/ML use cases across various environments.
AnalyticsArtificial Intelligence (AI)Clinical TrialsCloud ComputingHealth CareSoftware
Responsibilities
Administer and manage Databricks workspaces, clusters, and security configuration
Design and implement scalable ingestion pipelines (batch and streaming) into Delta Lake
Establish CI/CD practices for data pipelines and implement cost monitoring/optimization
Ensure alignment with FedRAMP, HIPAA, and FISMA security and compliance requirements
Partner with DevOps to implement infrastructure-as-code and automation practices
Collaborate with stakeholders (engineering, backend, DevOps, product) to identify platform gaps and design scalable, long-term solutions that unlock product innovation
Provide technical leadership and management of data engineers, fostering technical growth, improving team practices, and elevating the quality of engineering across the board
Collaborate with Analytics and Data Science leads to deliver trusted, reusable data assets
Ensure operational excellence by automating infrastructure, monitoring key systems, and leading incident response and root cause analysis for platform-related issues
Evaluate and introduce new technologies and practices (e.g., container orchestration, data lake solutions, workflow orchestration tools) based on evolving business needs
Define and enforce platform best practices for coding standards, testing, monitoring, and documentation
Perform other duties as assigned
Qualification
Required
7+ years of experience in data engineering, platform architecture, or related roles
Hands-on expertise with using and administering Databricks, Spark, Delta Lake, and orchestration tools (Airflow, Databricks Workflows, etc.)
Demonstrated experience in implementing Data Lakes/ Lakehouse architecture using Databricks and AWS technologies
Strong knowledge of AWS services (incl. GovCloud), IAM, VPC, and network security
Experience with Unity Catalog or equivalent governance/cataloging tools
Familiarity with regulated environments (HIPAA, FedRAMP, FISMA)
Demonstrated ability to design secure, cost-effective, and scalable data platforms
Strong problem-solving skills, attention to detail, and a passion for continuous learning
Experience working in a healthcare or regulated environment
Preferred
Familiarity with modern data frameworks like Data Fabric/ Mesh is a plus
Company
Commence
Commence delivers AI-driven healthcare data platform and clinical expertise that supports analytics, decisions, and workflow improvement.
501-1000 employees
Funding
Current Stage
Late StageCompany data provided by crunchbase