Centraprise ยท 8 hours ago
Data Architect (AI/ML and Databricks)
Centraprise is seeking a Data Architect with a strong focus on AI/ML and Databricks. The role involves designing and implementing cloud-native data architectures, with a deep understanding of Databricks, Apache Spark, and AI/ML pipeline integration.
Responsibilities
12+ years in data engineering or architecture, with a strong focus on Databricks (at least 4-5 years) and AI/ML enablement
Deep hands-on experience with Apache Spark, Delta Lake
Proficiency in AI/ML pipeline integration using Databricks MLflow or custom model deployment strategies
Strong knowledge of Apache Airflow, Databricks Jobs, and cloud-native orchestration patterns
Experience with structured streaming, Kafka, and real-time analytics frameworks
Proven ability to design and implement cloud-native data architectures
Solid understanding of data modeling, Lakehouse design principles, and lineage/tracking with Unity Catalog
Excellent communication and stakeholder engagement skills
Qualification
Required
12+ years in data engineering or architecture, with a strong focus on Databricks (at least 4-5 years) and AI/ML enablement
Deep hands-on experience with Apache Spark, Delta Lake
Proficiency in AI/ML pipeline integration using Databricks MLflow or custom model deployment strategies
Strong knowledge of Apache Airflow, Databricks Jobs, and cloud-native orchestration patterns
Experience with structured streaming, Kafka, and real-time analytics frameworks
Proven ability to design and implement cloud-native data architectures
Solid understanding of data modeling, Lakehouse design principles, and lineage/tracking with Unity Catalog
Excellent communication and stakeholder engagement skills
Preferred
Certification in Databricks Data Engineering Professional is highly desirable
Experience transitioning from in house data platforms to Databricks or cloud-native environments
Hands-on experience with Delta Lake, Unity Catalog, and performance tuning in Databricks
Expertise in Apache Airflow DAG design, dynamic workflows, and production troubleshooting
Experience with CI/CD pipelines, Infrastructure-as-Code (Terraform, ARM templates), and DevOps practices
Exposure to AI/ML model integration within real-time or batch data pipelines
Exposure to MLOps, MLflow, Feature Store, and model monitoring in production environments
Experience with LLM/GenAI enablement, vectorized data, embedding storage, and integration with Databricks is an added advantage
Company
Centraprise
Centraprise is an executive search and direct hire company.
H1B Sponsorship
Centraprise has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (34)
2024 (16)
2023 (49)
2022 (35)
2021 (42)
2020 (125)
Funding
Current Stage
Late StageLeadership Team
Company data provided by crunchbase