Verinext · 23 hours ago
Data Consultant (Databricks)
United States
Contract
Remote
Mid, Senior LevelVerinext has joined forces with Arctiq, expanding their team to tackle complex challenges in infrastructure and cloud solutions. They are seeking a Data Engineer to develop scalable data pipelines within the Databricks ecosystem, focusing on ETL/ELT processes and ensuring data governance and performance.
Cloud ComputingCyber SecurityConsultingInformation TechnologyIT InfrastructureIT ManagementNetwork Security
Responsibilities
Design and implement declarative, scalable pipelines using Lakeflow and Databricks Asset Bundles (DABs)
Establish reusable pipeline templates and patterns aligned with CI/CD best practices
Build a "configuration-as-code" approach for onboarding new sources and transformations quickly
Develop high-volume ingestion pipelines using Databricks AutoLoader
Implement CDC patterns (incremental loads, merges, deduping, late-arriving data)
Ensure ingestion is resilient, observable, and optimized for cost/performance
Configure and manage Unity Catalog for:
Metadata management
Access control / RBAC
Lineage and governance workflows
Help enforce standards for data quality, naming, and environment separation
Build and maintain complex workflows using Databricks Workflows / Jobs
Integrate with external orchestration tools when required
Improve operational reliability (retry logic, alerting, dependency handling)
Use Terraform to manage Databricks workspace resources and AWS components
Support AWS-aligned deployment patterns for:
S3 storage
Compute configuration
Workspace setup and environment parity
Qualification
Required
Strong hands-on experience with Databricks in production environments
Deep expertise in PySpark and advanced SQL
Experience with Delta Lake
Experience with ingestion pipelines (batch + streaming)
Experience with data transformation frameworks/patterns
Proven experience implementing CI/CD in Databricks
Proven experience implementing Databricks Asset Bundles (DABs)
Proven experience implementing declarative pipelines (Lakeflow)
Strong AWS infrastructure familiarity (S3, IAM, compute patterns)
Terraform experience specifically with Databricks + AWS resources
PowerShell scripting experience (asset)
Benefits
Retirement Plan (401k, IRA)
Work From Home
Health Care Plan
Company
Verinext
Verinext is an IT consultancy firm that offers networking, security, data protection, intelligent automation, and IT management services.
Founded in 1999
201-500 employees
H1B Sponsorship
Verinext has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2024 (1)
Funding
Current Stage
Growth StageTotal Funding
unknown2026-02-03Acquired
2019-01-01Private Equity
Leadership Team
Brian Glahn
Chief Executive Officer
Recent News
Business Wire
2026-02-04
2026-02-04
Company data provided by crunchbase