Apply on Employer Site

Visa · 12 hours ago

Data Engineer - Sr. Consultant

Bellevue, WA

Full-time

Hybrid

Senior Level, Lead/Staff

$152K/yr - $244K/yr

8+ years exp

Visa is a world leader in payments and technology, and they are seeking a Data Engineer to lead the architecture, modernization, and scaling of their enterprise data infrastructure. The role focuses on bridging existing SQL Server data warehouses with new Hadoop and Databricks platforms, ensuring optimized performance and seamless data flow to support AI-driven insights.

BankingFinanceFinancial ServicesInformation TechnologyMobile PaymentsPayments

Comp. & Benefits

No H1B

Responsibilities

Lead the transition from SQL Server data warehouse to the next-generation Hadoop/Databricks platform, ensuring performance, reliability, and minimal business disruption

Architect hybrid data pipelines that bridge legacy and modern ecosystems, enabling near real-time data access for analytics and AI applications

Optimize existing SQL Server models (facts, dimensions, indexes, stored procedures) and design modern equivalents in Hadoop and Spark environments

Define long-term migration strategy, data partitioning, and retention policies aligned with Visa’s data governance standards

Design and implement scalable, distributed data pipelines using Spark, Kafka, Airflow, and Delta Lake

Build robust ETL/ELT frameworks to process transactional, behavioral, and unstructured data at scale

Partner with the Agentic AI team to power RAG (Retrieval-Augmented Generation) pipelines, vector database integrations and LLM data provisioning

Lead proof-of-concept (POC) initiatives to evaluate and integrate new data engineering technologies

Provide guidance to junior Engineers to maintain existing SQL Server–based data warehouses, including database performance tuning, replication, encryption, and high-availability (HA) solutions

Implement best practices in T-SQL development, schema design, and stored procedure optimization

Perform proactive performance analysis, troubleshooting, and resolution of production issues in SQL Server and Hadoop clusters

Collaborate with DBA and application teams to ensure uptime within SLAs and compliance with Visa’s data policies

Define and enforce standards for data quality, lineage, and governance across both legacy and modern platforms

Ensure full compliance with Visa’s data privacy, encryption, and access control frameworks

Partner with InfoSec to embed data security principles into every phase of data lifecycle management

Mentor and coach data engineers, fostering an environment of technical excellence and innovation

Collaborate cross-functionally with product, ML, and platform engineering teams on architecture decisions and roadmap execution

Serve as a bridge between legacy enterprise data teams and next-gen AI platform engineers, ensuring knowledge continuity and execution speed

Qualification

HadoopSparkDatabricksETL/ELT frameworksSQL ServerPythonAWSAzureKafkaAirflowJavaScalaSQLT-SQLDockerKubernetesTerraformJenkinsData PrivacyData GovernanceData LineageLangChainLangGraph

Required

8 or more years of relevant work experience with a Bachelor's Degree or at least 5 years of experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) or 2 years of work experience with a PhD, OR 11+ years of relevant work experience

Preferred

9 or more years of relevant work experience with a Bachelor Degree or 7 or more relevant years of experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) or 3 or more years of experience with a PhD

Bachelor's or Master's degree in Computer Science, Data Engineering, or related field

Proven success supporting and modernizing SQL Server–based data warehouses in high-SLA environments

Production-level experience architecting Hadoop, Spark, and Databricks data pipelines

Expertise in ETL/ELT frameworks, data modeling, and schema design for analytical and operational use cases

Strong programming proficiency in Python, Java, or Scala

Hands-on experience with AWS or Azure (Glue, Synapse, Redshift, Delta Lake, S3)

Familiarity with Kafka, Airflow, Kubernetes, and containerized data services

Understanding of RAG pipelines, vector databases, and AI data flows

Experience designing data pipelines and APIs compatible with Model Context Protocol (MCP)-based agent frameworks, enabling seamless integration between AI agents, data services, and enterprise APIs

Strong SQL optimization, debugging, and production troubleshooting experience

Experience leading data warehouse migration or modernization from SQL Server to Hadoop/Spark ecosystems

Deep understanding of data lineage, metadata management, and governance frameworks (e.g., Atlas, Great Expectations)

Familiarity with LangChain, LangGraph, and MCP for integrating AI agents with data systems

Strong ability to balance innovation and stability across coexisting legacy and modern data architectures

Proven track record mentoring engineers and collaborating across global teams