Duke University · 1 day ago
Developer - Cloud Data Engineer
Durham, NC, US, 27710
Full-time
Onsite
New Grad, Entry, Mid, Senior Level
0+ years expDuke University is dedicated to compassionate care and innovation in health technology solutions. The Data Engineer will be responsible for designing, developing, and optimizing data pipelines while collaborating with data scientists and analysts to ensure robust data management solutions.
EdTechEducationHigher EducationUniversities
Responsibilities
Individual contributor works under the direct supervision of level 3 or above and collaborates as a member of a team
Basic understanding of local databases (Clarity; Caboodle) and underlying tables pertaining to role/domain
Basic understanding of ETL technologies and scripting languages and terminology related to role
Basic understanding of cloud data engineering technologies, including storage accounts, data pipelines, data lakes, structured vs. unstructured data, etc
Ability to design, construct and test data management systems under guidance of senior team members
Assists in server administration tasks, ensuring system performance and availability
Basic knowledge of SQL and relational databases
Follows all policies and procedures set forth by DUHS/DHTS/ACE as pertains to role
Responds to customer requests for assistance as assigned and prioritized by manager
Participates in code review and learning best practices in data engineering
Maintains ongoing communications (timely, clear, concise, and accurate) with manager keeping them apprised of request status
Maintains appropriate project documentation as assigned such as GITLAB or data dictionary
Meets all Competencies in Level 1 AND:
Works with new hires/Level I team members to train on use of tools and technologies to accomplish job duties
Demonstrates advanced knowledge of the assigned domain/role. Understands data, workflows, and tools related to role
Demonstrates advanced knowledge of, and experience with, cloud data engineering concepts, including (but not limited to) Microsoft Azure Lakehouses and Delta Tables, Azure Data Factory, Azure Synapse, and medallion-style data architecture
Demonstrates ability to work with stakeholders to understand business needs and perform business analysis independently >75% of the time
Develop technical requirements from business requirements independently >75% of the time
Can write/modify accurate SQL queries independently > 90% of the time
Ability to optimize SQL queries, stored procedures and scripts for performance reliability
Can write complex SQL inclusion/exclusion criteria for cohort development
Manages and optimizes server infrastructure for performance and reliability
Troubleshoot and resolve complex data pipeline, server and API issues
Ability to locate data elements independently > 75% of the time
Knowledge/utilization of basic data design and infrastructure
Participates in team code review
Actively participates/contributes to database QA/submit requests for new database tables/attributes
Actively seeks out participation in Special Projects and is able to effectively accomplish project assignments with minimal supervision
Ability to manage multiple projects to completion in a timely manor
Ability to develop and present topics as assigned/needed by customer and/or ACE team
Active participation in Duke Analytic Community or other networking forums applicable to job functions
Meets all Competencies in Level 2 AND:
Demonstrates expertise in systems and software related to project area to accomplish job duties (e.g. SSIS, Azure Data Factory, etc..)
Resolves escalated issues/incidents to ensure high availability and performance of ETL processes
Architects and designs robust data solutions
Expert knowledge and experience with Microsoft Azure Data Engineering tools, concepts, and industry best practices
Expert locating data elements as required. Assists team in locating data elements as needed
Works with team members to train on advanced skills in using tools and technologies to accomplish job duties
Leads the development of complex data pipelines and API integration, ensuring scalability and efficiency
Defines standards and best practices for data pipeline design, development and maintenance
Demonstrates ability to work independently to solve problems and complete assigned work >95% of the time
Demonstrates ability to work with customers to understand business need or problem to address independently >95% of the time
Leads Projects with minimal supervision
Change agent with regard to team processes, workflows when issues/gaps are found
Collaborates with data architect on design and implementation of data solutions (lakehouse, data warehouse, etc.)
Serves as moderator, leader or channel owner for a forum such as Duke Analytics Community or other professional networking organization
Qualification
Required
Bachelor's degree in a related field, or four years of equivalent technical experience required
Level 1 - No experience required beyond the minimum education (or equivalency) requirement
Level 2 - Two years of directly related experience is required
Level 3 - Four years of directly related experience is required
Basic understanding of local databases (Clarity; Caboodle) and underlying tables pertaining to role/domain
Basic understanding of ETL technologies and scripting languages and terminology related to role
Basic understanding of cloud data engineering technologies, including storage accounts, data pipelines, data lakes, structured vs. unstructured data, etc
Ability to design, construct and test data management systems under guidance of senior team members
Assists in server administration tasks, ensuring system performance and availability
Basic knowledge of SQL and relational databases
Follows all policies and procedures set forth by DUHS/DHTS/ACE as pertains to role
Responds to customer requests for assistance as assigned and prioritized by manager
Participates in code review and learning best practices in data engineering
Maintains ongoing communications (timely, clear, concise, and accurate) with manager keeping them apprised of request status
Maintains appropriate project documentation as assigned such as GITLAB or data dictionary
Preferred
Preferred certification in development technologies, as appropriate for the position
Company
Duke University
Duke University is a private research university providing undergraduate and postgraduate education in medicine and other disciplines.
10001+ employees
H1B Sponsorship
Duke University has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (274)
2024 (232)
2023 (202)
2022 (195)
2021 (148)
2020 (121)
Funding
Current Stage
Late StageTotal Funding
$71.15MKey Investors
North Carolina Biotechnology CenterUS Department of EnergyGlaucoma Research Foundation
2023-02-17Grant· $0.11M
2022-09-19Grant
2022-03-09Grant· $0.05M
Leadership Team
Barry Myers
Professor and Director
Dan Ariely
Professor
Recent News
2026-01-11
2025-12-18
Company data provided by crunchbase