IntraEdge ยท 12 hours ago
Data Engineer
IntraEdge is seeking a skilled Data Engineer to join their team in the transportation sector. The role involves building and maintaining data pipelines using modern cloud technologies to support analytics and operational insights.
Responsibilities
Design, develop, and maintain cloud-native data pipelines leveraging Databricks, Microsoft Azure Data Factory, and Microsoft Fabric to support robust data integration and analytics solutions
Implement incremental and real-time data ingestion strategies using medallion architecture for data lake storage
Write and optimize complex SQL queries to transform, integrate, and analyze data across enterprise systems
Support and troubleshoot legacy data platforms built on SSIS and SQL Server, ensuring high availability and performance of critical data processes
Develop features with a focus on scalability, maintainability, and testability
Troubleshoot and resolve data integration and quality issues, ensuring reliable data delivery
Participate in proof-of-concept projects, providing technical analysis and recommendations
Qualification
Required
5+ years of experience designing and building data solutions
Strong proficiency in SQL and Python for data analytics and transformation
Experience with ETL pipeline development and automation
Solid understanding of Data Lake architecture and design principles
Excellent collaboration skills and the ability to adapt in a dynamic environment
Preferred
Experience with Azure Cloud services and cloud-based ETL tools
Familiarity with data visualization tools such as Power BI or Tableau
Understanding of event-driven architectures, including queues, batch processing, and pub/sub models
Exposure to NoSQL databases like MongoDB or Cassandra