We are looking for a skilled Data Engineer to join our team in Kitchener, Ontario. In this role, you will design, build, and optimize data solutions to support business objectives, leveraging modern technologies and frameworks. You will work closely with cross-functional teams to implement robust data infrastructure and ensure seamless data integration across platforms.<br><br>Responsibilities:<br>• Develop, implement, and maintain scalable data pipelines using Databricks, Python, and Spark.<br>• Design and optimize data models and architectures, including Medallion Architecture, for both batch and streaming data processing.<br>• Integrate Azure Cloud components such as Azure Data Factory, Key Vault, and Blob Storage into data workflows.<br>• Collaborate with stakeholders to understand data requirements and deliver solutions that align with business goals.<br>• Utilize advanced data manipulation techniques to manage complex data structures and ensure high performance.<br>• Implement coding best practices, testing methodologies, and CI/CD pipelines within a DevOps framework.<br>• Participate in Agile or Spotify framework processes to ensure efficient project execution.<br>• Monitor and troubleshoot data workflows to maintain system reliability and integrity.<br>• Explore opportunities for incorporating Machine Learning and Artificial Intelligence concepts into data solutions.<br>• Provide documentation and training to support the use of developed data systems.
