Our Client is seeking a Report Developer / Azure Data Engineer to support enterprise data engineering, cloud integration, and analytics initiatives within the Ontario Data Platform (ODP).
The successful candidate will develop scalable data pipelines, optimize data processing workflows, and support enterprise analytics solutions using Microsoft Azure, Databricks, Python, SQL, and Spark.
Key Responsibilities
- Design, develop, and maintain batch and near real-time data pipelines using Azure Data Factory (ADF), Databricks, and Azure Data Lake Storage (ADLS).
- Build and optimize ETL/ELT processes for data ingestion, transformation, cleansing, and integration.
- Develop Databricks notebooks, workflows, and Spark-based data processing jobs.
- Implement Delta Lake and Lakehouse architecture, including Bronze, Silver, and Gold data layers.
- Create and maintain reliable data validation, reconciliation, monitoring, and quality-control processes.
- Optimize data storage, partitioning, and processing performance.
- Support data governance, metadata management, security, and access-control practices.
- Develop reusable Python scripts, automation utilities, and integration frameworks.
- Implement CI/CD pipelines, version control, automated testing, and deployment processes using Azure DevOps and Git.
- Collaborate with architects, analysts, developers, and business stakeholders to deliver enterprise data solutions.
- Support reporting, analytics, technical documentation, and Agile development activities.
Required Skills & Qualifications
- Hands-on experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and Azure Databricks.
- Strong proficiency in Python, SQL, Apache Spark, and PySpark.
- Experience with Delta Lake, Databricks Lakehouse, and Medallion Architecture.
- Knowledge of ETL/ELT development, data integration, and pipeline optimization.
- Experience with Azure DevOps, Git, CI/CD, and automated deployment.
- Understanding of data governance, data quality, privacy, and security principles.
- Experience working with REST APIs and enterprise integration patterns.
- Familiarity with Power BI and enterprise reporting environments.
- Strong communication, troubleshooting, and stakeholder collaboration skills.
- Public sector experience is an asset.
- Degree or diploma in Computer Science, Information Technology, Software Engineering, or a related discipline.
