Azure Databricks Cloud Migration & Platform Engineer
Position Type: Contract
Opportunity
We are seeking an experienced
Azure Databricks Cloud Migration & Platform Engineer to support a large-scale enterprise cloud migration and modernization initiative.
The ideal candidate will have strong hands-on experience in
Azure cloud infrastructure, Azure migration, platform engineering, and deploying Azure Databricks from the ground up in a greenfield enterprise environment.
This is a hands-on
Platform Engineering role, requiring someone who has personally deployed, configured, secured, and operationalized Azure Databricks environments, rather than simply developing data pipelines or working within an existing Databricks platform.
The successful candidate will collaborate with cloud architects, infrastructure engineers, security teams, and data engineers to build a secure, scalable, production-ready Azure Databricks platform while supporting broader Azure cloud migration activities.
Key Responsibilities
Azure Databricks Platform Engineering
- Deploy, configure, and operationalize Azure Databricks in a greenfield enterprise environment.
- Design and provision Databricks workspaces, compute, storage integrations, networking, identity, and security configurations.
- Configure Unity Catalog, workspace permissions, access controls, managed identities, and governance capabilities.
- Integrate Databricks with Azure Data Lake Storage Gen2 (ADLS Gen2), Azure Key Vault, Microsoft Entra ID, and other Azure services.
- Implement secure connectivity using Azure VNets, Private Endpoints, Private DNS, and network isolation.
- Support platform monitoring, troubleshooting, performance optimization, and production readiness.
Azure Cloud Migration & Infrastructure
- Support enterprise Azure cloud migration activities, including assessment, planning, deployment, validation, and post-migration support.
- Provision and configure Azure infrastructure within Azure Landing Zones.
- Identify and resolve infrastructure, networking, connectivity, and security dependencies.
- Support migration runbooks, deployment procedures, change management, and rollback planning.
- Troubleshoot Azure infrastructure and Databricks platform issues.
Infrastructure Automation & DevOps
- Develop and maintain Infrastructure-as-Code (IaC) using Terraform and/or Azure Bicep.
- Automate Azure infrastructure and Databricks workspace deployments.
- Develop deployment and configuration scripts using PowerShell, Bash, Python, and Azure CLI.
- Support CI/CD pipelines using Azure DevOps, GitHub Actions, or equivalent tools.
- Establish reusable deployment templates and enterprise platform engineering standards.
Collaboration & Delivery
- Work closely with cloud architects, infrastructure teams, security specialists, and data engineers.
- Participate in technical design, migration planning, deployment reviews, and troubleshooting.
- Document platform configurations, deployment procedures, and operational requirements.
- Support production deployment, knowledge transfer, and ongoing platform operations.
Required Skills & Experience
Must-Have Qualifications
- Proven hands-on experience deploying Azure Databricks from scratch in a GREENFIELD enterprise environment.
- Strong Azure Databricks Platform Engineering experience, including workspace deployment, configuration, security, networking, and production readiness.
- Experience with Azure cloud migration and enterprise Azure infrastructure deployments.
- 3+ years of hands-on Azure Cloud Engineering, Infrastructure Engineering, or Platform Engineering experience.
- Strong knowledge of Azure Landing Zones, Azure networking, identity, storage, and security.
- Experience integrating Databricks with ADLS Gen2, Azure Key Vault, Microsoft Entra ID, and Azure networking services.
- Experience with Databricks Unity Catalog, workspace administration, permissions, and access controls.
- Hands-on Infrastructure-as-Code experience with Terraform and/or Bicep.
- Knowledge of Azure VNets, subnets, NSGs, Private Endpoints, Private DNS, and routing.
- Experience with Azure CLI, PowerShell, Bash, or Python scripting.
- Familiarity with CI/CD pipelines, Azure DevOps, or GitHub Actions.
- Strong troubleshooting, documentation, and communication skills.
Nice-to-Have Skills
- Experience deploying Databricks across Development, Test, and Production environments.
- Databricks Terraform Provider, Databricks CLI, or Databricks Asset Bundles.
- Familiarity with Delta Lake, Apache Spark, PySpark, and enterprise lakehouse architecture.
- Experience with Azure Data Factory, Azure Synapse Analytics, or Microsoft Fabric.
- Knowledge of cloud governance, platform monitoring, cost optimization, and FinOps.
- Microsoft Azure or Databricks certifications.
Ideal Candidate Profile We are looking for a *
hands-on Azure Databricks Platform Engineer who has built and deployed enterprise Databricks environments from the ground up
,
