Description

We are seeking an experienced Databricks Platform Administrator to manage, secure, optimize, and support enterprise Databricks environments across cloud platforms. The ideal candidate will be responsible for workspace administration, cluster management, security governance, cloud integrations, platform monitoring, automation, and operational support. This role requires strong expertise in Azure Databricks (preferred), along with exposure to AWS and GCP Databricks environments.

Required Qualifications:

• Candidates authorized to work for any employer in Canada without employer-based visa sponsorship are welcome to apply. Infosys is unable to provide immigration sponsorship for this role at this time.
• Candidate must be located within commuting distance of Calgary, Alberta or be willing to relocate to the areas.
• Bachelor's degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.
• At least 4 years of Information Technology experience

Key Responsibilities
Platform Administration

• Administer and manage Databricks workspaces across development, test, and production environments.
• Configure and maintain Databricks workspaces, metastore, catalogs, clusters, pools, and policies.
• Manage Unity Catalog implementation, governance, and access controls.
• Perform platform upgrades, maintenance, and health checks.
• Support multi-workspace and multi-region deployments.

Cluster Management

• Create, configure, and manage all-purpose and job clusters.
• Implement cluster policies to control costs and ensure compliance.
• Optimize cluster sizing and auto-scaling configurations.
• Monitor cluster utilization, performance, and cost efficiency.
• Troubleshoot cluster startup, termination, and execution issues.

Security & Governance

• Implement RBAC and workspace access controls.
• Integrate Databricks with Azure AD, AWS IAM, or GCP Identity services.
• Manage service principals, managed identities, and secret scopes.
• Implement Unity Catalog governance and data security policies.
• Support audit logging, compliance, and platform security reviews.

Cloud Integration

• Integrate Databricks with:
  • Azure Data Lake Storage (ADLS)
  • AWS S3
  • Azure Key Vault
  • AWS Secrets Manager
  • SQL Databases
  • Event Hub
  • Kafka
  • Snowflake
  • Power BI

Monitoring & Performance

• Monitor platform health and resource utilization.
• Establish proactive alerting and monitoring mechanisms.
• Analyze Spark job performance and recommend optimizations.
• Troubleshoot platform incidents and performance bottlenecks.
• Support SLA-driven production environments.

DevOps & Automation

• Automate Databricks administration using:
  • Terraform
  • Azure DevOps
  • GitHub Actions
  • Databricks CLI
  • REST APIs
• Manage CI/CD pipelines for Databricks deployments.
• Support Infrastructure as Code (IaC) initiatives.

Operational Support

• Participate in incident, problem, and change management activities.
• Support patching, upgrades, certificate renewals, and maintenance activities.
• Create and maintain SOPs, runbooks, and technical documentation.
• Collaborate with engineering, infrastructure, security, and application teams.

Required Skills
Databricks

• Databricks Workspace Administration
• Unity Catalog
• Cluster Management
• Cluster Policies
• Instance Pools
• Jobs & Workflows
• SQL Warehouses
• Databricks CLI
• REST APIs
• Delta Lake

Cloud Platforms

• Azure Databricks (Preferred)
• AWS Databricks
• GCP Databricks

Preferred Skill and Experience

• Experience with enterprise-scale Azure Databricks environments.
• Strong understanding of data lake architecture and governance frameworks.
• Experience supporting production environments with strict SLA requirements.

Soft Skills
Strong analytical and troubleshooting skills.
Excellent communication and stakeholder coordination.
Ability to work in a global delivery model.
Documentation and knowledge ‑ sharing mindset.

Experience

• 8-10 years of experience in IT Industry
• 5+ years of Databricks Administration experience.
• Prior experience in large enterprise or managed services engagements preferred.
• Exposure to Infosys‑scale delivery environments is an advantage.

The job entails sitting as well as working at a computer for extended periods of time. Should be able to communicate by telephone, email or face to face. Travel may be required as per the job requirements