Job Description

Databricks Platform Administration  

• Deploy, configure, and maintain  Azure Databricks workspaces
• Manage  clusters ,  pools , and  jobs for optimal performance.
• Handle  Databricks Runtime upgrades and compatibility testing.
• Configure  Databricks notebooks, Repos, and Delta Lake for efficient workflows.

Security & Access Control:

• Implement  Identity and Access Management (IAM)using  Azure AD / Azure IAM / SCIM .
• Set up and enforce  Role-Based Access Control (RBAC)and  network security .
• Ensure compliance with  data governance policies (Unity Catalog, creation of Catalogs, schema etc.).
• Applying grants of catalogs and schemas
• Manage  personal access tokens (PATs)and  OAuth-based authentication .

Performance Monitoring & Optimization:

• Monitor  cluster performance, auto-scaling, and cost optimization .
• Analyze and fine-tune  Spark jobs to improve execution efficiency.
• Use  Databricks SQL for querying and performance diagnostics.

Integration & Automation:

• Integrate Databricks with  Azure Data Factory, Synapse, Snowflake, Kafka, and other data sources .
• Automate cluster management using  Terraform.
• Implement CI/CD pipelines for  Databricks notebooks using  GitHub or Azure DevOps .

Troubleshooting & Support:

• Investigate and resolve  job failures, cluster crashes, and query performance issues .
• Provide  L2/L3 support for Databricks infrastructure issues.
• Collaborate with  Databricks support for bug fixes and escalations.

Job Tags

Similar Jobs