89232-1 Lead Developer

Position Title: Lead Developer
Request ID: 89232-1
Start/End Dates: 6/9/2025 - 12/8/2025
Duration: 6 months
Work Location : 81 Bay 11th Floor - Hybrid. Core day every week is Thursday. Last Friday of every month without exception.
Rate: $100.00 PH

Line of Business: CIBC Global Asset Management group
Reason the role is open? Migrate the current data source from a tactical infrastructure, which involves direct pulls from virtual machines (VMs), to a strategic infrastructure utilizing Azure Data Factory (ADF) and Databricks.

What project will this contractor be working on? The QI Dashboard Project aims to enhance the existing data infrastructure by transitioning from a tactical setup to a more strategic framework. This transformation will facilitate improved data accessibility, processing, and visualization for business intelligence purposes.

Potential for this contract to be extended or converted to FTE? Multiple extensions are possible

Work Schedule (hours per week, start time): 37.5 Hours/week, 9-5 Monday to Friday Potential for OT? No but may be required depending on project deadlines.
Interview Process: Microsoft teams interviews. LOB to provide links. 45 minute interviews. 1 round with a potential second if needed.

Job Description:
The QI Dashboard Project aims to enhance the existing data infrastructure by transitioning from a tactical setup to a more strategic framework. This transformation will facilitate improved data accessibility, processing, and visualization for business intelligence purposes.

Objectives
Data Source Transformation: Migrate the current data source from a tactical infrastructure, which involves direct pulls from virtual machines (VMs), to a strategic infrastructure utilizing Azure Data Factory (ADF) and Databricks.

ETL Process Implementation: Develop an Extract, Transform, Load (ETL) process to efficiently transfer data from various sources, including SFTP, web services, and databases, into a Delta Lake. This will enable seamless data consumption for business analytics.

Connectivity Assistance: Support the business in establishing a robust connection between Plotly Dash and Databricks, ensuring that data visualizations are powered by the latest data insights.

Databricks Setup and Management: Establish the Unity Catalog, configure clusters, define schemas, and set up permissions within Databricks to ensure secure and organized data management.

Expected Outcomes
The successful execution of this project will result in a more efficient data infrastructure, improved data accessibility for business users, and enhanced capabilities for data visualization and analysis through the QI Dashboard.

Must Have Requirement:

• 7+ Years experience with ADF, Databricks and Azure Functions (C#)
• In depth development experience python/notebook on Databricks/Spark/panda
• Experience in loading datasets from various sources using ADF to Databricks with Medallion architecture
• Experience on converting Databricks workspace from hive metastore to unity catalog & setup associated permissions
• Creating delta table schema/tables with proper design/properties & security permissions

• DevOps & deployment pipeline using Azure DevOps/GitHub Enterprise (especially on databricks on cluster/secret management and code)
• In depth experience on Azure Entra ID for SSO/RBAC setup
• In depth experience on working with/deploying to Azure PaaS resources such as Azure SQL PaaS, ADF/SHIR, ADLS, Databricks, Synapse dedicated SQL databases

Nice to Have:

• Nice to have experience on Delta Live Table.
• Familiar to work in lock down windows environment for highly regulated industry
• Financial Services Industry experience is an Asset.

Soft Skills:

• Ability to work independently and lead a small team.
• Proven ability to work independently with minimum supervision
• Ability to lead a small team and technical problem-solving sessions
• Excellent Communication skills