Pipeline Development & Data Engineering - Lead end-to-end technical delivery of data engineering solutions aligned with business initiatives and data platform architecture. - Design, develop, test, and support ingestion and transformation (ETL/ELT) pipelines using Azure Databricks. Configure applications for best performance. - Develop data processing jobs using PySpark, SQL, and Python with an emphasis on reliability, maintainability, and performance. - Apply Lakehouse patterns to deliver curated data layers, Implement and manage governed data assets using Unity Catalog (organization, access controls, and best practices). Orchestration & Operational SDLC - Orchestrate end-to-end workflows using Azure Data Factory (ADF), including scheduling, dependency management, monitoring, and operational support. - Establish, improve, and implement SDLC processes including development, testing, and production deployments. - Apply Agile and DevOps practices. Ensure platform requirements (functional and non-functional) are met per design and architecture. Collaboration & Stakeholder Engagement - Collaborate with the Data/Business Analyst, Testing Engineer, and stakeholders to clarify requirements and ensure delivered outputs meet expectations. - Collaborate with cross-functional stakeholders including ZTD Infrastructure, IT, business, and Information Security teams to evolve technology solutions. - Engage and collaborate with technology partners and industry forums to stay abreast of technology trends, applying them as per business needs. Engineering Standards & Quality - Adhere to engineering standards through pull requests, code reviews, documentation, and reusable design patterns. - Troubleshoot development and pre-production issues and drive continuous improvements to stability and performance.

Technical Skills: - Azure Databricks (workspace management, cluster configuration, notebooks, jobs) - PySpark and Spark SQL for large-scale data processing - Python for scripting, automation, and data engineering tasks - Unity Catalog for data governance, access controls, and asset organization - Azure Data Factory (ADF) for pipeline orchestration, scheduling, and monitoring - Lakehouse architecture patterns (Bronze/Silver/Gold data layers) - Delta Lake format and optimization techniques - CI/CD pipelines using GitHub Actions - Git-based version control and collaborative development workflows - Strong SQL skills for data transformation and validation - Knowledge of data quality and testing frameworks - Knowledge of DevOps and Agile methodologies Knowledge, Skills & Abilities: - Strong analytics and problem-solving skills with experience across a broad spectrum of analytics - Strong written and verbal communication skills - Strong team player working on cross-functional deliverables