Job Title

Technical Lead – Microsoft Fabric Data Engineering

Location

Noida, India

Overview of the Role

As a Technical Lead – Microsoft Fabric Data Engineering, you will spearhead the design and implementation of advanced data solutions that harness the capabilities of Microsoft Fabric. Your expertise will drive robust, scalable, and high-quality data pipelines, enabling actionable business intelligence and supporting HCLTech’s strategic transformation initiatives. You will play a critical role in ensuring data integrity, optimizing performance, and fostering a data-driven culture across the organization.

Detailed Responsibilities

• Architect, develop, and implement scalable ETL (Extract, Transform, Load) processes using Microsoft Fabric, integrating data from diverse sources into cloud-based data warehouses and lakehouses.
• Collaborate closely with data analysts, engineers, and business stakeholders to define requirements and translate them into effective and sustainable data solutions.
• Build, maintain, and optimize Power BI dashboards and reports, transforming complex datasets into actionable insights for informed business decisions.
• Ensure data quality, integrity, and accuracy throughout the ETL lifecycle, adhering to industry best practices for data management and governance.
• Monitor, troubleshoot, and optimize ETL processes, proactively identifying and resolving issues or performance bottlenecks to maintain high system reliability.
• Document ETL workflows, data mappings, and process modifications to facilitate transparency and clear communication across technical and non-technical teams.
• Implement advanced data transformation and ingestion patterns, including Change Data Capture (CDC), schema evolution, error handling, and robust data operations.
• Uphold best practices in data security, privacy, and compliance throughout all data engineering activities.

Skill Requirements

Must-Have Skills & Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or related field.
• 7+ years of hands-on experience in ETL development, with recent projects utilizing Microsoft Fabric.
• Expertise in Microsoft Fabric Data Pipelines (activities, triggers, parameters) and Notebooks (PySpark/Spark SQL).
• Deep working knowledge of Delta Lake, including schema evolution, MERGE/UPSERT, OPTIMIZE/VACUUM, partitioning, and checkpointing.
• Strong proficiency in CDC ingestion patterns (watermarks, change tables, log-based ingestion).
• Advanced SQL skills (window functions, CTEs) and experience with PySpark for data transformations and performance tuning.
• Demonstrated experience in implementing data quality frameworks: rule patterns (null, range, referential, uniqueness), scorecards, and exception handling.
• Advanced error handling and observability: idempotency, retries, dead-letter handling, and alerting hooks.
• Solid understanding of business intelligence concepts with practical experience in Power BI for data visualization and reporting.
• Knowledge of data warehousing concepts, architectures, and modeling techniques.
• Familiarity with major cloud platforms (Azure, AWS, Google Cloud) and their respective data storage and processing services.
• Excellent analytical, troubleshooting, and problem-solving skills.
• Strong communication skills, with the ability to convey complex technical concepts to non-technical audiences.

Good-to-Have Skills

• Experience with additional ETL tools (e.g., SSIS) and data warehousing solutions.
• Understanding of data governance, data privacy, and compliance standards.
• Familiarity with programming languages such as Python or R for advanced data manipulation.
• Experience working with stored procedures within Fabric transformations.
• Knowledge of metadata-driven pipelines and parameterization (YAML/JSON controlled).
• Awareness of Power BI model readiness (star schema, aggregations).

Other Requirements (Optional)

• Professional certifications in cloud platforms (e.g., Azure Data Engineer Associate, AWS Certified Data Analytics) are advantageous.
• Experience in large-scale enterprise environments is a plus.
• Demonstrated ability to lead and mentor junior engineers or cross-functional teams.