Line of Service
Advisory
Industry/Sector
Not Applicable
Specialism
Data, Analytics AI
Management Level
Manager
Job Description Summary
At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data transformation solutions.
Responsibilities
- Design, develop, and maintain scalable data pipelines using Python, PySpark, and Spark (Scala/ PySpark) across Azure and GCP platforms.
- Build and manage data workflows using Databricks Workflows and Apache Airflow / Cloud Composer DAGs.
- Develop and optimize Delta Lake tables on Databricks, ensuring data reliability, performance, and governance.
- Implement and manage Databricks Unity Catalog for data access control and metadata management.
- Work with BigQuery for large-scale data warehousing and analytics.
- Develop event-driven and batch data processing solutions using Pub/Sub, Cloud Dataflow, and Cloud Functions.
- Implement ML pipelines on Databricks, including experimentation tracking using MLflow.
- Collaborate with data scientists, analytics teams, and business stakeholders to deliver end-to-end data solutions.
- Ensure best practices for data quality, security, cost optimization, and performance tuning.
- Support CI/CD pipelines and production deployments for data engineering workloads.
Mandatory skill sets:
- Azure Databricks
- Python, PySpark
- Advanced Spark (PySpark/ Scala)
- Databricks Delta Tables
- Databricks Workflows
- BigQuery
- Databricks Unity Catalog
Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.