Job Title: Data Engineer (ADF,ADB & Python)
Experience:
5–8 years in Data Engineering / Data Integration roles

Role Overview:
We are looking for a Senior Data Engineer with strong expertise in Azure Data Factory (ADF) , Databricks and Python to design, build, and optimize scalable data pipelines. The ideal candidate will have hands-on experience in ETL development, API integration, and data modeling, along with a solid understanding of database systems and performance optimization.

Key Responsibilities:
Design and develop scalable ETL/ELT pipelines using Azure Data Factory (ADF), Databricks and Python
Build and maintain robust data ingestion frameworks from multiple sources (APIs, databases, files, etc.)
Develop and consume REST APIs using Python for data integration and processing
Optimize data workflows for performance, reliability, and scalability
Design and manage database schemas, including efficient usage of temporary tables
Write complex and optimized SQL queries for data transformation and analysis
Ensure data quality, validation, and governance across pipelines
Collaborate with data analysts, data scientists, and business teams to understand requirements and deliver solutions
Troubleshoot and resolve data pipeline and integration issues
Implement best practices for code versioning, CI/CD, and deployment

Required Skills & Qualifications:
Strong hands-on experience with Azure Data Factory (ADF) , Databricks
Proficiency in Python for data engineering and API development
Experience in building and managing ETL/ELT pipelines
Solid understanding of REST API development and integration
Strong knowledge of SQL and relational databases (e.g., SQL Server, PostgreSQL, etc.)
Experience with schema design, including temporary tables and performance tuning
Familiarity with data warehousing concepts and data modeling techniques
Good understanding of data validation, error handling, and logging frameworks

Preferred Skills:
Experience with Azure ecosystem (Azure Data Lake, Synapse, Databricks)
Knowledge of CI/CD pipelines (Azure DevOps or similar tools)
Exposure to big data technologies (Spark, Hadoop)
Understanding of data security and governance practices

Soft Skills:
Strong problem-solving and analytical skills
Excellent communication and stakeholder management
Ability to work in an agile and fast-paced environment
Mentorship experience for junior engineers

Nice to Have:
Experience in cloud migration or modernization projects
Exposure to real-time or streaming data pipelines

• azure data factory
 
• Data Engineering & Pipeline Management: Databricks (Delta Lake
 
• Python
 
• #Rest API
 
• SQL
 
• Snowflake

• PySpark
 
• claude