Subham Nanda
Lead Data Engineer
Lead Data Engineer with 5+ years of experience architecting high-throughput distributed systems on Azure. Expert in designing Medallion Architectures using Delta Lake and Apache Iceberg to manage petabyte-scale data lakes. Specialized in Infrastructure as Code (Terraform) and CI/CD to ensure 99.9% system reliability.
Technical Proficiency
Data Ingestion: Databricks Auto Loader, Apache Kafka, AWS Glue, ADF
Data Warehousing: Delta Lake, Apache Iceberg, Synapse Analytics, Snowflake
Orchestration: Databricks Jobs, Azure Data Factory, Docker
DevOps: Terraform, CI/CD Pipelines, Azure DevOps, Jenkins, Git
Frameworks: Agile (SAFe), Unity Catalog, Dimensional Modeling
Languages: PySpark, SQL, Python, Java
Professional Experience
Lead Data Engineer | HCLTech (Client: Henkel)
June 2025 – Present- Engineered a systematic debugging framework for Databricks, resolving 40+ critical production defects and achieving 100% SLA compliance.
- Led the migration to Infrastructure as Code (IaC) using Terraform, reducing environment provisioning time by 70%.
- Optimized downstream reporting reliability (99.9% accuracy) by deploying complex transformation workflows in Azure Databricks.
- Designed a scalable onboarding program for new engineers, accelerating 'Time-to-Code' by 40%.
Software Engineer | Capgemini
March 2021 – May 2025- Designed and optimized ETL pipelines in ADF and Databricks, reducing total data processing time by 50%.
- Applied PySpark to process and transform large datasets, improving query performance by 3x.
- Monitored data pipelines on ADF, reducing failures by 60% and ensuring data consistency and quality.
- Mentored 3 junior engineers and led cross-functional teams in large-scale data engineering projects.
Educational Profile
| Course | Institution | Year | Result |
|---|---|---|---|
| B.Tech (Electronics & Comm.) | Om Dayal Group of Institutions | 2016 - 2020 | 7.8 CGPA |
Certifications
Certified SAFe 6 Agile Practitioner (2025)
Data Science with Azure Databricks (Microsoft)
Databases and SQL for Data Science (IBM)
Foundations of Project Management (Google)