About
Highly accomplished Senior Data Engineer with over 14 years in IT, including 5+ years specializing in Azure data platforms. Expertly designs and implements scalable data pipelines using Azure Databricks, Delta Lake, and Azure Data Factory, consistently improving pipeline performance by up to 40% and reducing processing time. Proven leader in banking domain cloud migration, data governance, and security, leveraging strong expertise in PySpark and Medallion architecture to deliver robust, high-quality data solutions.
Work
Conduent
|App Dev & Support Engineer
Hyderabad, Telangana, India
→
Summary
Set to lead application development and support initiatives for Conduent, focusing on enhancing operational efficiency and system reliability.
Virtusa
|Lead Consultant
Hyderabad, Telangana, India
→
Summary
Led complex data engineering initiatives and implemented scalable solutions for diverse banking clients, driving significant improvements in data processing and governance.
Highlights
Designed and implemented scalable ETL pipelines processing over 10M records daily using Azure Databricks and Azure Data Factory, while optimizing Spark configurations to reduce execution time by 35%.
Led the migration of over 100 TB of Oracle data to Azure, enhancing scalability and achieving a 30% reduction in infrastructure costs for banking clients.
Architected and deployed Delta Lake solutions, enabling ACID transactions and enhancing data reliability, alongside developing PySpark transformation frameworks that boosted processing speed by 40%.
Developed Delta Live Tables (DLT) pipelines for automated data quality checks and comprehensive lineage tracking, and automated critical workflows using Databricks Jobs, reducing manual intervention by 60%.
Implemented Unity Catalog for centralized data governance and fine-grained access control, ensuring PCI DSS compliance through robust data masking and security policies.
Provided critical support for large-scale banking applications, managing high-volume transactional data, improving SQL performance by 25%, and reducing production defects by 20% through rigorous testing and impact analysis.
Education
Nagarjuna University
→
B.Sc
Computers
Languages
English
Skills
Cloud
Microsoft Azure.
Data Engineering
Azure Databricks, Apache Spark, PySpark, Delta Lake, Unity Catalog, Delta Live Tables (DLT), Workflows, Jobs API, Azure Data Factory (ADF).
Databases
Azure SQL, SQL Server, Oracle.
Programming
Python, SQL, PL/SQL.
Concepts
ETL/ELT, Medallion Architecture, Data Modeling, Data Governance, Performance Optimization.