Data Engineer · Azure · Databricks · Available for new work

Gurgaon, India

PriyanshiGupta

Data Engineer — I build the underneath.

Four years turning messy, inconsistent sources into pipelines a team can actually trust. Cloud platforms on Azure and AWS, European energy data, and the unglamorous plumbing that makes a dashboard mean something.

Peel the result

06 outcomes

The build

2022 — present

01 / Current

AFRY

Data Engineer
Jan 2025 — Present

Energy markets
  • Architected and deployed a greenfield cloud data platform on Azure, orchestrating workflows through Data Factory and wiring generative AI into the engineering process itself.
  • Led migration of 15+ legacy sources to cloud-native architecture, building a Python ETL framework for complex European energy datasets — Elexon, Nord Pool.
  • Engineered resilient pipelines with automated metadata management and error logging, cutting latency 50% and improving freshness 90%.
  • Wrote custom ingestion for awkward inputs: web scraping, FTP, and NetCDF files optimised into high-performance Parquet tables.
  • Replaced a manual CSV-and-local-storage handoff with a web-app-triggered pipeline that generates, formats, and files reports automatically — removing the support team from the loop entirely.
  • Moved 20+ analysts onto Databricks, resolved the logic gaps that came with it, and shipped Genie dashboards for pipeline health.

02 / Previous

ZS Associates

Data Engineer
Jun 2022 — Dec 2024

Pharma consulting
  • Delivered the ETL pipelines behind a $10M consulting programme, processing multi-million-dollar quarterly incentives on Databricks, Airflow, and Amazon Redshift.
  • Automated QA workflows in Python and SQL, lifting productivity 60% and returning 180+ hours a quarter.
  • Led mass data migration and unification across Redshift and Databricks for the client analytics team, holding integrity throughout.
  • Onboarded a $50K system across projects and trained the team on it, pulling completion time down 25%.
  • Built PostgreSQL ETL pipelines for migration work and backend integrations with external APIs.
  • Ran quantitative analysis on complex customer datasets in SQL, Python, and Excel to surface decisions the business could act on.

Materials

Filled = daily driver
Languages
SQLPython PySparkC++
Cloud
Microsoft AzureAzure Data Factory AWSAmazon S3
Data & ETL
DatabricksApache Spark Apache AirflowRedshiftPostgreSQL
Formats
ParquetNetCDF FTPWeb scraping
Visualisation
TableauPower BIDatabricks Genie
AI & web
LLM integrationClaude API JavaScriptHTML / CSSGoogle Apps Script

Built on my own time

02 shipped

Consumer AI

Bare Label

A skincare advisor that recommends at the ingredient level, not the marketing level.

  • Onboarding profiles skin type, concerns, and sensitivity, then curates a routine and a realistic timeline for it.
  • Google Apps Script backend runs lead capture and follow-up — a working CRM pipeline with no server behind it.
  • Mobile-first editorial UI, built for people who don't want to talk to a chatbot.
Claude APIJavaScriptApps Script

Internal tooling

Work Tracker

A dashboard that reads timesheet and project data and says what's actually going wrong.

  • Bandwidth analysis, client health scoring, and AI-detected risk alerts for anomalous gaps and unowned action items.
  • Goal tracking with pace projections against real delivery data.
  • Ask "why is this client eating so much bandwidth?" in plain English, get an answer from your own numbers.
Claude APIPrompt engineeringVanilla JS

Education

B.Tech, Computer Science — Business Analytics

University of Petroleum and Energy Studies 2018 — 2022