PYSPARK · APACHE SPARK APACHE AIRFLOW · ETL/ELT REDSHIFT · SNOWFLAKE · BIGQUERY AWS S3 · GLUE · EMR KAFKA · SPARK STRUCTURED STREAMING
Data Engineer · AWS, Spark & ETL/ELT

Saiyashwanth
Vottikoti

pipelines that carry the weight of the business

3+ years building scalable batch and real-time ETL/ELT pipelines with Python, SQL, PySpark, and Apache Airflow — processing 100M+ daily events and 2TB+ of data for analytics, risk, compliance, billing, and Customer 360 use cases across telecom and finance.

Saiyashwanth Vottikoti SV / Data Engineer
scroll
01

Summary

I build pipelines that data teams can trust — the kind that hold up under regulatory scrutiny and peak-hour load alike. My work sits at the intersection of telecom-scale batch processing and finance-grade compliance analytics, turning raw usage logs and transaction records into datasets that risk, compliance, and reporting teams depend on every day, developed and optimized to process 100M+ daily events and 2TB+ of data.

3+
Years in Production Data Engineering
100M+
Daily Events Processed
2TB+
Data Handled Daily
02

Experience

Data Engineer (Contract) APR 2024 — PRESENT
CitiGroup, Hybrid — Sunnyvale, CA
  • Built batch ETL pipelines integrating loans, deposits & card data (10M+ records/day) into a unified AWS lake, cutting reconciliation effort by ~30%.
  • Designed Customer 360 datasets covering 5M+ customer profiles, powering credit risk, exposure & compliance reporting across 3 business units.
  • Implemented 15+ automated reconciliation/validation checks, boosting audit accuracy to 99.8% and reducing regulatory reporting errors by ~25%.
  • Optimized large-scale Spark joins & aggregations on 2TB+/day datasets, cutting job runtime 30-40% through tuning and partitioning.
  • Automated 20+ data workflows via Apache Airflow, ensuring 100% on-time regulatory delivery and saving ~15 hrs/week in manual scheduling.
  • Delivered 10+ analytics-ready datasets ($B+ exposure) with risk/fraud/compliance teams using Snowflake & SQL AST-based lineage parsers.
Technologies: AWS S3, AWS Glue, Apache Spark, Apache Airflow, SQL, ETL/ELT, Snowflake, SQL Abstract Syntax Tree
ETL Developer (Full-Time) JAN 2023 — MAY 2023
Reliance Jio Infocomm Ltd — Remote, Hyderabad, India
  • Built Spark batch ETL jobs processing 50M+ CDR, SMS & internet usage records daily, feeding billing systems for 10M+ subscribers.
  • Designed daily aggregation pipelines computing usage metrics for 10M+ subscribers, cutting aggregation runtime by ~25% via partition tuning.
  • Applied watermarking & checkpointing to handle late-arriving, out-of-order events across 5+ sources, reducing data loss/duplication by ~40%.
  • Optimized Spark transformations & joins on 1TB+ daily telecom datasets, achieving 99%+ SLA compliance for downstream billing deadlines.
  • Managed 15+ Apache Airflow workflows automating & monitoring daily batch jobs, cutting manual intervention by ~20 hours/month.
  • Monitored production pipelines handling peak loads of 100M+ daily events, resolving failures within SLA & maintaining 99.9% uptime.
Technologies: Apache Spark, Apache Airflow, SQL, PySpark, ETL/ELT Data Pipelines
ETL/ELT (Intern) DEC 2021 — NOV 2022
Reliance Jio Infocomm Ltd — Remote, Hyderabad, India
  • Supported 5+ ETL pipelines using SQL, PySpark, and Apache Spark, processing 5M+ records.
  • Performed data validation, transformation, and quality checks across 3+ telecom data sources to ensure data consistency.
  • Monitored 10+ Apache Airflow workflows and supported pipeline troubleshooting and daily data processing.
Technologies: SQL, Python, PySpark, Apache Spark, Apache Airflow, ETL/ELT
03

Projects

AWS S3 · EMR · Airflow · Redshift · Tableau

End-to-End Batch ETL Pipeline on AWS with Airflow

Built a batch ETL pipeline extracting 1M+ records/day from a public API into an AWS S3 data lake using PySpark on EMR. Modeled a star-schema warehouse in Redshift, orchestrated with Airflow (99%+ DAG success), and visualized in Tableau.

Kafka · Spark Structured Streaming · S3 · Redshift

Real-Time Streaming Pipeline — Kafka & Spark Streaming

Built a real-time Kafka pipeline streaming 100K+ clickstream events/hour into Spark Structured Streaming with sub-second latency. Applied watermarking and 5-min windowed aggregations, persisting results to S3 and Redshift for near-real-time analytics.

04

Stack, by Pipeline Stage

01
Languages
SQL Python
02
Ingest & Process
Apache Kafka Spark Structured Streaming Real-Time Data Processing Apache Spark PySpark Batch Processing Data Validation
03
Cloud & Storage
AWS S3 AWS Redshift Azure Data Lake Storage Google Cloud Storage BigQuery Snowflake Star Schema Data Warehousing
04
Orchestrate & Transform
Apache Airflow AWS Glue AWS EMR Azure Data Factory Azure Databricks Dataproc Kubernetes DBT ETL/ELT Pipelines
05
Serve
Tableau Power BI
05

Education & Certifications

Master of Science in Management, Data Analytics
Indiana Wesleyan University, Marion, Indiana
AUG 2023 — APR 2025
Bachelor of Technology in Mechanical Engineering
Sri Indu College of Engineering & Technology, Hyderabad
AUG 2017 — JUL 2021