Senior Developer - Python and Spark

Citi — Pune Maharashtra India

Citi is looking for a Senior Data Engineer to design and build the next-generation data processing and analytics platform that sits at the core of our AI-driven business innovation. In this role, you will architect scalable ETL/ELT pipelines using Python and PySpark, while also integrating Generative AI and Machine Learning models to push the boundaries of what our data systems can do. Working within a high-performing engineering team, you will shape how data flows, performs, and delivers value across the organisation at scale. Responsibilities Design and build highly scalable ETL/ELT pipelines in Python and PySpark to power reliable data ingestion, transformation, and integration at scale. Develop production-grade code and elevate team capability through structured code reviews and hands-on peer mentorship. Integrate Generative AI and Machine Learning models into data workflows, using AI development tools to deliver innovative solutions and resolve complex technical challenges. Establish and enforce data quality standards across all pipelines, ensuring accuracy, consistency, and reliability of data throughout its lifecycle. Implement an AI-first engineering approach by exploring and embedding Agentic AI workflows to increase development velocity and platform intelligence. Automate CI/CD pipelines for builds and deployments, advancing DevOps maturity across the data platform. Apply performance tuning and optimisation techniques to process large volumes of structured and semi-structured data efficiently and at speed. Collaborate directly with engineers and product owners to align technical direction with delivery goals and accelerate outcomes. Required Qualifications & Skills Deep hands-on expertise in Python and Apache Spark, including Spark Core, Spark SQL, and DataFrames/Datasets, applied to production data systems. Advanced SQL capability across complex query writing, data transformation logic, and query performance optimisation. Demonstrated experience designing

View on HireSetu