Data Services Engineering - Assistant Vice President

State Street — Bangalore, India

Role Description We are seeking a Data Engineer with strong Databricks expertise to support the NGP (Next Generation Platform) data ecosystem. The role focuses on building scalable data pipelines, optimizing Databricks jobs, and enabling high-performance data processing frameworks within a cloud-based architecture. The candidate will work closely with Snowflake, Azure, and distributed data platforms to deliver robust, production-grade data solutions supporting financial analytics and reference data systems. Databricks & Data Engineering Design, develop, and maintain Databricks-based ETL pipelines using Apache Spark. Build scalable data processing frameworks for ingestion, transformation, and validation Optimize Databricks jobs, clusters, and workflows for performance, reliability, and cost efficiency. Develop and manage job pipelines, scheduling, and orchestration in Databricks environments. Troubleshoot Databricks job failures, performance bottlenecks, and pipeline issues . Improve execution efficiency through cluster tuning, job parallelization, and batching strategies . Support production stability and L4-level troubleshooting for critical pipelines. The role requires close collaboration with cross-functional teams in an Agile environment to deliver data-driven solutions. Exposure to AI tools such as GitHub Copilot and modern development practices will be considered an added advantage Core/Must have skills Strong experience in: Databricks + Apache Spark SQL / PySpark / Scala Hands-on with: ETL pipeline development and optimization Distributed data processing frameworks Experience with: Snowflake and data warehousing concepts Azure cloud platform Strong analytical and debugging skills in complex data systems Good to have skills Experience in: Delta Lake / Unity Catalog CI/CD pipelines (Harness / Azure DevOps) Data quality frameworks / validation pipelines Exposure to: Financial domain / performance analytics / reference data platforms Knowledge of AI-assisted

View on HireSetu