Lead Cloud Data Engineer (Databricks) - Assistant Vice President

State Street — Hyderabad, India

Who we are looking for: A strong Sr. Data Engineer with deep hands-on expertise in Databricks and modern data engineering platforms. Proven ability to design, build, and optimize scalable data pipelines on cloud platforms. Experience working in agile teams and collaborating across business, architecture, and engineering stakeholders. Strong problem-solving mindset with focus on delivering high-quality, reliable data solutions. What you will be responsible for: Design, develop, and maintain scalable data pipelines and workflows using Databricks and Spark. Build and optimize ETL/ELT pipelines for batch and near real-time data processing. Collaborate with architects and business stakeholders to translate requirements into technical implementation. Ensure data quality, governance, and consistency across data platforms. Perform performance tuning and optimize cost efficiency of data workloads. Support production deployments, troubleshoot issues, and ensure reliability of data systems. What we value: Strong engineering and ownership mindset with attention to quality and performance. Collaboration and ability to work effectively in global distributed teams. Continuous learning and adoption of modern data engineering practices. Ability to communicate technical concepts clearly to both technical and non-technical stakeholders. Mandatory skills / experience: 14-18 Hands-on experience with Databricks and Apache Spark (Python/Scala). Strong experience in Data Engineering and Big Data technologies. Experience with cloud platforms such as AWS or Azure. Strong SQL and Python/Scala programming skills. Experience with data pipeline design, ETL/ELT frameworks, and data modeling. Experience in performance optimization and scalability of data solutions. Additional requirements / Good to have skills: Experience with streaming technologies such as Kafka. Knowledge of Delta Lake and data lakehouse architecture. Experience with CI/CD and DevOps practices for data engineering. Familiarity w

View on HireSetu