Big Data Engineer – Scala
Capco — Pune, Maharashtra
₹3–7 LPA (HireSetu estimate) · Entry level · Freshers eligible · Full-time
HireSetu listing reference 7a2091 — confirm requirements below, then apply on the employer site.
Quick answer: An experienced big data engineering job in Pune, built around Scala and Apache Spark on the Hadoop stack, delivering pipelines for Capco's banking and insurance clients.
From Capco's job posting
Job Title: Big Data Engineer - SCala
You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.
MAKE AN IMPACT
Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.
#BEYOURSELFATWORK
Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.
CAREER ADVANCEMENT
With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.
Job Description: Big Data Engineer – Scala
We are looking for an experienced Big Data Engineer with strong expertise in Scala and Apache Spark to design, develop, and optimize scalable data-processing solutions. The ideal candidate should have hands-on experience with distributed data platforms, large-scale ETL/data pipelines, and big data technologies.
Key Responsibilities
- Design, develop, and maintain scalable Big Data applications and data pipelines.
- Develop high-performance data-processing solutions using Scala and Apache Spark.
- Build and optimize ETL/ELT pipelines for large volumes of structured and unstructured data.
- Work with technologies such as Hadoop, HDFS, Hive, Kafka, and Spark SQL.
- Perform Spark job performance tuning and optimization.
- Develop reusable and maintainable data-processing frameworks.
- Troubleshoot production issues related to data pipelines and distributed applications.
- Implement data-quality checks, monitoring, and error-handling mechanisms.
- Collaborate with data architects, analysts, business teams, and other engineering teams.
- Follow software engineering best practices, including Git, code reviews, testing, and CI/CD.
Required Skills
- Strong hands-on programming experience in Scala.
- Strong experience with Apache Spark / Spark SQL.
- Good understanding of Big Data and distributed computing concepts.
- Experience with Hadoop ecosystem technologies, including Hadoop/HDFS and Hive.
- Experience building large-scale ETL/data engineering pipelines.
- Strong SQL skills.
- Experience working with Kafka or similar messaging/streaming technologies.
- Understanding of data formats such as Parquet, Avro, and JSON.
- Experience with performance optimization and troubleshooting of Spark applications.
- Good understanding of software development lifecycle and version-control systems.
Good to Have
- Experience with AWS, Azure, or GCP.
- Knowledge of Databricks.
- Experience with Airflow or similar orchestration tools.
- Exposure to Delta Lake or other modern lakehouse technologies.
- Knowledge of Docker, Kubernetes, and CI/CD pipelines.
- Experience working in Agile/Scrum environments.
Core Skills
Scala | Apache Spark | Spark SQL | SQL | Hadoop | HDFS | Hive | Kafka | ETL | Data Engineering | Big Data
The employer’s company overview, benefits, equal-opportunity statement and generic skills tables are left out here; they are on the original posting.
Text from Capco's official job posting, formatted by HireSetu. The employer's own page is the final word on details.
Scala is the filter
Most data teams in India write Spark in Python and keep Scala for a few specialists. This listing wants strong hands-on Scala, which narrows the field considerably. If Scala was one project and Python did the rest, the gap will show.
The other half of the requirement is software engineering: reusable frameworks, Git, code reviews, testing and CI/CD all appear in the responsibilities. They want someone who builds maintainable code, not notebooks. Put your Scala and Spark work at the top of the CV, with the data volumes you handled and what your changes did to run times.
Tuning is a separate job
Job performance tuning and production troubleshooting sit alongside building new pipelines, so roughly half this work is fixing what already runs badly. Revise the levers you would reach for on a slow Spark job: shuffle partitions, join strategy, skew, partitioning, small files, caching. Parquet, Avro and JSON are named, so know when each format earns its place.
Data-quality checks, monitoring and error handling are part of the build here. Have one story about a pipeline that silently dropped or duplicated records, and what you changed.
Hadoop and Hive still count
Hive and HDFS sit in the required list, carrying as much weight as Spark, which is not the case in every modern stack. If you have spent the last few years on Databricks or Snowflake alone, refresh your Hive SQL before applying.
Kafka is required for streaming. Airflow, Databricks, Delta Lake, Docker, Kubernetes and the three clouds are good-to-have. Cloud experience on any one of AWS, Azure or GCP transfers, since Capco runs projects across all three.
Client delivery, with banks on the other side
This is consulting, so you join a project rather than a product. Capco's clients are banks, insurers and payment service providers, and the stack changes with each engagement, which explains the long good-to-have list.
Pune is the base. How much time you spend at a client location depends on the engagement. Ask which project you would be staffed on and where it sits before you accept.
Prepared from Capco's job posting and checked against it before publishing. See our Editorial Policy.
How to apply
- The Apply button opens Capco's application form on job-boards.greenhouse.io.
- Its requisition number is 8348883; search for it on the careers site if the link ever stops working.
- No account is needed.
No fees, ever. HireSetu never charges candidates, and no genuine employer asks for money to apply, interview or join. If anyone asks you to pay, stop and report the job.
Similar openings on HireSetu
- Associate – Big Data Hadoop Administrator / Cloudera Admin at EXL — India — Software / IT · Full-time
- Data Analyst at Solstice Advanced Materials — Pune — Data Science · Full-time
- Apprentice – IT Development Program (Master Data Management) at Eaton — Pune — Data Science · Full-time
- Platform Infrastructure Data Analyst at EY — Kochi — Data Science · Full-time
- Data Engineer at Dentsu — Bengaluru — Data Science · Full-time
View Big Data Engineer – Scala on HireSetu · Data Science jobs · Jobs in Pune · Career Insights · FAQ · Companies