Bigdata Developer with PySpark

Citi — Pune Maharashtra India

The Applications Development Intermediate Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities. Responsibilities: Core Responsibilities Develop and maintain data pipelines: Design, develop, and optimize scalable ETL (Extraction, Transformation, Loading) pipelines using PySpark to process large datasets. Coding and software engineering: Write clean, efficient, well-documented code primarily in Python (PySpark) and Java, often utilizing frameworks like Spring Boot. Collaboration and communication: Work with cross-functional teams, including senior developers, data engineers, analysts, and business partners, to understand data requirements and ensure seamless integration of solutions. Troubleshooting and optimization: Debug and resolve data processing issues and performance bottlenecks in Spark applications and other big data technologies. Full lifecycle involvement: Participate in the entire software development lifecycle (SDLC), from requirements analysis and design to testing, deployment, and operations, often using Agile/Scrum methodologies. Data integrity and quality: Ensure data quality and integrity throughout the data lifecycle. Key Technical Skills & Qualifications Programming Languages: Strong understanding of Core Java and Object-Oriented Programming (OOP) concepts. Proficiency in Python, specifically for PySpark development. Big Data Technologies: Hands-on experience or familiarity with Apache Spark (PySpark), Hadoop, and related ecosystem components like Hive and Sqoop. S hould have knowledge of Linux OS and hands-on experience in writing shell scripts . Knowledge of distributed NoSQL databases (e.g., Elasticsearch, Cassandra, MongoDB) is a plus. Databases & SQL: Basic knowledge of SQL

View on HireSetu