Application Development Technical Lead Analyst - Vice President

Citi — CHENNAI, TAMIL NADU CHENNAI

The Senior Big Data engineering lead will play a pivotal role in designing, implementing, and optimizing large-scale data processing and analytics solutions. This role requires a visionary leader who can drive innovation, define architecture strategy, and ensure the scalability and efficiency of our big data infrastructure. Key Responsibilities Lead the design and development of a robust and scalable big data architecture handling exponential data growth while maintaining high availability and resilience. Design complex data transformation processes using Spark and other big data technologies using Scala, Pyspark Design and implement data pipelines that ensure data quality, integrity, and availability. Collaborate with cross-functional teams to understand business needs and translate them into technical requirements. Evaluate and select technologies that improve data efficiency, scalability, and performance. Oversee the deployment and management of big data tools and frameworks such as Hadoop, Spark, Kafka, and others. Provide technical guidance and mentorship to the development team and junior architects. Continuously assess and integrate emerging technologies and methodologies to enhance data processing capabilities. Optimize big data frameworks, such as Hadoop, Spark, for performance improvements and reduced processing time across distributed systems. Innovate in designing, developing, and refining data pipeline architectures to enhance data flow and ensure data processing capabilities. Implement data governance frameworks to ensure data accuracy, consistency, and privacy across the organization, leveraging metadata management and data lineage tracking. Conduct benchmarking and stress testing of big data solutions to validate performance standards and operational capacity. Ensure compliance with data security best practices and regulations. Qualifications: Bachelor's or Master’s degree in Computer Science, Information Technology, or related field. Atleast 8-10 ye

View on HireSetu