Manager- Data Engineer

Harmonic Inc. — Bangalore, Karnataka, India

Key responsibilities • Design, implement, and optimize end-to-end ETL pipelines in Microsoft Fabric, from ingestion through multi-stage transformations to data loading and delivery. • Build pipelines and notebooks using Python and PySpark; implement data validation, error handling, and quality controls. • Collaborate with business analysts and stakeholders to translate requirements and accounting logic into transformation rules and data solutions. • Work with data architects to design schemas and data models (star/snowflake) and, where needed, OLAP cubes aligned to application requirements. • Ensure efficient, accurate processing from source systems into Fabric’s data layers; optimize performance and scalability (partitioning, indexing, resource tuning). • Leverage modern tooling and practices (e.g., Azure DevOps for boards, repos, CI/CD); uphold consistent development standards across the global team. • Conduct unit, integration, and end-to-end testing; troubleshoot and continuously improve ETL processes. • Maintain comprehensive, up-to-date documentation (processes, sources, data flows, models) accessible to stakeholders; ensure compliance with policies and standards. Required skills and experience • High proficiency with Microsoft Fabric ETL; experience with related tools such as Azure Data Factory. • Strong SQL for extraction, transformation, and querying; hands-on with SQL Server, Azure SQL Database, and Synapse Analytics. • Data engineering fundamentals: data modeling and schema design (star/snowflake), transformation, and optimization for warehousing/analytics; experience with OLAP where applicable. • Proficiency in Python and PySpark for ETL development within Fabric notebooks and pipelines. • Experience loading and optimizing data at scale in Fabric and prior exposure to Azure Synapse and Azure Data Lake. • Familiarity with Azure DevOps workflows (work tracking, version control, pipelines) and modern development practices. • Rigorous testing approach (unit,

View on HireSetu