Specialist Software Engineer - Python + Spark - (26000KL6)

Unlock New Career Horizons with SA Technologies Inc. At SA Technologies Inc., we believe in turning career aspirations into reality. As an NSE Listed company, recognized as a CMMI Level 5 organization, a Great Place to Work Certified company, and one of the Best Places for Women, we are committed to creating a workplace that is inclusive, excellent, and supportive Role Summary Experience: 5–8 Years Design, develop, and maintain scalable data engineering solutions using Databricks, PySpark, SQL, and Delta Lake. Build and optimize batch and real-time data pipelines on AWS using services such as S3, Lambda, Glue, EMR, Redshift, and related cloud-native components. Develop, implement, and support Delta Live Tables (DLT) pipelines for reliable, high-quality, and auditable data processing. Work with Unity Catalog to implement data governance, access control, lineage, and security best practices. Develop and optimize ETL/ELT pipelines with a focus on performance tuning, scalability, reliability, and cost efficiency. Implement infrastructure and platform automation using Terraform and Infrastructure as Code (IaC) principles. Build and maintain CI/CD pipelines using Jenkins, Git, and related DevOps tools to enable automated deployments and testing. Support deployment, monitoring, and maintenance of Databricks workloads across development, testing, and production environments. Collaborate with architects, product owners, data scientists, and business stakeholders to understand requirements and deliver high-quality solutions. Participate in cloud migration, modernization, and data platform transformation initiatives on AWS. Contribute to data quality, observability, logging, monitoring, and alerting frameworks to ensure platform reliability. Work with APIs, data integration frameworks, and external systems to enable seamless data exchange across platforms. Troubleshoot production issues, perform root cause analysis, and drive continuous improvement initiatives. Follow software engineering best practices, participate in code reviews, and ensure adherence to coding, security, and operational standards. Work effectively within Agile teams, contributing to sprint planning, estimation, development, and release activities. Collaborate with global teams and support business-critical deliveries across different time zones when required. Profile Profile Required: Senior‑level Software Engineer with deep Databricks, AWS, and Python data platform expertise, capable of designing and leading enterprise‑scale AI‑enabled data solutions. Minimum Qualifications: Deep hands‑on experience with Databricks (PySpark, SQL, Delta Lake) in large‑scale production environments. Proven experience designing and leading end‑to‑end data pipelines on Databricks. Strong hands‑on experience with Delta Live Tables (DLT) for governed data processing. Advanced experience with Unity Catalog for data governance, access control, lineage, and metadata. Expertise in Spark performance tuning, scalability, and cost optimization. Experience managing Databricks workflows, monitoring, and production operations. Strong knowledge of cloud‑based data platforms (AWS preferred). Proven ability to mentor engineers and lead technical delivery. Preferred Qualifications: Experience building and operating AI agents for workflow automation and decision support. Strong understanding of data observability, platform reliability, and FinOps practices in Databricks. Experience working in regulated or financial services domains. Exposure to enterprise‑scale AI governance, compliance, and audit requirements. Ability to influence architecture decisions and align stakeholders. Excellent communication skills with both technical and non‑technical audiences. Why SA Technologies Inc.? Empowering Work Environment: A Great Place to Work, especially celebrated as one of the Best Places for Women. Excellence and Quality: A CMMI Level 5 Company, ensuring operational and service excellence. Globally Acclaimed: A Microsoft Gold Partner and Google Partner, with over 20 years of global IT consulting and Development solutions deliver. Embark on a rewarding journey together with us! At SA Technologies Inc., your journey begins with a friendly technical assessment, providing a stage to highlight your unique skills.