Lead – Data Devops

About us:

We are proudly owned by Auto & General Insurance Company. The A&G group is made up of trusted and well-known brands such as iSelect, Compare the Market and Budget Direct. In late 2024, iSelect and Compare the Market formed a new Aggregation business within the A&G group and now support millions of Australians to compare and buy their personal finance and household products such as insurance, energy, and loans.

iSelect

At iSelect, we’re passionate about making Australians’ lives easier by saving them time, money, and effort. With over 20 years’ experience, we are a one-stop destination for comparison and purchasing across insurance, utilities, and personal finance products. We compare a wide range of brands and our trained comparison experts help customers to compare, select and save more, putting them in control.

Compare the Market

At CTM, our Noble Purpose is to make a difference in the lives of Australians by simplifying the process of making sound financial decisions. This Noble Purpose is our cultural ‘north star’ guiding our direction, prioritisation and decision making.

How you fit:

The Lead - Data DevOps is responsible for designing, implementing, and maintaining highly available, scalable, and secure data pipelines and platforms. This role ensures efficient CI/CD processes, infrastructure automation, and monitoring to support the organization's data ecosystem.

The role requires a strong background in DataOps, DevOps, cloud data platforms, and infrastructure automation. They will work closely with Data Engineering, Security, IT Operations, and Analytics teams to streamline data workflows, enhance system reliability, and optimize cloud-based data environments.

The primary focus areas include infrastructure as code (IaC), CI/CD for data pipelines, monitoring & observability, cost optimization, and cloud security.

What you do:

 Data Platform & Pipeline Automation

• Design, implement, and maintain CI/CD pipelines for data engineering workflows.
• Automate data pipeline deployments, monitoring, and scaling to ensure reliability and performance.
• Develop and enforce infrastructure-as-code (IaC) practices using tools like Terraform, AWS CloudFormation, or Azure Bicep.
• Work with Data Engineers and IT teams to optimize ETL/ELT processes and cloud data platforms.

Cloud Infrastructure & Security

• Oversee data infrastructure in cloud environments (AWS, Azure, GCP), ensuring high availability and security.  
• Implement role-based access control (RBAC), encryption, and network security best practices within data environments.
• Collaborate with Security and Compliance teams to ensure regulatory compliance
• Optimize cloud storage, compute, and networking costs while maintaining performance

Monitoring, Observability & Incident Response

• Implement monitoring solutions for real-time alerting, anomaly detection, and performance optimization.
• Utilize logging and observability tools such as Prometheus, Grafana, Datadog, Splunk, or CloudWatch.
• Establish incident response processes, including automated rollback strategies and disaster recovery plans.
• Work with Site Reliability Engineering (SRE) teams to ensure high system availability and minimal downtime.

Data Ops & DevOps Best Practices

• Advocate for DataOps methodologies to improve collaboration between Data Engineering, IT, and Analytics teams.
• Implement version control, automated testing, and release management for data pipelines.
• Continuously improve CI/CD workflows to enhance deployment frequency and reduce failures.
• Enforce best practices for data governance, lineage tracking, and auditability.

Collaboration & Stakeholder Engagement

• Work closely with Data Engineers, Analytics, Security, and IT Operations to streamline data infrastructure.
• Act as the subject matter expert (SME) on Data DevOps, providing training and mentorship to team members.
• Engage with business leaders and data users to align infrastructure improvements with business goals.

What you need:

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Cloud Computing, or a related field/relevant experience.
• Experience & Skills:
• 3-5+ years of experience in DevOps, Data Engineering, or Cloud Infrastructure roles.
• Hands-on experience with CI/CD tools (e.g., GitHub Actions, Jenkins, Azure DevOps).
• Proficiency in containerization and orchestration (Docker, Kubernetes, ECS, EKS, AKS) as plus.
• Experience implementing infrastructure as code (IaC) using Terraform
• Expertise in cloud data platforms (AWS, Azure, GCP) with services like AWS Lambda, S3, RDS, Redshift, Glue, EMR, Databricks, BigQuery, etc.
• Strong understanding of cloud networking, security policies, and identity management (IAM, VPC, KMS, Secrets Manager, etc.).
• Experience setting up monitoring and alerting using tools like Prometheus, Grafana, CloudWatch, Datadog etc.
• Ability to troubleshoot performance issues and optimize system reliability.
• Strong experience in Python, Bash, or PowerShell for automation and scripting.
• Ability to write automation scripts for infrastructure provisioning, deployments, and data pipeline management.
• Ability to lead a small team and mentor junior engineers and promote DevOps best practices across teams.
• Strong communication skills to work with both technical and non-technical stakeholders

 What's in it for you:

•  Career Opportunities
• A competitive salary plus generous uncapped commission scheme
• The opportunity to work with a well-known, household brand and public listed company
• Work with a team of highly energised, motivated and driven professionals
• 3 'recharge' days per year (yes, in addition to your annual leave!)
• A free coffee each day, a funky café and a state of the art work space
• An impressive Employee Benefits Programs

If you are someone who fosters team spirit, has resilience and aspires to continue learning and developing their skills to drive commercial outcomes, you will have tremendous success in helping us achieve our progressive growth agendas. High energy levels and a love of having fun are also important to working in this energetic business.