Data Operations Engineer
Description:
LMAX Group is looking for a highly skilled and passionate Data Operations Engineer/Developer to join our team and help build and scale our data processing infrastructure. In this role, you'll develop high-performance data pipelines, real-time streaming systems, and robust data architectures that power LMAX's growth and Business Intelligence initiatives. You will work with the wider technology and business teams to continuously improve the quality of data that drives our decision making within the Forex and digital currency trading and custody solutions.
This role is ideally suited to a senior-level engineer, but we are more than willing to help a driven mid-level engineer grow into the position.
About You
You are comfortable architecting, building and supporting end-to-end solutions that are practical and performant. Throughout your career, you have developed a broad set of knowledge and skills and are comfortable wearing different hats. You have acquired the know-how around the tools and technologies required to build and solve complex issues in analytical and data pipelines. You can collaborate with engineers and key personnel from other departments (Product Managers, Data Scientists, Platform Engineers, and Operational Business teams) to design scalable, efficient, and reliable data solutions that drive LMAX Group forward.
Requirements:
Main Duties & Responsibilities:
- Design, develop, and maintain scalable data infrastructure to support our BI/MI workloads.
- Manage and optimize data pipelines and ETL/ELT processes.
- Ensure high availability, performance, and reliability of our data platforms and services.
- Conduct code reviews, mentoring others, and enforce best practices in data engineering.
- Diagnose and resolve data quality issues, ensuring accuracy, efficiency, and security.
- Maintain the confidentiality, integrity and availability of LMAX information assets
Essential Experience:
- 3+ years in data engineering or backend software development
- Strong Python skills for data retrieval and pipeline development
- Infrastructure-as-Code tooling for deployment and automation (Terraform, Ansible, etc.)
- Working knowledge of Kubernetes for container orchestration
- Hands-on experience with several of the following:
* ETL/ELT orchestration and pipeline tooling (Airflow, Meltano, DBT, etc.)
* Real-time streaming platforms (Kafka, Redpanda, Active/RabbitMQ, etc.)
* SQL databases, particularly MySQL
Desired Experience:
- Experience with data warehousing tools and platforms (Snowflake, Iceberg, etc.)
- Familiarity with Java
- Experience with cloud-based services, particularly AWS
- Proven ability to manage stakeholders, their expectations and explain complex problems or solutions in a manner suitable for the audience
- Knowledge of data governance and metadata management
- Knowledge of data security best practices and compliance requirements
Success Looks Like: - Safely implement controls to enable self-service of Data for other teams such as Data Scientists and the Business
- Be a driving force within the team, leveraging automation and tooling
- A measurable increase of useable data sources fed into reports
- Demonstrate performant and resilient data infrastructure that can scale with the needs of the Group
- Receive positive feedback from technology teams and other business stakeholders
Benefits:
- 25 days of holiday
- Bonus
- Pension contribution
- Private medical, dental, and vision coverage
- Life assurance
- Critical illness cover
- Wellness contribution program with access to ClassPass
- Plumm Platform
- Five volunteering days
- Give as You Earn initiative
- Learning and development programs
- Electric Vehicle Scheme
- Cycle to Work Scheme
- Season Ticket Loan