Staff / Senior ML Engineer, Post-Training & Inference

About Aleroy

Aleroy is building a unified platform for high-performance AI inference, making it easier for companies to deploy and run AI models at scale.
Through a single API, Aleroy provides access to ready-to-use and custom AI models across cloud infrastructure, dedicated capacity, and customers' own hardware. Our platform optimizes performance, cost, and deployment flexibility while giving enterprises control over where and how their AI workloads run.
We work in small teams, collaborate directly, and focus on solving complex engineering challenges to build reliable, scalable AI infrastructure.

About the Role

You make open models fit each customer and keep them getting better. You post-train them on the customer's data, build the evaluations that prove each new version is better, and serve the result fast on the customer's own GPUs. Our engine is built on vLLM.
We're looking for a Staff / Senior ML Engineer, Post-Training & Inference. This is a hands-on technical role. You write code every week, own model customisation end to end, and mentor one or two engineers. It is not a people-management role. You work with our Tech Lead and CTO.

What You Will Own

• Post-training

  • Post-train open models for a customer, from LoRA to full-parameter: SFT, DPO and ORPO, reinforcement learning with custom rewards, and distillation into smaller models

  • Choose the method for each customer from their evaluation, not by default

  • Let customers bring their own losses, rewards and rollouts, and run them on our training stack

  Automatic improvement for each customer

  • Turn a customer's production traces and product signals into evaluation sets and reward functions

  • Run the loop on a schedule: train, validate against the model in production, ship only if it wins, roll back if it regresses

  • Keep training and serving numerically consistent, so the model you evaluated is the model you serve

  Serving what you trained

  • Promote a trained adapter or model to production in one step, including many adapters on shared capacity and on quantised base models

  • Draft models for speculative decoding, trained on each customer's traffic

  • Throughput at the customer's latency target: sharding, prefill, decode and cache layout, batching, and FP8 or FP4 with a quality check that includes Arabic

  • Tune each deployment automatically: agents propose serving changes (sharding, batching, drafters, precision), and a change ships only if it is faster and still correct on the customer's evaluation

  Combining models

  • Lower cost at equal quality with routing and cascades across models, measured on the customer's own evaluation

About You

• 5+ years writing production software; fluent Python and PyTorch

• Hands-on post-training of LLMs for real use: SFT and LoRA, plus at least one of DPO or ORPO, reinforcement learning (GRPO, PPO or similar) or distillation

• You have built an evaluation set or a reward function from real usage data, and shown a fine-tuned model beat its baseline on it

• Training across several GPUs with FSDP, DeepSpeed or similar

• You have shipped a model you trained to production on vLLM, SGLang, TensorRT-LLM or similar, measured it, and fixed what broke after release

• For Staff: you have led a technical project lasting several months across several engineers, and made the trade-offs

• English level: Advanced (C1).

What We Offer

• Real ownership of the backend that powers Aleroy inference platform

• Influence over backend architecture and product direction

• Hands-on work on hard problems in distributed systems, real-time streaming, and LLM serving

• A small, senior team with fast execution and low bureaucracy

• Competitive salary with performance-based incentives

• 24 days annual leave plus public holidays

• Health insurance

• Modern office in Yas Creative Hub