Senior AI Platform Engineer

Описание

You will own the infrastructure that serves our large language models in production: the serving stack, the GPUs behind it and the API layer in front of it. The systems are live and used every day; the engineering discipline around them - capacity planning, repeatable deployment, monitoring - is yours to build.

Отговорности

• Own the LLM serving stack in production: engine configuration, model rollout and version upgrades
• Tune inference for latency, throughput and cost - batching, caching, parallelism, quantization - and prove each change with measurements
• Plan GPU capacity and produce the numbers behind model and hardware decisions
• Own the API layer in front of the models: authentication, rate limits, routing and failover, usage metering
• Build the delivery layer: CI/CD, reproducible deployments, infrastructure as code, alerting and dashboards
• Operate what you build: diagnose out-of-memory, startup and performance failures, and write the post-mortem
• Keep a model and service registry, so nothing critical depends on one machine or one person

Изисквания

• 5+ years in production engineering, with hands-on ownership of GPU inference workloads
• Hands-on experience with vLLM, SGLang, TGI or TensorRT-LLM: real configurations, real incidents, real fixes
• You can quote latency, throughput and cache-hit numbers from your own work, and explain what moved them
• Working knowledge of transformer internals: attention, KV cache, prefill versus decode, mixture-of-experts routing, quantization
• Strong Python, including reading engine and model source and patching around a bug
• Linux fluency: systemd, Docker, nginx or HAProxy, GPU drivers, monitoring

Preferred Qualifications

• GPU profiling with nsys or ncu; model-flops-utilisation reasoning.
• Multi-node inference and interconnect experience (NVLink, NVSwitch, InfiniBand, RoCE).
• Introducing CI/CD and infrastructure as code into a hand-rolled environment.
• Rust or C++ for performance-critical services.
• Kubernetes experience, and a view on when it is worth it.

Technical Stack

• Serving: vLLM, SGLang, CUDA tooling
• Languages: Python; Rust or C++ for performance-critical work
• Services: Flask, FastAPI, nginx, HAProxy
• Data: PostgreSQL, Redis, ClickHouse, a vector database, S3-compatible storage
• Operations: Docker, systemd, Prometheus, Linux