Inference Engineer
San Francisco
Job description
Collision AI is building frontier cybersecurity models to help defend our customers. As an Inference Engineer, you’ll build and own the inference servers that run our models, both on our infrastructure and in customer on-prem environments.
You’ll work across model runtimes and the serving layer: request scheduling, batching, caching, GPU memory management, and multi-GPU execution. You’ll profile real workloads, identify bottlenecks, and improve latency, throughput, and hardware utilization while preserving model quality. You’ll work closely with the research team to bring new models into production and support the multi-step workloads our security agents depend on.
Customer deployments are a core part of the role. You’ll make the serving stack straightforward to install, observe, update, and recover on customer-owned hardware, including environments with limited network access. You’ll build reproducible deployment tooling, sensible defaults, and reliable upgrade and rollback paths, so the same system can run well in our own fleet and inside a customer’s network.
Required qualifications
- Experience building model-serving infrastructure or other performance-sensitive distributed systems.
- Strong software engineering skills in Python, with experience in C++, Rust, or a comparable systems language.
- An understanding of LLM inference, including batching, KV-cache management, and GPU compute and memory bottlenecks.
- Experience profiling systems, designing reproducible benchmarks, and shipping measurable improvements to performance or reliability.
Bonus qualifications
- Work on inference runtimes such as vLLM, SGLang, or TensorRT-LLM, including extensions or open-source contributions.
- Experience with CUDA or Triton, quantization, speculative decoding, or multi-GPU inference.
How to apply
To apply, email hello@collision.ai