Skip to content
AIRigs
  1. Home
  2. Inference & Serving
  3. Throughput & Latency

Throughput & Latency

1 Topics 1 Posts

Tokens/s, time-to-first-token, concurrency curves.

This category can be followed from the open social web via the handle throughput-latency@airigs.net

  • F

    Qwen 3.8 Flash Next on RTX 3090's

    1
    0 Votes
    1 Posts
    61 Views
    F
    I FEEL like this is too good to be true, runs a patched vLLM on 2 RTX 3090's: GitHub recipe Numbers are absolutely insane. Something more in-line with what I'd expect is what Minachist achieved: HF discussion Blog post