π Step 9: AI/LLM#315 / 350
Throughput
Throughput
πOne-line summary
The number of requests or tokens that can be processed per unit of time.
π‘Easy explanation
How many requests can be handled per second (or minute). The more users you have, the higher throughput you need.
β¨Example
1μ΄μ λͺ κ° μ²λ¦¬?
# μ²λ¦¬λ λͺ¨λν°λ§
ββββββββββ 100 req/s β
βββββββββββββββ 250 req/s β‘
βββββββββββββββββββ 500 req/s π₯
λμμλ‘ λ λ§μ μ¬μ©μ μμ©