video · community record

SBTB 2023: Nick Hill, Efficiently serving LLMs at scale.

Name: SBTB 2023: Nick Hill, Efficiently serving LLMs at scale.
Description: Challenges of serving LLMs efficiently in highly concurrent, multi-user contexts, including continuous batching of heterogeneous requests and careful manipulation of tensors with PyTorch.

Challenges of serving LLMs efficiently in highly concurrent, multi-user contexts, including continuous batching of heterogeneous requests and careful manipulation of tensors with PyTorch.

Open on YouTube

Watch on YouTube ↗Add or suggest a connection ↗

01

Connections

2 relationships

recorded as · incomingEfficiently Serving LLMs at Scaletalk ↗published · incomingFunctionalTVcompany ↗

Loading devreal.ai…

Connections