large language models on Sadman Kabir Soumik

large language models on Sadman Kabir Soumik https://soumik.blog/tags/large-language-models/ Recent content in large language models on Sadman Kabir Soumik Hugo -- gohugo.io en Copyright © 2022, Sadman Kabir Soumik Sun, 09 Aug 2026 00:00:00 +0000 How vLLM Works for Serving Large Language Models https://soumik.blog/artificial-intelligence/how_vllm_works_for_serving_llms/ Sun, 09 Aug 2026 00:00:00 +0000 https://soumik.blog/artificial-intelligence/how_vllm_works_for_serving_llms/ Introduction Serving a large language model, or LLM, is different from calling an ordinary function. A normal function usually receives an input, does some work, and returns one result. An LLM server keeps a large model loaded in memory, accepts many requests, produces one token at a time, and sends partial results back to users while the answer is still being created. This creates a difficult engineering problem. The model weights use a lot of memory on a graphics processing unit, or GPU.