LMSYS Org

LMSYS Org, formally Large Model Systems (LMSYS Corp.), is a 501(c)(3) non-profit focused on incubating open-source projects and research. It co-develops open models, datasets, systems, and evaluation tools. Its projects include Chatbot Arena, SGLang, FastChat, and Vicuna. Chatbot Arena is a benchmark platform for LLMs that uses anonymous, randomized battles in a crowdsourced manner.

LMSYS originated from a multi-university collaboration involving UC Berkeley, Stanford, UCSD, CMU, and MBZUAI in 2023. It was incorporated as a non-profit in September 2024. It is supported by donations from institutions including NVIDIA, Google Cloud, and Kaggle.

SGLang is a serving engine for large language models and multimodal models. FastChat is a platform for training, finetuning, serving, and evaluating LLM-based chatbots.