Nab theme, more professional navigation theme

Ctrl + D Favorites

Local Deployment of Open Source Large Modeling Tools

 Submit Website

llm.pdf: experimental project to run a large-scale language model in a PDF file
General Introduction llm.pdf is an open source project that allows users to run large language models (LLMs) directly in PDF files. Developed by EvanZhouDev and hosted on GitHub, this project demonstrates an innovative approach: llama.cpp via Emscripten...
05-05 840kudos
Aana SDK: An Open Source Tool for Easy Deployment of Multimodal AI Models
综合介绍 Aana SDK 是一个开源框架，由 Mobius Labs 开发，名字源于马拉雅拉姆语“ആന”（大象）。它帮助开发者快速部署和管理多模态 AI 模型，支持处理文本、图像、音频和视频等多种数据。Aana SDK 基于 Ray 分布...
03-25 3550kudos
BrowserAI: Running AI Models Locally in the Browser with WebGPUs
综合介绍 BrowserAI 是一个开源工具，让用户直接在浏览器中运行本地 AI 模型。它由 Cloud-Code-AI 团队开发，支持像 Llama、DeepSeek、Kokoro 这样的语言模型。用户无需服务器或复杂设置，就能通过浏览器...
03-16 3650kudos
LitServe: Rapidly Deploying Enterprise-Grade General AI Model Reasoning Services
Comprehensive Introduction LitServe is an open source AI model service engine from Lightning AI, built on FastAPI, focused on rapidly deploying inference services for general-purpose AI models. It supports a wide field of classical machine learning models from large language models (LLMs), visual models, audio models, to...
03-10 2960kudos
Nexa: a small multimodal AI solution for local operation
Comprehensive Introduction Nexa AI is a platform focused on multimodal AI solutions that run locally. It offers a wide range of AI models, including Natural Language Processing (NLP), Computer Vision, Speech Recognition and Generation (ASR and TTS), all of which can be run locally on devices without relying on cloud...
02-01 6810kudos
vLLM: LLM reasoning and service engine for efficient memory utilization
Comprehensive Introduction vLLM is a high-throughput and memory-efficient reasoning and service engine designed for Large Language Modeling (LLM). Originally developed by the Sky Computing Lab at UC Berkeley, it has become a community project driven by academia and industry. vLLM aims to provide...
01-17 5170kudos
Llama 3.2 Reasoning WebGPU: Running Llama-3.2 in a Browser
综合介绍 Transformers.js 是 Hugging Face 提供的一个 JavaScript 库，旨在将最先进的机器学习模型直接在浏览器中运行，无需服务器支持。该库与 Hugging Face 的 Python 版 transf...
01-15 7030kudos
Harbor: a containerized toolset for easily managing and running AI services with one-click deployment of local LLM development environments
Comprehensive Introduction Harbor is a revolutionary containerized LLM toolset focused on simplifying the deployment and management of local AI development environments. It enables developers to launch and manage all AI services including LLM backends, API interfaces, front-end interfaces, etc. with a single click through a clean command line interface (CLI) and companion application...
01-02 9470kudos
Xinference: Easy Distributed AI Model Deployment and Serving
综合介绍 Xorbits Inference（简称Xinference）是一个强大且多功能的库，专注于为语言模型、语音识别模型和多模态模型提供分布式部署和服务。通过Xorbits Inference，用户可以轻松部署和服务自己的模型或内置的...
01-02 4680kudos
AI Dev Gallery: Windows Native AI Model Development Toolset, End-Side Model Integration into Windows Applications
Comprehensive Introduction AI Dev Gallery is an AI development tools application from Microsoft (currently in public preview) designed for Windows developers. It provides a comprehensive platform to help developers easily integrate AI features into their Windows applications. The most notable feature of the tool...
12-30 9100kudos
LightLLM: An Efficient Lightweight Framework for Reasoning and Serving Large Language Models
综合介绍 LightLLM 是一个基于 Python 的大语言模型（LLM）推理和服务框架，以其轻量级设计、易于扩展和高效性能而著称。该框架利用了多种知名的开源实现，包括 FasterTransformer、TGI、vLLM 和 Flash...
12-17 5180kudos
Transformers.js: running nearly 700 AI macromodels in the local web
综合介绍 Transformers.js 是由 Hugging Face 开发的一个 JavaScript 库，旨在让用户能够直接在浏览器中运行最先进的机器学习模型，而无需服务器支持。该库与 Hugging Face 的 Python 版 ...
12-02 1.0 K0kudos
GLM Edge: Smart Spectrum Releases End-Side Large Language Model and Multi-Modal Understanding Model for Mobile, Car and PC Platforms
综合介绍 GLM-Edge 是清华大学（智谱轻言）推出的一系列专为端侧设备设计的大语言模型和多模态理解模型。这些模型包括 GLM-Edge-1.5B-Chat、GLM-Edge-4B-Chat、GLM-Edge-V-2B 和 GLM-Edg...
12-01 9630kudos
EXO: Running distributed AI clusters using idle home devices with support for multiple inference engines and automated device discovery.
综合介绍 Exo是一个开源项目，旨在利用日常设备（如iPhone、iPad、Android、Mac、Linux等）运行自己的AI集群。通过动态模型分区和自动设备发现，Exo能够将多个设备统一成一个强大的GPU，支持多种模型如LLaMA、Mi...
11-28 1.7 K0kudos
LocalAI: open source local AI deployment solutions, support for multiple model architectures, WebUI unified management of models and APIs
综合介绍 LocalAI 是一个开源的本地AI替代方案，旨在提供与OpenAI、Claude等兼容的API接口。它支持在消费级硬件上运行，不需要GPU，能够进行文本、音频、视频、图像生成和语音克隆等多种任务。LocalAI 由 Ettore...
11-28 1.3 K0kudos
llamafile: Distribute and run LLMs using a single file, simplify LLM deployment, cross-platform support for LLMs
综合介绍 llamafile 是一个由 Mozilla Builders 项目推出的工具，旨在简化大语言模型（LLM）的部署和运行。通过将 llama.cpp 与 Cosmopolitan Libc 结合，llamafile 将复杂的 LL...
11-21 9550kudos
Petals: distributed shared GPU running and fine-tuning of large language models, sharing GPU resources like a BitTorrent network
综合介绍 Petals 是一个由 BigScience Workshop 开发的开源项目，旨在通过分布式计算的方式运行大型语言模型（LLMs）。用户可以在家中使用消费级 GPU 或 Google Colab 运行和微调 LLMs，如 Lla...
11-20 8960kudos
Aphrodite Engine: an efficient LLM inference engine that supports multiple quantization formats and distributed inference.
综合介绍 Aphrodite Engine是PygmalionAI的官方后端引擎，旨在为PygmalionAI网站提供推理端点，并支持Hugging Face兼容模型的快速部署。该引擎利用vLLM的Paged Attention技术，实现了...
11-20 8950kudos
llama.cpp: efficient inference tool, supports multiple hardware, easy to implement LLM inference
综合介绍 llama.cpp是一个用纯C/C++实现的库，旨在简化大语言模型（LLM）的推理过程。它支持多种硬件平台，包括Apple Silicon、NVIDIA GPU和AMD GPU，并提供多种量化选项以提高推理速度和减少内存使用。该项...
11-20 1.2 K0kudos

English