Repository:google-ai-edge/LiteRT-LM
LiteRT-LM: Google's Open-Source Inference Framework for On-Device LLMs
LiteRT-LM is an open-source inference orchestration layer from Google designed to run large language models directly on edge devices. It leverages GPU and NPU acceleration and powers on-device generative AI features in Chrome, Pixel Watch, and Chromebook Plus.
6.4kstarsC++
From Video: Weekly AI Update: New Models, Agent Workflows, and Infrastructure Shifts