inference-snap
Chat with generative AI Language Models that runs locally without internet. tldr-pages
On Linux
Linux
Chat with generative AI Language Models that runs locally without internet. Here inference-snap is one of the supported models: deepseek-r1, gemma3, gemma4, nemotron3-nano, qwen-vl.
Start a chat in terminal (and a chat server in background on first start)
inference-snap chatIdentifyShow current engine and status of chat server
inference-snap statusIdentifySelect specific engine (eg. cuda for NVIDIA GPU). Download and use the appropriate model for the engine
sudo inference-snap use-engine engineIdentifyShow detailed device hardware information (RAM, architecture, etc.)
sudo inference-snap show-machineIdentifyGet all configuration (http.host, http.port) of running chat server
inference-snap getIdentify