koboldcpp
An AI text-generation application for GGML and GGUF models. tldr-pages
On several platforms
Several platforms
Load one or more model files
koboldcpp --model path/to/model_fileIdentifySet port to listen on (defaults to 5001)
koboldcpp --port portIdentifySet amount of threads to use
koboldcpp --threads amount_of_threadsIdentifyLaunch web browser when boot is complete
koboldcpp --launchIdentifyStart without the GUI launcher
koboldcpp --skiplauncherIdentify