VAF 0.1.0a35: local models can now also run on vLLM
VAF 0.1.0a35 brings vLLM on Linux for NVIDIA GPUs from Turing on and for supported AMD GPUs; llama.cpp stays the default.
0.1.0a35 is out. Besides llama.cpp, which stays the default, VAF can now run local models on vLLM, meant for servers and high-end PCs that answer many requests at once. vaf engine vllm checks the hardware, installs vLLM after asking, downloads a model chosen by GPU memory, checks that it fits and switches only once vLLM answers a test start. Settings, AI & Model offers the same choice.
vaf updateWith an NVIDIA GPU, vLLM needs Linux, a Turing or newer card, driver 580 or newer and a recent distribution, for example Ubuntu 20.04 on a PC or 22.04 on ARM. With a supported AMD GPU such as a Radeon RX 7900 or a Ryzen AI MAX, it needs Linux, ROCm 7.2 or newer, Python 3.12 and glibc 2.39 or newer, for example on Ubuntu 24.04. VAF checks this first and says what is missing.
Two features are new:
- Settings, AI & Model now opens with “Who does what”. The overview shows which provider and model handles each job, from chat to failover, gives the reason when a job cannot run and lets you change both right there. In the terminal, vaf providers shows the same rows.
- When several people use VAF at once, chat, the local speech services, the browser agent on the shared browser and the document preview take next whoever was served least recently. A waiting message shows its place and the expected wait.
Two bugs are fixed:
- Files and logins in the shared browser no longer cross from one account to the next. If a file transfer or the previous person's sign-off was still running at the handover, that person's workspace files could appear in the next person's file picker. In that case a download could also reach the wrong account, and the previous person's saved logins could take in the next person's. Personal browsers were not affected.
- A sub-agent provider chosen in the web settings now takes effect when saved. Before, the choice had no effect and sub-agents kept running on the main provider. A choice stored that way by an older version stays off until you choose it again and save.
VAF now reads saved workflows in ~/.vaf/workflows as data without running the file, so a workflow whose name or text was crafted to run code on load cannot. A file you write there by hand must therefore hold a plain WORKFLOW = {...}. If it computes its content, VAF skips it and names it in the log.