michael/AI-Profile-Router
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Replace compression sidecar with Hermes micro-compaction

Browse Source
This commit is contained in:
Mikei386
2026-08-26 06:18:51 +02:00
parent 438bd34b72
commit 896c12193e
8 changed files with 23 additions and 140 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
-5
View File
@@ -12,11 +12,6 @@ XTTS_CACHE_DIR=/data/models/xtts-v2-cache
XTTS_GPU_DEVICE=GPU-4834d9d7-5b61-3004-1fb3-4ae49d482d4b
AI_DNS=192.168.1.1
# Experimental context-compression benchmark service on the RTX 3060.
COMPRESSION_MODEL_FILE=qwen3.5-4b-compression/Qwen_Qwen3.5-4B-Q4_K_M.gguf
COMPRESSION_CONTEXT=65536
COMPRESSION_GPU_DEVICE=GPU-4834d9d7-5b61-3004-1fb3-4ae49d482d4b
FAST_MODEL_FILE=qwen-mix/Qwen3.8-27B-IQ4-MIX.gguf
MEDIUM_MODEL_FILE=qwen-pure/qwen3.8-27b-IQ4_XS-pure.gguf
LARGE_MODEL_FILE=qwen-pure/qwen3.8-27b-IQ4_XS-pure.gguf