Upload 2 files
Browse filesAdded usage of: unsloth/Qwen3.8-27B-GGUF but it is slow on a 32GB laptop
README.md
CHANGED
|
@@ -101,6 +101,7 @@ license: apache-2.0
|
|
| 101 |
* [MoT AI: unsloth/gemma-4-26B-A4B-it-GGUF](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF/tree/main) β efficient
|
| 102 |
* [Dense AI: unsloth/gemma-4-e2b-it-GGUF](https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF/tree/main) β fast debug cycles
|
| 103 |
* [Dense AI: unsloth/gemma-4-e4b-it-GGUF](https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF/tree/main) β balanced
|
|
|
|
| 104 |
2. Place them in the `ai/` folder inside the repo.
|
| 105 |
3. Then copy the GGUF ai name and paste it in the config.py under `_ALPHA_INTELLIGENCE_TO_LOAD` and choose `_MODEL_TYPE: "qwen" OR "gemma"`**
|
| 106 |
|
|
|
|
| 101 |
* [MoT AI: unsloth/gemma-4-26B-A4B-it-GGUF](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF/tree/main) β efficient
|
| 102 |
* [Dense AI: unsloth/gemma-4-e2b-it-GGUF](https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF/tree/main) β fast debug cycles
|
| 103 |
* [Dense AI: unsloth/gemma-4-e4b-it-GGUF](https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF/tree/main) β balanced
|
| 104 |
+
* [Dense AI (works with Linux RAM Swap Memory): unsloth/Qwen3.8-27B-GGUF](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/tree/main) β slow on laptop
|
| 105 |
2. Place them in the `ai/` folder inside the repo.
|
| 106 |
3. Then copy the GGUF ai name and paste it in the config.py under `_ALPHA_INTELLIGENCE_TO_LOAD` and choose `_MODEL_TYPE: "qwen" OR "gemma"`**
|
| 107 |
|
config.py
CHANGED
|
@@ -147,9 +147,11 @@ METATRON_TO_LOAD: list = [
|
|
| 147 |
# ββ AI models recommended ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 148 |
"""
|
| 149 |
Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf
|
| 150 |
-
gemma-4-E2B-it-UD-Q4_K_XL.gguf
|
| 151 |
-
gemma-4-E4B-it-UD-Q4_K_XL.gguf
|
| 152 |
gemma-4-26B-A4B-it-UD-Q6_K_XL.gguf
|
|
|
|
|
|
|
|
|
|
|
|
|
| 153 |
"""
|
| 154 |
# ββ AI to load for each hemisphere βββββββββββββββββββββββββββββββββββββββββββββββ
|
| 155 |
_ALPHA_INTELLIGENCE_TO_LOAD: dict = {
|
|
|
|
| 147 |
# ββ AI models recommended ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 148 |
"""
|
| 149 |
Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf
|
|
|
|
|
|
|
| 150 |
gemma-4-26B-A4B-it-UD-Q6_K_XL.gguf
|
| 151 |
+
gemma-4-E4B-it-UD-Q4_K_XL.gguf
|
| 152 |
+
gemma-4-E2B-it-UD-Q4_K_XL.gguf
|
| 153 |
+
# Dense 27b: Large on 32GB laptop. Less than 1token/sec
|
| 154 |
+
Qwen3.8-27B-UD-Q6_K_M.gguf
|
| 155 |
"""
|
| 156 |
# ββ AI to load for each hemisphere βββββββββββββββββββββββββββββββββββββββββββββββ
|
| 157 |
_ALPHA_INTELLIGENCE_TO_LOAD: dict = {
|