|
|
--- |
|
|
tags: |
|
|
- gguf-connector |
|
|
widget: |
|
|
- text: a cat in a hat |
|
|
output: |
|
|
url: https://raw.githubusercontent.com/calcuis/gguf-pack/master/w8g.png |
|
|
- text: a raccoon in a hat |
|
|
output: |
|
|
url: https://raw.githubusercontent.com/calcuis/gguf-pack/master/w8f.png |
|
|
- text: a raccoon in a hat |
|
|
output: |
|
|
url: https://raw.githubusercontent.com/calcuis/gguf-pack/master/w6a.png |
|
|
- text: a dog walking in a cyber city with joy |
|
|
output: |
|
|
url: https://raw.githubusercontent.com/calcuis/gguf-pack/master/w6b.png |
|
|
- text: a dog walking in a cyber city with joy |
|
|
output: |
|
|
url: https://raw.githubusercontent.com/calcuis/gguf-pack/master/w6c.png |
|
|
- text: a dog walking in a cyber city with joy |
|
|
output: |
|
|
url: https://raw.githubusercontent.com/calcuis/gguf-pack/master/w8e.png |
|
|
--- |
|
|
## self-hosted api |
|
|
- run it with `gguf-connector`; activate the backend in console/terminal by |
|
|
``` |
|
|
ggc w8 |
|
|
``` |
|
|
- choose your model* file |
|
|
> |
|
|
>GGUF available. Select which one to use: |
|
|
> |
|
|
>1. sd3.5-2b-lite-iq4_nl.gguf [[1.74GB](https://huggingface.co/calcuis/sd3.5-lite-gguf/blob/main/sd3.5-2b-lite-iq4_nl.gguf)] |
|
|
>2. sd3.5-2b-lite-mxfp4_moe.gguf [[2.86GB](https://huggingface.co/calcuis/sd3.5-lite-gguf/blob/main/sd3.5-2b-lite-mxfp4_moe.gguf)] |
|
|
> |
|
|
>Enter your choice (1 to 2): _ |
|
|
> |
|
|
*accept sd3.5 2b model gguf recently, this will give you the fastest experience for even low tier gpu; frontend https://test.gguf.org or localhost (see **decentralized frontend** section below) |
|
|
|
|
|
 |
|
|
|
|
|
- or opt fastapi **lumina** connector |
|
|
``` |
|
|
ggc w7 |
|
|
``` |
|
|
- choose your model* file |
|
|
> |
|
|
>GGUF available. Select which one to use: |
|
|
> |
|
|
>1. lumina2-q4_0.gguf [[1.47GB](https://huggingface.co/calcuis/lumina-gguf/blob/main/lumina2-q4_0.gguf)] |
|
|
>2. lumina2-q8_0.gguf [[2.77GB](https://huggingface.co/calcuis/lumina-gguf/blob/main/lumina2-q8_0.gguf)] |
|
|
> |
|
|
>Enter your choice (1 to 2): _ |
|
|
> |
|
|
*as lumina is no lite version recently, might need to increase the step to around 25 for better output |
|
|
|
|
|
- or opt fastapi **flux** connector |
|
|
``` |
|
|
ggc w6 |
|
|
``` |
|
|
- choose your model* file |
|
|
> |
|
|
>GGUF available. Select which one to use: |
|
|
> |
|
|
>1. flux-dev-lite-q2_k.gguf [[4.08GB](https://huggingface.co/calcuis/krea-gguf/blob/main/flux-dev-lite-q2_k.gguf)] |
|
|
>2. flux-krea-lite-q2_k.gguf [[4.08GB](https://huggingface.co/calcuis/krea-gguf/blob/main/flux-krea-lite-q2_k.gguf)] |
|
|
> |
|
|
>Enter your choice (1 to 2): _ |
|
|
> |
|
|
*accept any flux model gguf, lite is recommended for saving loading time |
|
|
|
|
|
 |
|
|
|
|
|
- flexible frontend choice (see below) |
|
|
|
|
|
## decentralized frontend |
|
|
- option 1: navigate to https://test.gguf.org |
|
|
|
|
|
 |
|
|
|
|
|
- option 2: localhost; keep the backend running and open a new terminal session then execute |
|
|
``` |
|
|
ggc b |
|
|
``` |
|
|
 |
|
|
|
|
|
<Gallery /> |
|
|
|
|
|
## self-hosted api (edit) |
|
|
- run it with `gguf-connector`; activate the backend in console/terminal by |
|
|
``` |
|
|
ggc e8 |
|
|
``` |
|
|
- choose your model file |
|
|
> |
|
|
>GGUF available. Select which one to use: |
|
|
> |
|
|
>1. flux-kontext-lite-q2_k.gguf [[4.08GB](https://huggingface.co/calcuis/kontext-gguf/blob/main/flux-kontext-lite-q2_k.gguf)] |
|
|
> |
|
|
>Enter your choice (1 to 1): _ |
|
|
> |
|
|
|
|
|
 |
|
|
|
|
|
## decentralized frontend - opt `Edit` from pulldown menu (stage 1: exclusive for 🐷 holder trial recently) |
|
|
- option 1: navigate to https://gguf.org |
|
|
|
|
|
 |
|
|
|
|
|
 |
|
|
|
|
|
- option 2: localhost; keep the backend running and open a new terminal session then execute |
|
|
``` |
|
|
ggc a |
|
|
``` |
|
|
 |
|
|
|
|
|
## self-hosted api (plus) |
|
|
- run it with `gguf-connector`; activate the backend in console/terminal by |
|
|
``` |
|
|
ggc e9 |
|
|
``` |
|
|
- choose your model file |
|
|
> |
|
|
>Safetensors available. Select which one to use: |
|
|
> |
|
|
>1. sketch-s9-20b-fp4.safetensors (for blackwell card [11.9GB](https://huggingface.co/calcuis/sketch/blob/main/sketch-s9-20b-fp4.safetensors)) |
|
|
>2. sketch-s9-20b-int4.safetensors (for non-blackwell card [11.5GB](https://huggingface.co/calcuis/sketch/blob/main/sketch-s9-20b-int4.safetensors)) |
|
|
> |
|
|
>Enter your choice (1 to 2): _ |
|
|
> |
|
|
|
|
|
## decentralized frontend - opt `Plus` from pulldown menu (stage 1: exclusive for 🐷 holder trial recently) |
|
|
- option 1: navigate to https://gguf.org |
|
|
|
|
|
 |
|
|
|
|
|
- option 2: localhost; keep the backend running and open a new terminal session then execute |
|
|
``` |
|
|
ggc a |
|
|
``` |
|
|
|
|
|
 |
