Instructions to use NeverSleep/Noromaid-13b-v0.2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use NeverSleep/Noromaid-13b-v0.2-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf NeverSleep/Noromaid-13b-v0.2-GGUF:Q4_K_S # Run inference directly in the terminal: llama cli -hf NeverSleep/Noromaid-13b-v0.2-GGUF:Q4_K_S
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf NeverSleep/Noromaid-13b-v0.2-GGUF:Q4_K_S # Run inference directly in the terminal: llama cli -hf NeverSleep/Noromaid-13b-v0.2-GGUF:Q4_K_S
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf NeverSleep/Noromaid-13b-v0.2-GGUF:Q4_K_S # Run inference directly in the terminal: ./llama-cli -hf NeverSleep/Noromaid-13b-v0.2-GGUF:Q4_K_S
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf NeverSleep/Noromaid-13b-v0.2-GGUF:Q4_K_S # Run inference directly in the terminal: ./build/bin/llama-cli -hf NeverSleep/Noromaid-13b-v0.2-GGUF:Q4_K_S
Use Docker
docker model run hf.co/NeverSleep/Noromaid-13b-v0.2-GGUF:Q4_K_S
- LM Studio
- Jan
- Ollama
How to use NeverSleep/Noromaid-13b-v0.2-GGUF with Ollama:
ollama run hf.co/NeverSleep/Noromaid-13b-v0.2-GGUF:Q4_K_S
- Unsloth Desktop
- Docker Model Runner
How to use NeverSleep/Noromaid-13b-v0.2-GGUF with Docker Model Runner:
docker model run hf.co/NeverSleep/Noromaid-13b-v0.2-GGUF:Q4_K_S
- Lemonade
How to use NeverSleep/Noromaid-13b-v0.2-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull NeverSleep/Noromaid-13b-v0.2-GGUF:Q4_K_S
Run and chat with the model
lemonade run user.Noromaid-13b-v0.2-GGUF-Q4_K_S
List all available models
lemonade list
- Atomic Chat
| license: cc-by-nc-4.0 | |
|  | |
| --- | |
| # Disclaimer: | |
| ## This is a ***VERY EXPERIMENTAL*** version, don't expect everything to work!!! | |
| ## If you don't like this model, use [Noromaid 0.1.1](https://huggingface.co/NeverSleep/Noromaid-13b-v0.1.1) | |
| You may use our custom **prompting format**(scroll down to download them!), or simple alpaca. **(Choose which fits best for you!)** | |
| Expect that many things will change in the next version!! | |
| --- | |
| Mergemonster and a new dataset were used. | |
| If you want a 7b, or 20b hit us up in the Community tab! | |
| # This model is a collab between [IkariDev](https://huggingface.co/IkariDev) and [Undi](https://huggingface.co/Undi95)! | |
| Test model. Suitable for RP, ERP and general stuff. | |
| [Recommended settings - No settings yet(Please suggest some over in the Community tab!)] | |
| <!-- description start --> | |
| ## Description | |
| <!-- [Recommended settings - contributed by localfultonextractor](https://files.catbox.moe/ue0tja.json) --> | |
| This repo contains GGUF files of Noromaid-13b-v0.2. | |
| [FP16 - by IkariDev and Undi](https://huggingface.co/NeverSleep/Noromaid-13b-v0.2) | |
| <!-- [GGUF - By TheBloke](https://huggingface.co/TheBloke/Athena-v4-GGUF)--> | |
| <!-- [GPTQ - By TheBloke](https://huggingface.co/TheBloke/Athena-v4-GPTQ)--> | |
| <!-- [exl2[8bpw-8h] - by AzureBlack](https://huggingface.co/AzureBlack/Echidna-13b-v0.3-8bpw-8h-exl2)--> | |
| <!-- [AWQ - By TheBloke](https://huggingface.co/TheBloke/Athena-v4-AWQ)--> | |
| <!-- [fp16 - by IkariDev+Undi95](https://huggingface.co/IkariDev/Athena-v4)--> | |
| [GGUF - by IkariDev and Undi](https://huggingface.co/NeverSleep/Noromaid-13b-v0.2-GGUF) | |
| <!-- [OLD(GGUF - by IkariDev+Undi95)](https://huggingface.co/IkariDev/Athena-v4-GGUF)--> | |
| ## Ratings: | |
| Note: We have permission of all users to upload their ratings, we DONT screenshot random reviews without asking if we can put them here! | |
| No ratings yet! | |
| If you want your rating to be here, send us a message over on DC and we'll put up a screenshot of it here. DC name is "ikaridev" and "undi". | |
| <!-- description end --> | |
| <!-- prompt-template start --> | |
| ## Prompt template: Custom format, or Alpaca | |
| ### Custom format: | |
| UPDATED!! SillyTavern config files: [Context](https://files.catbox.moe/ifmhai.json), [Instruct](https://files.catbox.moe/ttw1l9.json). | |
| ### Alpaca: | |
| ``` | |
| Below is an instruction that describes a task. Write a response that appropriately completes the request. | |
| ### Instruction: | |
| {prompt} | |
| ### Response: | |
| ``` | |
| ## Training data used: | |
| - [no_robots dataset](https://huggingface.co/Undi95/Llama2-13B-no_robots-alpaca-lora) let the model have more human behavior, enhances the output. | |
| - [Aesir Private RP dataset] New data from a new and never used before dataset, add fresh data, no LimaRP spam, this is 100% new. Thanks to the [MinvervaAI Team](https://huggingface.co/MinervaAI) and, in particular, [Gryphe](https://huggingface.co/Gryphe) for letting us use it! | |
| - [Another private Aesir dataset] | |
| ## Others | |
| Undi: If you want to support me, you can [here](https://ko-fi.com/undiai). | |
| IkariDev: Visit my [retro/neocities style website](https://ikaridevgit.github.io/) please kek |