Instructions to use dreamgen/lucid-v1-nemo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use dreamgen/lucid-v1-nemo with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="dreamgen/lucid-v1-nemo")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("dreamgen/lucid-v1-nemo") model = AutoModelForCausalLM.from_pretrained("dreamgen/lucid-v1-nemo", device_map="auto") - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use dreamgen/lucid-v1-nemo with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "dreamgen/lucid-v1-nemo" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dreamgen/lucid-v1-nemo", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/dreamgen/lucid-v1-nemo
- SGLang
How to use dreamgen/lucid-v1-nemo with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "dreamgen/lucid-v1-nemo" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dreamgen/lucid-v1-nemo", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "dreamgen/lucid-v1-nemo" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dreamgen/lucid-v1-nemo", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Unsloth Desktop
- Docker Model Runner
How to use dreamgen/lucid-v1-nemo with Docker Model Runner:
docker model run hf.co/dreamgen/lucid-v1-nemo
Force talking as user or assistant in SillyTavern
Hello! First of all, congrats on the impressive work on this model! It's a bit hard to handle at first, but damn it seems good! I was wondering, since the RP template uses "writer character" user and char, how do you talk as user or system to integrate some OOC message in there please? It's possibly a ST feature I'm not aware of... thank you!
EDIT! Just realized the SillyTavern part mentions the /sys command! My bad! I'm getting a small problem though, after sending a /sys message, when I click send for the next message, the character does not respond...
Hello! Glad you seem to like it :)
That's right /sys is for instructions (OOC) and the way ST handles it, you need to click the "send" (airplane) icon again after you add it.
I recorded a short demo as it's maybe easier (the action happens at the bottom -- the progress bar may obscure it):
Oh, that's strange! Could you maybe record it? Then we can ask Cohee or someone else on the ST discord. Maybe it's some setting? Actually, ideally, I would like /sys to start generating response right away without having to press the button again.
I can't record I think... but basically it's like it's hitting a stop sequence token instantly after pressing send to generate the next response. (it says EOS token triggered directly in the console with an empty output.)
@Varkoyote perfect :D happy to hear!
@Varkoyote To change automatically between narrative and character is not possible with SillyTavern (at least not with my skill level -- maybe with some extension / plugin?)
BUT! I created a dedicated "only" writing preset: https://huggingface.co/dreamgen/lucid-v1-nemo#story-writing
You might need to adapt your card a bit to make it work well though.
I am attaching what it would look like for the model to automatically switch between characters and narrative in role-play mode:
Fixed! I had "Send" to Continue option enabled in my user settings for some reason, disabling it fixed it π
I have this disabled, but I keep getting the same behavior you were experiencing. Any way I can fix this? Using /sys then typing instruction just adds the message to the chat but the model doesn't answer.
@ElvisM You have 2 options:
- use
/sysfollowed by clicking the airplane icon like in the demo video here; or- use
/sys <instruction> | /trigger. For example:/sys Seraphina starts dancing | /trigger.
Adding |/trigger at the end made it work, but clicking on the paper plane icon didn't. I opened an issue on Silly Tavern's Github page. Hopefully someone there knows how to fix it.