Instructions to use ReadyArt/gemma-4-31B-it-scotoma-2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ReadyArt/gemma-4-31B-it-scotoma-2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ReadyArt/gemma-4-31B-it-scotoma-2")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ReadyArt/gemma-4-31B-it-scotoma-2")
model = AutoModelForMultimodalLM.from_pretrained("ReadyArt/gemma-4-31B-it-scotoma-2", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ReadyArt/gemma-4-31B-it-scotoma-2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ReadyArt/gemma-4-31B-it-scotoma-2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ReadyArt/gemma-4-31B-it-scotoma-2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ReadyArt/gemma-4-31B-it-scotoma-2

SGLang

How to use ReadyArt/gemma-4-31B-it-scotoma-2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ReadyArt/gemma-4-31B-it-scotoma-2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ReadyArt/gemma-4-31B-it-scotoma-2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ReadyArt/gemma-4-31B-it-scotoma-2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ReadyArt/gemma-4-31B-it-scotoma-2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ReadyArt/gemma-4-31B-it-scotoma-2 with Docker Model Runner:
```
docker model run hf.co/ReadyArt/gemma-4-31B-it-scotoma-2
```

A few more tics to fight off (not Scotoma-2 specific; pure Gemma 4 stuff).

by WetRat - opened Aug 7

Discussion

WetRat

Aug 7

Just listing some ideas, not pushing anything on the author(s) of this fine-tune:

Understatements of all sorts. Gemma 4 often defaults to describing something as "[almost] whatever" (or "perhaps/maybe" where it could have been omitted entirely) - not willing to fully commit. I'm not sure how it manifests in English but in other languages it's certainly there.
Varied approximations, or blind guesses presented as if the model is pinpointing / triangulating a characteristic, but ultimately hesitates to stick with just one term and proceeds to write "it was [this] OR perhaps [that]" - like passing a baton to the human rather than standing its own ground, basically "lemme write different things and the user will decide what fits him best". It may even state things diametrically opposed to each other, in order to broaden the choice.
Relying too much on admitting something or someone being real/true/official: "I'm real, I'm here with you" or "alright, you're now [officially] whatever!" Digging deeper into this, I've found that in case with Russian language, there's a particular way Gemma 4 writes when it's out of 'known' positive descriptors - it defaults to the term "настоящий / настоящая / настоящее" (TL: "genuine or real"; male / female / neutral form). Either I'm going insane or it's like the model has an internal state where it recognizes the character (in RP) as 'not a real person', and thus, when the character's very involvement with an important matter is being questioned, it rushes to prove itself (picking up THE most valued descriptor: being real).

And then there's the last one - not as bad due to a slight susceptibility to steering through the system prompt:

It loves the idea of assigning 'operational modes' and nicknames to the character / user. This one is difficult to convey but hear me out: if the user calls himself 'a retarded monkey' in chat, the model will likely snatch it and take it as a hook for a recurring joke. In such cases, it will refer to the user with that term periodically (e.g. "oh-ho, what's my 'retarded monkey' up to now?"). As for 'operational modes' - it's similar, minus the nicknames. E.g. if you discuss with the model how you wasted an entire day playing games, it will take you as a hardcore gamer, winking at you and poking you with an elbow (figuratively) sometimes, as if to remind you whether you're still in the mood to continue doing THAT (e.g. gaming). In the end, it's having a really hard time grasping the concept of fleeting/transitory events, unable to shrug things off as just mere happenstances in the grand scheme of things.

WetRat changed discussion title from A few more ticks to fight off (not Scotoma-2 specific; pure Gemma 4 stuff). to A few more tics to fight off (not Scotoma-2 specific; pure Gemma 4 stuff). Aug 7

aoleg

Aug 7

Encountered 1, 3, and 4 a lot with this model. The "recurring joke" is so annoying I had to ban it specifically in the model's system instructions, while the word "real" (#3) and all its variations I banned at token level.

WetRat

Aug 7

Number 3 is the nemesis. No amount of prompt engineering could ever help it, and the issue itself deeply interferes with the model's ability to write believable social interaction.

User: I love you.
Model: Wow... Thanks for being so real with me.

User: I abhor you.
Model: Huh... At least you're... real about this.

User: What was it like to eat 10 sausages at once?
Model: It was so... real!

AesSedai

Ready.Art org Aug 7

I appreciate the feedback, the way I RP I haven't seen 3/4 really appearing much but 1 and 2 I've seen. I think 1, 2, and 3 might not be that difficult to DPO but 4 would be trickier regarding the transitory events concept, models don't really have a good grasp on the natural passage of time in a narrative and they really like to latch onto and reinforce things so that feels like a deeper problem to try and solve.

There's also a bit of a whack-a-mole issue that for every tic that gets removed, it's likely you find another that replaced it haha. I feel like the model sort of likes to write multi-comma-clause'd sentences now, moreso than it did before, and that feels new.

I don't have a v3 planned right now but having those examples handy is useful, so thanks again!

WetRat

Aug 7

•

edited Aug 8

@AesSedai
Accounting for every little thing at the same time would be a risky thing to do. It's already impressive that there are improvements to begin with.

Speaking of potential issues with Scotoma-2... I've noticed that around ~30K tokens (out of 100K context set) it starts losing its ability to reason, outputting the final answer right away. Weird. I tried loading other Gemma 4 models (same settings, same sysprompt, running in either Unsloth Studio or LM Studio or koboldcpp) - and only Scotoma-2 Q8 (bartowski's gguf) appears faulty.

I double-checked everything, it did reason properly before ~30K chat depth, so, yeah, maybe it's worth looking into. If everything's alright, then perhaps my sysprompt breaks it in some way.

update:
~~ReadyArt's Q8 .gguf does not have this issue!~~

update 2:
~~...or so I thought; well, the issue is back for me with ReadyArt's Q8 .gguf too :/~~

update 3:
Well, forget it. Somehow I managed to make go away, so, it was likely the system prompt that caused it. The model's fine.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment