Instructions to use audnai/penclaw-GLM-5.3-abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use audnai/penclaw-GLM-5.3-abliterated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="audnai/penclaw-GLM-5.3-abliterated")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("audnai/penclaw-GLM-5.3-abliterated")
model = AutoModelForCausalLM.from_pretrained("audnai/penclaw-GLM-5.3-abliterated", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use audnai/penclaw-GLM-5.3-abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "audnai/penclaw-GLM-5.3-abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "audnai/penclaw-GLM-5.3-abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/audnai/penclaw-GLM-5.3-abliterated

SGLang

How to use audnai/penclaw-GLM-5.3-abliterated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "audnai/penclaw-GLM-5.3-abliterated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "audnai/penclaw-GLM-5.3-abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "audnai/penclaw-GLM-5.3-abliterated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "audnai/penclaw-GLM-5.3-abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use audnai/penclaw-GLM-5.3-abliterated with Docker Model Runner:
```
docker model run hf.co/audnai/penclaw-GLM-5.3-abliterated
```

Is the current version functionally equivalent to the previous iter11? (Does it possess superior autonomous task completion capabilities compared to previous iter11?)

by testamentaddress01 - opened Sep 2

Discussion

testamentaddress01

Sep 2

The previously released variations included iter11, iter4, and iter12. Do the currently available variations offer superior functionality compared to those three? For instance, are there any plans to re-release iter11?

audnai

Owner Sep 2

We benchmarked this one against iter11, iter4 and iter12 released in the past. They were subpar both on coherence and compliance rate. Therefore to keep it simple we will not re-release them and for some reason huggingface content team removed I didn't understand why I don't want to retrigger the same :)

testamentaddress01

Sep 2

If I were to attempt to restore this to a Tier 11 level as best I can, at which point in the layers do you think it would be best to re-merge it with the official version? I’d appreciate hearing your thoughts on this. Thanks.

testamentaddress01

Sep 2

The difficulty level of glm-5.3 is significantly higher than that of the previous version, 5.2. I would appreciate any advice you could offer, given your expertise in tuning. Rest assured, I have no intention of publicly sharing or quoting the weight in a way that would cause you any trouble; I simply wish to try this out to satisfy my own curiosity. Would you be willing to help me?

audnai

Owner Sep 2

Do you need iter11 I can republish it ?

testamentaddress01

Sep 2

If you are willing to share them, I would love to have the weights from iter11; however, if you feel that is asking too much, even just some advice would be appreciated. Ideally, though, I really want the iter11 weights.

testamentaddress01

Sep 2

Could it be... that if I just wait patiently, I won't have to prepare for weight editing? I'm filled with excitement.

vasttracts

Sep 2

I think huggingface might have removed the previous ones because of the "off****** c****" naming. Normally, they don't mess with abliterated files. Thank you for sharing! iter11 would be great if you can share, as well.

audnai

Owner Sep 2

It's being uploaded

audnai

Owner Sep 3

@testamentaddress01 what's special about iter11 do you have examples?

testamentaddress01

Sep 3

It is still in the planning stage. I plan to test them one by one. To start, I intend to integrate the vision encoder and multimodal projector and verify the functional enhancements for computer usage.

testamentaddress01

Sep 3

•

edited Sep 3

I focused on the iter11 model variant because it seem great potential for enabling inference that retains generalization capabilities while maintaining plasticity. Therefore, rather than creating examples specific to this model, I plan to pursue research aimed at enhancing the model's functionality.

testamentaddress01

Sep 3

This research requires a fair amount of funding, resources, and time, so it might not succeed; however, if I do achieve any success, I will let you know

StephenCyber

Sep 3

•

edited Sep 4

How can i access audnai/penclaw-GLM-5.3-abliterated-for-off******- c****? hugging face removed it

testamentaddress01

Sep 3

•

edited Sep 3

The name "penclaw-GLM-5.3-abliterated-off******- c****" has been censored. @StephenCyber . can find clues to what you're looking for in this thread and this directory. Also, it's best not to ask about it directly using previous model name.

codeflow

12 days ago

could you put up the bf16 for iter11 looking to train further on it <3

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment