metadata
language:
- en
license: apache-2.0
base_model: EleutherAI/gpt-j-6b
tags:
- gpt-j
- lora
- peft
- reasoning
- thinking
- sft
- qwen3-template
datasets:
- nohurry/Opus-4.6-Reasoning-3000x-filtered
- TeichAI/claude-4.5-opus-high-reasoning-250x
- Jackrong/Qwen3.5-reasoning-700x
pipeline_tag: text-generation
GPT-J 6B — Thinking SFT · LoRA Adapter
Raw LoRA adapter (QLoRA, r=32, α=32) for EleutherAI/gpt-j-6b, trained with the Qwen3 chain-of-thought format.
The fully merged float16 model (ready to use without PEFT) is at
👉 ping98k/gpt-j-6b-thinking-sft
Adapter Details
| Property | Value |
|---|---|
| Base model | EleutherAI/gpt-j-6b |
| LoRA rank | 32 |
| LoRA alpha | 32 |
| Target modules | q_proj, k_proj, v_proj, out_proj, fc_in, fc_out |
| Trainable parameters | ~50 M |
| Vocabulary additions | `< |
Usage with PEFT
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model_id = "EleutherAI/gpt-j-6b"
adapter_id = "ping98k/gpt-j-6b-thinking-sft-lora"
tokenizer = AutoTokenizer.from_pretrained(adapter_id) # resized vocab
model = AutoModelForCausalLM.from_pretrained(
base_model_id,
torch_dtype=torch.float16,
device_map="auto",
)
model.resize_token_embeddings(len(tokenizer))
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()
Tip: For most use cases, prefer the pre-merged model at ping98k/gpt-j-6b-thinking-sft — no PEFT dependency needed.
License
Apache 2.0 (inherits from EleutherAI/gpt-j-6b).