Instructions to use csharikrishna/snif-qwen-deepseek with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use csharikrishna/snif-qwen-deepseek with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="csharikrishna/snif-qwen-deepseek", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("csharikrishna/snif-qwen-deepseek", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use csharikrishna/snif-qwen-deepseek with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "csharikrishna/snif-qwen-deepseek"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "csharikrishna/snif-qwen-deepseek",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/csharikrishna/snif-qwen-deepseek

SGLang

How to use csharikrishna/snif-qwen-deepseek with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "csharikrishna/snif-qwen-deepseek" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "csharikrishna/snif-qwen-deepseek",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "csharikrishna/snif-qwen-deepseek" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "csharikrishna/snif-qwen-deepseek",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use csharikrishna/snif-qwen-deepseek with Docker Model Runner:
```
docker model run hf.co/csharikrishna/snif-qwen-deepseek
```

SNIF: StateGate Neural Inference Fabric (Qwen2.5-0.5B + DeepSeek-R1-1.5B)

SNIF (StateGate Neural Inference Fabric) is an open-source composite causal language model that dynamically connects Qwen2.5-0.5B (Edge Prefill and Fast Decode) and DeepSeek-R1-1.5B (Cognitive Reasoning) via a continuous 2048-dimensional latent interconnect.

Author: CS Hari Krishna (csharikrishna1806@gmail.com)
Repository: csharikrishna/stategate
License: MIT

Architecture Highlights

Continuous Latent Deliberation: Zero intermediate text tokens transferred across models during routing.
Asymmetric Compute Efficiency: Confident queries complete via lightweight edge prefill; complex queries escalate dynamically to deep reasoning.
Low Hardware Footprint: Active memory footprint is ~1,980 MiB VRAM, enabling execution on consumer GPUs and free Google Colab environments.
Concise Decode: Bounded conclusion decoding avoids repetitive multi-paragraph derivations, delivering up to 3.5x faster decode latency.

Quickstart

1. Installation

pip install torch transformers safetensors accelerate sentencepiece tiktoken

2. Inference

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "csharikrishna/snif-qwen-deepseek"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if torch.cuda.is_available() else torch.float32

# Load tokenizer and composite model
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    trust_remote_code=True,
    torch_dtype=dtype,
    device_map="auto" if torch.cuda.is_available() else None,
)
if device == "cpu":
    model = model.to("cpu")
model.eval()

# 1. Factual Query (Fast exit locally via Qwen)
prompt1 = "What is the boiling point of water at sea level?"
inputs1 = tokenizer(prompt1, return_tensors="pt").to(model.device)
out1 = model.generate(**inputs1, max_new_tokens=40, do_sample=False)
print("Answer 1:", tokenizer.decode(out1[0][inputs1.input_ids.shape[-1]:], skip_special_tokens=True).strip())
print("Telemetry 1:", model.last_routing_info)

# 2. Logic Query (Escalates via 2048D NeuralBus to DeepSeek-R1 with Dynamic CHOP)
prompt2 = "If all bloops are razzies and some razzies are snarks, are all bloops snarks? State conclusion concisely."
inputs2 = tokenizer(prompt2, return_tensors="pt").to(model.device)
out2 = model.generate(**inputs2, max_new_tokens=40, do_sample=False)
print("Answer 2:", tokenizer.decode(out2[0][inputs2.input_ids.shape[-1]:], skip_special_tokens=True).strip())
print("Telemetry 2:", model.last_routing_info)
# Example Telemetry:
# {
#   'route_taken': 'ESCALATE_CHOP',
#   'p_uncertainty': 0.4888,
#   'reasoning_layers_executed': 13,
#   'reasoning_layers_saved': 15,  # 53.6% compute reduction!
#   'chop_stability': 0.8769,
#   'intermediate_text_tokens': 0,
#   'elapsed_ms': 3058.4
# }

System Architecture

Small Model (Edge): Qwen/Qwen2.5-0.5B-Instruct (896D, 24 layers, 494M params)
Big Model (Reasoning): deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B (1536D, 28 layers, 1.78B params)
Latent Interconnect: Universal 2048D Lingua Franca Hub
Routing Policy: Pre-generation latent routing evaluating intermediate activations at Layer 9.

Citation

@misc{stategate2026,
  author = {CS Hari Krishna},
  title = {StateGate: Pre-Generation Latent Meta-Routing for Cost-Optimal Language Model Inference},
  year = {2026},
  publisher = {GitHub},
  howpublished = {\url{https://github.com/csharikrishna/stategate}}
}

Note: Research paper and extended theoretical evaluation in preparation.

License and Trademark Notice

Software and Model Weights License: MIT License
Trademark Notice: The names "StateGate", "SNIF", and "NeuralBus" are project and research identifiers belonging to CS Hari Krishna. This open-source license grants permission to use the software and weights, but does not grant rights to use these names to endorse or promote derivative products without attribution.

Downloads last month: 175

Safetensors

Model size

16.9M params

Tensor type

F32