Instructions to use inarikami/gemma-12b-medical-mc-qlora-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use inarikami/gemma-12b-medical-mc-qlora-v2 with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("google/gemma-3-12b-it")
model = PeftModel.from_pretrained(base_model, "inarikami/gemma-12b-medical-mc-qlora-v2")

Transformers

How to use inarikami/gemma-12b-medical-mc-qlora-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="inarikami/gemma-12b-medical-mc-qlora-v2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("inarikami/gemma-12b-medical-mc-qlora-v2", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use inarikami/gemma-12b-medical-mc-qlora-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "inarikami/gemma-12b-medical-mc-qlora-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inarikami/gemma-12b-medical-mc-qlora-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/inarikami/gemma-12b-medical-mc-qlora-v2

SGLang

How to use inarikami/gemma-12b-medical-mc-qlora-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "inarikami/gemma-12b-medical-mc-qlora-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inarikami/gemma-12b-medical-mc-qlora-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "inarikami/gemma-12b-medical-mc-qlora-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inarikami/gemma-12b-medical-mc-qlora-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use inarikami/gemma-12b-medical-mc-qlora-v2 with Docker Model Runner:
```
docker model run hf.co/inarikami/gemma-12b-medical-mc-qlora-v2
```

Gemma 12B Medical MC QLoRA v2

A medical multiple choice focused fine-tuned version of Google's Gemma 3 12B Instruct model using QLoRA (Quantized Low-Rank Adaptation) techniques, specifically trained on medical multiple choice datasets.

Model Details

Model Description

This model is a QLoRA adapter fine-tuned on Google's Gemma 3 12B Instruct model specifically for medical multiple choice applications. The model has been trained for 7000 steps on medical multiple choice datasets to understand and generate medical content in a multiple choice format while maintaining the conversational capabilities of the base model.

Developed by: inarikami
Model type: Causal Language Model (QLoRA Adapter)
Language(s): English
Base Model: google/gemma-3-12b-it
License: Same as base model (Gemma License)
Checkpoint: 7000 steps

Model Sources [optional]

Repository: [More Information Needed]
Paper [optional]: [More Information Needed]
Demo [optional]: [More Information Needed]

Uses

Direct Use

This model is designed for medical question-answering, educational purposes, and healthcare-related conversations. It can be used to:

Answer general medical questions
Provide educational content about health topics
Assist with medical terminology explanations
Support healthcare professionals with information retrieval

Downstream Use

The model can be further fine-tuned for specific medical domains such as:

Clinical documentation
Medical summarization tasks
Specialized medical subspecialties
Integration into healthcare chatbots or educational platforms

Out-of-Scope Use

Important: This model should NOT be used for:

Medical diagnosis or treatment recommendations
Emergency medical situations
Replacing professional medical advice
Making clinical decisions without human oversight
Patient care without qualified medical supervision

Bias, Risks, and Limitations

This model has several important limitations:

Medical Accuracy: While trained on medical data, the model may generate inaccurate or outdated medical information
Training Data Bias: The model may reflect biases present in medical literature and training data
Language Limitations: Primarily trained on English medical content
Hallucination Risk: Like all large language models, it may generate plausible-sounding but incorrect information
Regional Variations: Medical practices and guidelines vary by region; the model may not reflect local standards

Recommendations

Critical Safety Guidelines:

Always verify medical information with qualified healthcare professionals
Use only as a supplementary educational tool, never for medical decision-making
Implement human oversight for any healthcare-related applications
Regularly update and validate outputs against current medical guidelines
Be aware of potential biases in medical recommendations across different populations

How to Get Started with the Model

from peft import PeftModel
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load base model and tokenizer
base_model = "google/gemma-3-12b-it"
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(base_model)

# Load QLoRA adapter
model = PeftModel.from_pretrained(model, "your-username/gemma-12b-medical-qlora-v2")

# Generate medical responses
prompt = "What are the symptoms of diabetes?"
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(**inputs, max_length=200, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Training Details

Training Data

This model was fine-tuned on a medical multiple choice question dataset. The training data consists of:

Medical multiple choice questions and answers
Healthcare-related Q&A pairs in multiple choice format
Medical terminology and concept explanations structured as multiple choice problems

Note: The model's training on multiple choice data means it may be particularly well-suited for medical education scenarios involving multiple choice questions, but may require additional fine-tuning for other medical text generation tasks.

Training Procedure

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: [More Information Needed]
Hours used: [More Information Needed]
Cloud Provider: [More Information Needed]
Compute Region: [More Information Needed]
Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]

Framework versions

PEFT 0.16.0

Downloads last month: 1

Model tree for inarikami/gemma-12b-medical-mc-qlora-v2

Base model

google/gemma-3-12b-pt

Finetuned

google/gemma-3-12b-it

Adapter

(415)

this model

Paper for inarikami/gemma-12b-medical-mc-qlora-v2

Quantifying the Carbon Emissions of Machine Learning

Paper • 1910.09700 • Published Oct 21, 2019 • 61