mehmet1899/llama32-3b-instruct-nl2sql-lora

Llama 3.2 3B Instruct NL2SQL LoRA

This repository contains the final LoRA adapter for the Llama 3.2 3B Instruct model line used in a master's thesis project on local large language models for NL2SQL generation.

The adapter generates SQLite queries from natural-language questions and a relational database schema. It must be loaded together with the corresponding base model.

Base model

• Model: meta-llama/Llama-3.2-3B-Instruct
• Revision: 0cb88a4f764b7a12671c53f0838cd831a0843b95
• Access to the base model may require acceptance of Meta's license terms on Hugging Face.

Adapter

• Method: LoRA supervised fine-tuning
• LoRA rank: 8
• LoRA alpha: 16
• LoRA dropout: 0.05
• Target modules: all suitable linear modules
• Quantization during training: none
• Maximum training sequence length: 2,048 tokens
• Best checkpoint: checkpoint-509
• The published root adapter corresponds to the selected best checkpoint.

SHA-256 of adapter_model.safetensors:

fcd4241f7a2e8e0388f13f0dd9517486cbee43fc3169c983a54e7b716c0e502d

Training configuration

• Training examples: 25,000
• Spider Train examples: 6,960
• SQL Create Context examples: 18,040
• Validation set: MixedVal2500-v2
• Validation examples: 2,500
• Learning rate: 1e-4
• Scheduler: constant
• Train batch size: 2
• Gradient accumulation steps: 4
• Effective batch size: 8
• Seed: 42
• Maximum epochs: 5
• Early stopping patience: 2
• Early stopping threshold: 0.001
• Precision: FP16
• Gradient checkpointing: enabled
• Attention implementation: FlashAttention 2

Spider Dev was not used for training, validation, early stopping, or checkpoint selection.

Evaluation

The final adapter was evaluated on all 1,032 Spider Dev cases.

Zero-shot evaluation:

• Execution Match Accuracy: 61.05% (630/1,032)
• Execution Success Rate: 86.82% (896/1,032)
• Maximum input length: 2,048 tokens
• Maximum generated tokens: 256

The corresponding starting model achieved an Execution Match Accuracy of 55.04% (568/1,032) under the same zero-shot evaluation condition.

Intended prompt behavior

The model is instructed to return only a valid SQLite query:

• no explanation
• no Markdown
• no comments
• no unnecessary tables or columns
• only SELECT or WITH queries
• output terminated with a semicolon

The native Llama chat template and the exact project-specific prompt construction are documented in the accompanying GitHub repository.

Loading

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_id = "meta-llama/Llama-3.2-3B-Instruct"
adapter_id = "mehmet1899/llama32-3b-instruct-nl2sql-lora"
adapter_revision = "87afdd0c565da4570ebd129a4098f50719e0f76e"

tokenizer = AutoTokenizer.from_pretrained(
    base_model_id,
    revision="0cb88a4f764b7a12671c53f0838cd831a0843b95",
)

model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    revision="0cb88a4f764b7a12671c53f0838cd831a0843b95",
    torch_dtype=torch.float16,
    device_map="auto",
)

model = PeftModel.from_pretrained(
    model,
    adapter_id,
    revision=adapter_revision,
)
model.eval()

Reproducibility

Code, training and evaluation configurations, environment information, run manifests, and result summaries are available at:

https://github.com/md181099/nl2sql-masterthesis

The files training_metadata.json, training_history.csv, and training_history.jsonl provide additional training provenance.

Limitations

• The adapter was evaluated primarily on the Spider benchmark and SQLite databases.
• Performance on other database systems or unseen schema conventions is not guaranteed.
• Access to the base model is governed by Meta's model license.
• Execution Match depends on the database contents and the execution-based evaluation procedure.
• The model may still generate invalid, incomplete, or semantically incorrect SQL.
• The adapter should not be used to execute unrestricted queries against production databases without validation and access controls.