---
license: apache-2.0
language:
- en
tags:
- text-to-sql
- code
- sql
- fine-tuned
- unsloth
- lora
base_model: unsloth/gpt-oss-20b
---

# unsloth/gpt-oss-20b Fine-tuned for NL2SQL++ v5.1

This model is a fine-tuned version of [unsloth/gpt-oss-20b](https://huggingface.co/unsloth/gpt-oss-20b) on the private NL2SQL++ v5.1 dataset with code-with-thought reasoning.

## Model Details

- **Base Model**: unsloth/gpt-oss-20b
- **Task**: Text-to-SQL generation
- **Dataset**: NL2SQL++ v8 with code-with-thought reasoning
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation) with Unsloth
- **Quantization**: 16-bit merged weights
- **Training Dataset Size**: (12265, 1) examples
- **Validation Dataset Size**: (1000, 1) examples

## Training Configuration

- **output_dir**: trainer_output
- **overwrite_output_dir**: False
- **do_train**: False
- **do_eval**: True
- **do_predict**: False
- **eval_strategy**: IntervalStrategy.STEPS
- **prediction_loss_only**: False
- **per_device_train_batch_size**: 8
- **per_device_eval_batch_size**: 8
- **per_gpu_train_batch_size**: None
- **per_gpu_eval_batch_size**: None
- **gradient_accumulation_steps**: 8
- **eval_accumulation_steps**: 10
- **eval_delay**: 0
- **torch_empty_cache_steps**: None
- **learning_rate**: 0.0002
- **weight_decay**: 0.01
- **adam_beta1**: 0.9
- **adam_beta2**: 0.999
- **adam_epsilon**: 1e-08
- **max_grad_norm**: 1.0
- **num_train_epochs**: 3.0
- **max_steps**: -1
- **lr_scheduler_type**: SchedulerType.COSINE
- **lr_scheduler_kwargs**: {}
- **warmup_ratio**: 0.1
- **warmup_steps**: 0
- **log_level**: passive
- **log_level_replica**: warning
- **log_on_each_node**: True
- **logging_dir**: trainer_output/runs/Jan14_15-36-27_ip-172-31-42-36.ap-southeast-2.compute.internal
- **logging_strategy**: IntervalStrategy.STEPS
- **logging_first_step**: False
- **logging_steps**: 0.004
- **logging_nan_inf_filter**: True
- **save_strategy**: SaveStrategy.BEST
- **save_steps**: 0.04
- **save_total_limit**: 2
- **save_safetensors**: True
- **save_on_each_node**: False
- **save_only_model**: False
- **restore_callback_states_from_checkpoint**: False
- **no_cuda**: False
- **use_cpu**: False
- **use_mps_device**: False
- **seed**: 3407
- **data_seed**: None
- **jit_mode_eval**: False
- **bf16**: True
- **fp16**: False
- **fp16_opt_level**: O1
- **half_precision_backend**: auto
- **bf16_full_eval**: False
- **fp16_full_eval**: False
- **tf32**: None
- **local_rank**: 0
- **ddp_backend**: None
- **tpu_num_cores**: None
- **tpu_metrics_debug**: False
- **debug**: []
- **dataloader_drop_last**: False
- **eval_steps**: 0.04
- **dataloader_num_workers**: 0
- **dataloader_prefetch_factor**: None
- **past_index**: -1
- **run_name**: None
- **disable_tqdm**: False
- **remove_unused_columns**: True
- **label_names**: None
- **load_best_model_at_end**: True
- **metric_for_best_model**: eval_loss
- **greater_is_better**: False
- **ignore_data_skip**: False
- **fsdp**: []
- **fsdp_min_num_params**: 0
- **fsdp_config**: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- **fsdp_transformer_layer_cls_to_wrap**: None
- **accelerator_config**: AcceleratorConfig(split_batches=False, dispatch_batches=None, even_batches=True, use_seedable_sampler=True, non_blocking=False, gradient_accumulation_kwargs=None, use_configured_state=False)
- **parallelism_config**: None
- **deepspeed**: None
- **label_smoothing_factor**: 0.0
- **optim**: OptimizerNames.PAGED_ADAMW
- **optim_args**: None
- **adafactor**: False
- **group_by_length**: False
- **length_column_name**: length
- **report_to**: ['wandb']
- **project**: huggingface
- **trackio_space_id**: trackio
- **ddp_find_unused_parameters**: None
- **ddp_bucket_cap_mb**: None
- **ddp_broadcast_buffers**: None
- **dataloader_pin_memory**: True
- **dataloader_persistent_workers**: False
- **skip_memory_metrics**: True
- **use_legacy_prediction_loop**: False
- **push_to_hub**: False
- **resume_from_checkpoint**: None
- **hub_model_id**: None
- **hub_strategy**: HubStrategy.EVERY_SAVE
- **hub_token**: None
- **hub_private_repo**: None
- **hub_always_push**: False
- **hub_revision**: None
- **gradient_checkpointing**: True
- **gradient_checkpointing_kwargs**: None
- **include_inputs_for_metrics**: False
- **include_for_metrics**: []
- **eval_do_concat_batches**: True
- **fp16_backend**: auto
- **push_to_hub_model_id**: None
- **push_to_hub_organization**: None
- **push_to_hub_token**: None
- **_n_gpu**: 1
- **mp_parameters**: 
- **auto_find_batch_size**: False
- **full_determinism**: False
- **torchdynamo**: None
- **ray_scope**: last
- **ddp_timeout**: 1800
- **torch_compile**: False
- **torch_compile_backend**: None
- **torch_compile_mode**: None
- **include_tokens_per_second**: False
- **include_num_input_tokens_seen**: no
- **neftune_noise_alpha**: None
- **optim_target_modules**: None
- **batch_eval_metrics**: False
- **eval_on_start**: False
- **use_liger_kernel**: False
- **liger_kernel_config**: None
- **eval_use_gather_object**: False
- **average_tokens_across_devices**: True
- **model_init_kwargs**: None
- **chat_template_path**: None
- **dataset_text_field**: text
- **dataset_kwargs**: None
- **dataset_num_proc**: None
- **eos_token**: None
- **pad_token**: None
- **max_length**: 1024
- **packing**: False
- **packing_strategy**: bfd
- **padding_free**: False
- **pad_to_multiple_of**: None
- **eval_packing**: None
- **completion_only_loss**: None
- **assistant_only_loss**: False
- **loss_type**: nll
- **activation_offloading**: False
- **vllm_sampling_params**: None
- **unsloth_num_chunks**: -1
- **max_seq_length**: 15000
- **model_name**: unsloth/gpt-oss-20b
- **train_batch_size**: 8
- **val_batch_size**: 1
- **num_epochs**: 2
- **lora_r**: 32
- **lora_alpha**: 64


## Train Dataset Example
```
<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.
Knowledge cutoff: 2024-06
Current date: 2026-01-14

Reasoning: low

# Valid channels: analysis, commentary, final. Channel must be included for every message.
Calls to these tools must go to the commentary channel: 'functions'.<|end|><|start|>user<|message|>
You are an expert in SQL++ query generation. You will be given a document schema and a natural language query. You need to generate a valid SQL++ query equivalent to the natural language query.

Bucket Name: `travel-sample`
Scope Name: `inventory`

Use the given document schema to generate the SQL++ query.

Document Schema:
{"landmark": {"properties": {"activity": {"samples": ["buy"], "type": "string"}, "address": {"samples": [["21 Elizabeth St, SW1W 9RP"]], "type": "string"}, "alt": {"samples": [["Kings Head"]], "type": "string"}, "city": {"samples": [["Hornchurch"]], "type": "string"}, "content": {"samples": ["All manner of Middle Eastern f..."], "type": "string"}, "country": {"samples": ["France"], "type": "string"}, "directions": {"samples": [["Located just south of Hornchur..."]], "type": "string"}, "email": {"samples": [["contact@noevalleybakery.com"]], "type": "string"}, "geo": {"properties": {"accuracy": {"samples": ["APPROXIMATE"], "type": "string"}, "lat": {"samples": [50.120496], "type": "number"}, "lon": {"samples": [-5.535092], "type": "number"}}, "type": "object"}, "hours": {"samples": [["M-F 7:30AM-4PM, Sa 8AM-4PM, Su..."]], "type": "string"}, "id": {"samples": [11965], "type": "number"}, "image": {"samples": [["https://en.wikivoyage.org/wiki..."]], "type": "string"}, "image_direct_url": {"samples": ["https://upload.wikimedia.org/w..."], "type": "string"}, "name": {"samples": ["Friar's Inn"], "type": "string"}, "phone": {"samples": [["+44 1708 450894"]], "type": "string"}, "price": {"samples": [["Prices and opening hours set i..."]], "type": "string"}, "state": {"samples": [["California"]], "type": "string"}, "title": {"samples": ["London/Hammersmith and Fulham"], "type": "string"}, "tollfree": {"samples": [["+1 800 546-2070"]], "type": "string"}, "type": {"samples": ["landmark"], "type": "string"}, "url": {"samples": [["http://www.farmtablesf.com"]], "type": "string"}}, "type": "object"}, "route": {"properties": {"airline": {"samples": ["AS"], "type": "string"}, "airlineid": {"samples": ["airline_1316"], "type": "string"}, "destinationairport": {"samples": ["ATL"], "type": "string"}, "distance": {"samples": [960.5866347178], "type": "number"}, "equipment": {"samples": [["73G"]], "type": "string"}, "id": {"samples": [11746], "type": "number"}, "schedule": {"items": {"properties": {"day": {"type": "number"}, "flight": {"type": "string"}, "utc": {"type": "string"}}, "type": "object"}, "samples": [[{"day": 0, "flight": "AS298", "utc": "06:09:00"}]], "type": "array"}, "sourceairport": {"samples": ["CMN"], "type": "string"}, "stops": {"samples": [0], "type": "number"}, "type": {"samples": ["route"], "type": "string"}}, "type": "object"}, "airport": {"properties": {"airportname": {"samples": ["Harrisburg Intl"], "type": "string"}, "city": {"samples": ["Harrisburg"], "type": "string"}, "country": {"samples": ["France"], "type": "string"}, "faa": {"samples": [["JFK"]], "type": "string"}, "geo": {"properties": {"alt": {"samples": [8], "type": "number"}, "lat": {"samples": [25.79325], "type": "number"}, "lon": {"samples": [-80.290556], "type": "number"}}, "type": "object"}, "icao": {"samples": [["EGEO"]], "type": "string"}, "id": {"samples": [3542], "type": "number"}, "type": {"samples": ["airport"], "type": "string"}, "tz": {"samples": ["America/Anchorage"], "type": "string"}}, "type": "object"}, "airline": {"properties": {"callsign": {"samples": [["AIRLINAIR"]], "type": "string"}, "country": {"samples": ["France"], "type": "string"}, "iata": {"samples": [["A5"]], "type": "string"}, "icao": {"samples": ["CCM"], "type": "string"}, "id": {"samples": [10], "type": "number"}, "name": {"samples": ["40-Mile Air"], "type": "string"}, "type": {"samples": ["airline"], "type": "string"}}, "type": "object"}, "hotel": {"properties": {"address": {"samples": [["39 Abbey Street, Armagh, BT61 ..."]], "type": "string"}, "alias": {"samples": [["Inn Exile"]], "type": "string"}, "checkin": {"samples": [["3PM"]], "type": "string"}, "checkout": {"samples": [["11AM"]], "type": "string"}, "city": {"samples": [["Argyll and Bute"]], "type": "string"}, "country": {"samples": ["France"], "type": "string"}, "description": {"samples": ["28 bed SYHA hostel, with beds ..."], "type": "string"}, "directions": {"samples": [["M\u00e9tro: Concorde"]], "type": "string"}, "email": {"samples": [["info@Lajoyainn.com"]], "type": "string"}, "fax": {"samples": [["+1 415 665-5440"]], "type": "string"}, "free_breakfast": {"samples": [false], "type": "boolean"}, "free_internet": {"samples": [false], "type": "boolean"}, "free_parking": {"samples": [false], "type": "boolean"}, "geo": {"properties": {"accuracy": {"samples": ["APPROXIMATE"], "type": "string"}, "lat": {"samples": [52.44779], "type": "number"}, "lon": {"samples": [-6.6584], "type": "number"}}, "type": "object"}, "id": {"samples": [1097], "type": "number"}, "name": {"samples": ["Armagh City Youth Hostel"], "type": "string"}, "pets_ok": {"samples": [false], "type": "boolean"}, "phone": {"samples": [["+44 121 236 4031"]], "type": "string"}, "price": {"samples": [["$12 a night"]], "type": "string"}, "public_likes": {"items": {"type": "string"}, "samples": [["Caden Schinner"]], "type": "array"}, "reviews": {"items": {"properties": {"author": {"samples": ["Imelda Renner"], "type": "string"}, "content": {"samples": ["Dont be mislead by some hotel ..."], "type": "string"}, "date": {"samples": ["2012-11-10 17:29:26 +0300"], "type": "string"}, "ratings": {"properties": {"Business service": {"samples": [-1], "type": "number"}, "Business service (e.g., internet access)": {"samples": [1], "type": "number"}, "Check in / front desk": {"samples": [-1], "type": "number"}, "Cleanliness": {"samples": [-1], "type": "number"}, "Location": {"samples": [-1], "type": "number"}, "Overall": {"samples": [1], "type": "number"}, "Rooms": {"samples": [-1], "type": "number"}, "Service": {"samples": [-1], "type": "number"}, "Sleep Quality": {"samples": [2], "type": "number"}, "Value": {"samples": [-1], "type": "number"}}, "type": "object"}}, "type": "object"}, "samples": [[{"author": "Bart Simonis PhD", "content": "I booked two rooms for Saturday so my wife and I could take our two young children and her parents t...", "date": "2013-12-18 22:51:08 +0300", "ratings": {"Cleanliness": 5, "Location": 5, "Overall": 3, "Rooms": 3, "Service": 5, "Sleep Quality": 5, "Value": 5}}]], "type": "array"}, "state": {"samples": [["California"]], "type": "string"}, "title": {"samples": ["Armagh"], "type": "string"}, "tollfree": {"samples": [["+1-800-962-0186"]], "type": "string"}, "type": {"samples": ["hotel"], "type": "string"}, "url": {"samples": [["http://www.cpbirminghamnechote..."]], "type": "string"}, "vacancy": {"samples": [false], "type": "boolean"}}, "type": "object"}}

Natural Language Query
Which routes in the route collection rank first by the shortest distance within each destination airport when limited to seven results?

SQL++ Query:
<|end|><|start|>assistant<|message|>
```sql++
SELECT d.id, d.destinationairport, ROW_NUMBER() OVER (PARTITION BY d.destinationairport ORDER BY d.distance NULLS FIRST) AS `row` FROM route AS d LIMIT 7;
```
<|return|>
```