metadata
license: apache-2.0
base_model:
- nvidia/Llama-3_3-Nemotron-Super-49B-v1_5
pipeline_tag: text-generation
datasets:
- ConicCat/Gutenberg-SFT
- ConicCat/Condor-SFT-Filtered
ConicCat/Llama3_3-Nemo-Super-Writer-49B
A writing / roleplay finetune of Nemo Super 49B.
Features:
- Improved longform writing capabilites; output context extension allows for prompting for up to 4000 words of text in one go.
- Markedly less AI slop in writing.
- Fewer 'soft' refusals in writing.
Datasets
- internlm/Condor-SFT-20K for instruct; even though instruct capabilities are not the primary focus, adding some instruct data helps mitigate forgetting and maintains general intellect and instruction following capabilites.
- ConicCat/Gutenberg-SFT. A reformatted version of the original Gutenberg DPO dataset by jondurbin for SFT with some slight augmentation to address many of the samples being overly long.
- A dataset of backtranslated books. Unfortunately, I am unable to release this set as all of the data is under copyright.
- Some synthetic GLM-4.7 roleplay data.
- A dash of a certain third owned archive.