metadata
license: agpl-3.0
datasets:
- Bachstelze/PAWS_CoT_explanation
- Bachstelze/GEC_CoT_explanation
- HuggingFaceTB/smol-smoltalk
- nomic-ai/gpt4all-j-prompt-generations
- ZenMoore/RoleBench
- THUDM/AgentInstruct
- Open-Orca/SlimOrca
- WizardLMTeam/WizardLM_evol_instruct_V2_196k
- GAIR/lima
- hamishivi/tulu-3-unfiltered
- ConiferLM/Conifer
- argilla/magpie-ultra-v1.0
- kaist-ai/CoT-Collection
language:
- en
base_model:
- HuggingFaceTB/SmolLM2-135M
SmolSynformer: SmolLM2 as Syntax-aware transformer
SmolSynformer is trained on various instructions including GEC, paraphrase identification and universal dependency generation. Code and math are not included. This model is overfitted on in-context learning and does sometimes generate follow up questions and answers.
Inference with transformer
from transformers import AutoTokenizer, AutoModelForCausalLM, AutoConfig, pipeline
test_model = "Bachstelze/smolSynformer"
model = AutoModelForCausalLM.from_pretrained(test_model)
tokenizer = AutoTokenizer.from_pretrained(test_model)
config = AutoConfig.from_pretrained(test_model)
prompt_pipeline = pipeline("text-generation", model=test_model, tokenizer=tokenizer, max_new_tokens=250)
print(prompt_pipeline("Why is syntax relevant for language modeling and instruction following?\n"))
Example answer:
Syntax is relevant for language modeling and instruction following because it provides a structured and organized way to represent and analyze language. It allows for the creation of rules and patterns that govern how language is used, which can be used to train models to recognize and generate language. Additionally, syntax can be used to identify and classify different types of language, such as grammatical or idiomatic language.