GIST-small-cosmopedia-v1

A fine-tuned version of the bert architecture (BertForSequenceClassification) optimized for the text-classification task.

• Model type: bert
• Problem Type: single_label_classification
• Number of Labels: 113
• Vocabulary Size: 30522
• License: MIT

Use

To get started with this model in Python using the Hugging Face Transformers library, run the following code:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "agentlans/GIST-small-cosmopedia-v1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Replace this with your input text."
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
predicted_class_name = model.config.id2label[predicted_class_id]

print(f"Predicted Class ID: {predicted_class_id}")
print(f"Predicted Class Name: {predicted_class_name}")

Intended Uses & Limitations

Intended Use

This model is designed for sequence classification tasks. Below are the specific class labels mapped to their corresponding IDs:

Label ID	Label Name
0	Addiction and Mental Illness
1	American Football
2	Arts and Crafts
3	Astrology
4	Astronomy and Astrophysics
5	Audio Equipment and Home Theater Systems
6	Automotive Parts and Accessories
7	Baseball
8	Biochemistry and Molecular Biology
9	Business and Entrepreneurship
10	Business and Management
11	Cannabis and CBD Products
12	Career Development and Job Opportunities
13	Christian Theology and Spirituality
14	Christianity and Theology
15	Cleaning and Maintenance
16	Computer Antivirus Software and Security
17	Computer Hardware and Graphics Cards
18	Computer Programming and Web Development
19	Computer Science
20	Computer Security & Privacy
21	Cooking and Baking
22	Cooking and Recipes
23	Cosmetic Surgery and Body Modifications
24	Cricket
25	Cryptocurrency and Blockchain Technology
26	Culinary Arts and Beverages
27	Data Privacy and Protection
28	Dentistry
29	Digital Imaging and Photography
30	Digital Marketing and Business
31	Economics and Finance
32	Education
33	Education and Youth Development
34	Electric Vehicles and Battery Technology
35	Energy and Environmental Policy
36	Energy and Natural Resources
37	Entomology and Apiculture
38	Events and Community Happenings
39	Fashion & Apparel
40	Fiction and Fantasy Writing
41	Finance and Investment
42	Fire Incidents
43	Football/Soccer
44	Genetics and Mental Health
45	Geography and Weather
46	Gun Control and Violence
47	HIV Treatment and Care
48	Hair Care
49	Hair Care and Styling
50	Health and Lifestyle
51	Healthcare & Medical Services
52	Healthcare and Operations Management
53	Home Improvement and Maintenance
54	Human Resources / Organizational Management
55	Human Resources and Education
56	Ice Hockey
57	Infant Feeding and Child Development
58	Insurance
59	International Relations and Conflict
60	International Relations and Current Events
61	International Relations and Politics
62	Jewelry Design and Manufacturing
63	Leadership and Education
64	Legal Services and Issues
65	Legal Studies / Law
66	Legal Studies and Public Policy
67	Lighting Design and Technology
68	Literature and Creative Writing
69	Loans and Mortgages
70	Marketing and Business Strategies
71	Medicine
72	Mental Health Counseling
73	Mental Health and Therapy
74	Molecular Biology and Genetics
75	Moving Services and Logistics
76	Music
77	Nutrition and Health
78	Online Chat Platforms and Data Privacy
79	Online Dating & Relationships
80	Online Platforms & Web Technologies
81	Performing Arts
82	Personal Development and Empowerment
83	Personal Finance and Investments
84	Pets and Pet Care
85	Pharmaceutical manufacturing and technology
86	Physical Fitness and Health
87	Political Science
88	Politics and Government
89	Product Marketing and Design
90	Professional Basketball/NBA
91	Professional Wrestling and Sports Entertainment
92	Psychology
93	Public Administration and Policy
94	Public Safety and Emergency Response
95	Public Transit and Transportation
96	Real Estate & Investment
97	Recreational Fishing
98	Skincare and Beauty Products
99	Sports and Education
100	Taxation and Finance
101	Technology and Computer Science
102	Technology and Consumer Electronics
103	Tennis
104	Transportation and City Planning
105	Travel
106	Video Games
107	Visual Arts and Art Appreciation
108	Waste Management and Recycling
109	Watchmaking and Horology
110	Weddings
111	Wine & Winemaking
112	Writing and Storytelling

Training Details

Hyperparameters

The following hyperparameters were used during fine-tuning:

• Learning Rate: 5e-05
• Train Batch Size: 8
• Eval Batch Size: 8
• Optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• Number of Epochs: 3.0
• Mixed Precision: BF16

Show Advanced Training Configuration

Optimization & Regularization

• Gradient Accumulation Steps: 1
• Learning Rate Scheduler: SchedulerType.LINEAR
• Warmup Steps: 0
• Warmup Ratio: None
• Weight Decay: 0.0
• Max Gradient Norm: 1.0

Hardware & Reproducibility

• Number of GPUs: 1
• Seed: 42

Training Results & Evaluation

During fine-tuning, the model achieved the following results on the evaluation set:

Metric	Value
Train Loss	1.0233
Validation Loss	1.0927
Validation F1 Score	0.6313
Total FLOPs	9.8351e+15

For performance on the test set, click here.

Speed Performance

• Training Runtime: 1304.8729 seconds
• Train Samples per Second: 456.771
• Evaluation Runtime: 26.6644 seconds
• Eval Samples per Second: 1862.781

Show Detailed Training Logs

Training Logs History

Step	Epoch	Learning Rate	Training Loss	Validation Loss	Validation F1
500	0.02	4.9665e-05	3.8688	N/A	N/A
1000	0.04	4.9330e-05	2.8619	N/A	N/A
1500	0.06	4.8994e-05	2.4201	N/A	N/A
2000	0.081	4.8658e-05	2.161	N/A	N/A
2500	0.101	4.8323e-05	1.9853	N/A	N/A
3000	0.121	4.7987e-05	1.8134	N/A	N/A
3500	0.141	4.7652e-05	1.708	N/A	N/A
4000	0.161	4.7316e-05	1.6879	N/A	N/A
4500	0.181	4.6981e-05	1.648	N/A	N/A
5000	0.201	4.6645e-05	1.6214	N/A	N/A
5500	0.221	4.6310e-05	1.5573	N/A	N/A
6000	0.242	4.5974e-05	1.4505	N/A	N/A
6500	0.262	4.5639e-05	1.4213	N/A	N/A
7000	0.282	4.5303e-05	1.4116	N/A	N/A
7500	0.302	4.4967e-05	1.4007	N/A	N/A
8000	0.322	4.4632e-05	1.361	N/A	N/A
8500	0.342	4.4296e-05	1.3746	N/A	N/A
9000	0.362	4.3961e-05	1.2952	N/A	N/A
9500	0.383	4.3625e-05	1.2967	N/A	N/A
10000	0.403	4.3290e-05	1.3186	N/A	N/A
10500	0.423	4.2954e-05	1.316	N/A	N/A
11000	0.443	4.2619e-05	1.2962	N/A	N/A
11500	0.463	4.2283e-05	1.2782	N/A	N/A
12000	0.483	4.1948e-05	1.2217	N/A	N/A
12500	0.503	4.1612e-05	1.2416	N/A	N/A
13000	0.523	4.1276e-05	1.2418	N/A	N/A
13500	0.544	4.0941e-05	1.2475	N/A	N/A
14000	0.564	4.0605e-05	1.2107	N/A	N/A
14500	0.584	4.0270e-05	1.2335	N/A	N/A
15000	0.604	3.9934e-05	1.2252	N/A	N/A
15500	0.624	3.9599e-05	1.2382	N/A	N/A
16000	0.644	3.9263e-05	1.1816	N/A	N/A
16500	0.664	3.8928e-05	1.2079	N/A	N/A
17000	0.685	3.8592e-05	1.2031	N/A	N/A
17500	0.705	3.8256e-05	1.2082	N/A	N/A
18000	0.725	3.7921e-05	1.2155	N/A	N/A
18500	0.745	3.7585e-05	1.1903	N/A	N/A
19000	0.765	3.7250e-05	1.1614	N/A	N/A
19500	0.785	3.6914e-05	1.1556	N/A	N/A
20000	0.805	3.6579e-05	1.1657	N/A	N/A
20500	0.825	3.6243e-05	1.1904	N/A	N/A
21000	0.846	3.5908e-05	1.1701	N/A	N/A
21500	0.866	3.5572e-05	1.1741	N/A	N/A
22000	0.886	3.5237e-05	1.14	N/A	N/A
22500	0.906	3.4901e-05	1.1826	N/A	N/A
23000	0.926	3.4565e-05	1.1334	N/A	N/A
23500	0.946	3.4230e-05	1.1562	N/A	N/A
24000	0.966	3.3894e-05	1.0916	N/A	N/A
24500	0.987	3.3559e-05	1.1356	N/A	N/A
24835	1.0	N/A	N/A	1.1201	0.5718
25000	1.007	3.3223e-05	1.0691	N/A	N/A
25500	1.027	3.2888e-05	0.949	N/A	N/A
26000	1.047	3.2552e-05	0.9389	N/A	N/A
26500	1.067	3.2217e-05	0.9352	N/A	N/A
27000	1.087	3.1881e-05	0.9251	N/A	N/A
27500	1.107	3.1546e-05	0.9181	N/A	N/A
28000	1.127	3.1210e-05	0.9674	N/A	N/A
28500	1.148	3.0874e-05	0.9783	N/A	N/A
29000	1.168	3.0539e-05	0.9192	N/A	N/A
29500	1.188	3.0203e-05	0.9751	N/A	N/A
30000	1.208	2.9868e-05	0.9555	N/A	N/A
30500	1.228	2.9532e-05	0.917	N/A	N/A
31000	1.248	2.9197e-05	0.9963	N/A	N/A
31500	1.268	2.8861e-05	0.9668	N/A	N/A
32000	1.289	2.8526e-05	0.9155	N/A	N/A
32500	1.309	2.8190e-05	0.8989	N/A	N/A
33000	1.329	2.7855e-05	0.9209	N/A	N/A
33500	1.349	2.7519e-05	0.9203	N/A	N/A
34000	1.369	2.7183e-05	0.9156	N/A	N/A
34500	1.389	2.6848e-05	0.9333	N/A	N/A
35000	1.409	2.6512e-05	0.904	N/A	N/A
35500	1.429	2.6177e-05	0.9448	N/A	N/A
36000	1.45	2.5841e-05	0.9495	N/A	N/A
36500	1.47	2.5506e-05	0.9248	N/A	N/A
37000	1.49	2.5170e-05	0.9464	N/A	N/A
37500	1.51	2.4835e-05	0.9223	N/A	N/A
38000	1.53	2.4499e-05	0.9354	N/A	N/A
38500	1.55	2.4163e-05	0.9917	N/A	N/A
39000	1.57	2.3828e-05	0.921	N/A	N/A
39500	1.59	2.3492e-05	0.9341	N/A	N/A
40000	1.611	2.3157e-05	0.9443	N/A	N/A
40500	1.631	2.2821e-05	0.9163	N/A	N/A
41000	1.651	2.2486e-05	0.9375	N/A	N/A
41500	1.671	2.2150e-05	0.9126	N/A	N/A
42000	1.691	2.1815e-05	0.9315	N/A	N/A
42500	1.711	2.1479e-05	0.9323	N/A	N/A
43000	1.731	2.1144e-05	0.9468	N/A	N/A
43500	1.752	2.0808e-05	0.9616	N/A	N/A
44000	1.772	2.0472e-05	0.9257	N/A	N/A
44500	1.792	2.0137e-05	0.9199	N/A	N/A
45000	1.812	1.9801e-05	0.9237	N/A	N/A
45500	1.832	1.9466e-05	0.9252	N/A	N/A
46000	1.852	1.9130e-05	0.9198	N/A	N/A
46500	1.872	1.8795e-05	0.9251	N/A	N/A
47000	1.892	1.8459e-05	0.8996	N/A	N/A
47500	1.913	1.8124e-05	0.8661	N/A	N/A
48000	1.933	1.7788e-05	0.9119	N/A	N/A
48500	1.953	1.7453e-05	0.9093	N/A	N/A
49000	1.973	1.7117e-05	0.8893	N/A	N/A
49500	1.993	1.6781e-05	0.9163	N/A	N/A
49670	2.0	N/A	N/A	1.0531	0.6141
50000	2.013	1.6446e-05	0.7699	N/A	N/A
50500	2.033	1.6110e-05	0.7102	N/A	N/A
51000	2.054	1.5775e-05	0.7316	N/A	N/A
51500	2.074	1.5439e-05	0.7196	N/A	N/A
52000	2.094	1.5104e-05	0.706	N/A	N/A
52500	2.114	1.4768e-05	0.7302	N/A	N/A
53000	2.134	1.4433e-05	0.719	N/A	N/A
53500	2.154	1.4097e-05	0.6997	N/A	N/A
54000	2.174	1.3761e-05	0.7478	N/A	N/A
54500	2.194	1.3426e-05	0.7221	N/A	N/A
55000	2.215	1.3090e-05	0.6901	N/A	N/A
55500	2.235	1.2755e-05	0.7306	N/A	N/A
56000	2.255	1.2419e-05	0.6928	N/A	N/A
56500	2.275	1.2084e-05	0.7387	N/A	N/A
57000	2.295	1.1748e-05	0.6852	N/A	N/A
57500	2.315	1.1413e-05	0.7277	N/A	N/A
58000	2.335	1.1077e-05	0.6858	N/A	N/A
58500	2.356	1.0742e-05	0.6819	N/A	N/A
59000	2.376	1.0406e-05	0.7408	N/A	N/A
59500	2.396	1.0070e-05	0.7442	N/A	N/A
60000	2.416	9.7349e-06	0.7565	N/A	N/A
60500	2.436	9.3994e-06	0.7308	N/A	N/A
61000	2.456	9.0638e-06	0.6853	N/A	N/A
61500	2.476	8.7283e-06	0.7294	N/A	N/A
62000	2.496	8.3927e-06	0.7037	N/A	N/A
62500	2.517	8.0572e-06	0.7001	N/A	N/A
63000	2.537	7.7216e-06	0.7086	N/A	N/A
63500	2.557	7.3861e-06	0.7082	N/A	N/A
64000	2.577	7.0505e-06	0.7219	N/A	N/A
64500	2.597	6.7150e-06	0.6902	N/A	N/A
65000	2.617	6.3794e-06	0.6866	N/A	N/A
65500	2.637	6.0439e-06	0.7045	N/A	N/A
66000	2.658	5.7083e-06	0.7051	N/A	N/A
66500	2.678	5.3728e-06	0.6698	N/A	N/A
67000	2.698	5.0372e-06	0.6839	N/A	N/A
67500	2.718	4.7017e-06	0.6723	N/A	N/A
68000	2.738	4.3661e-06	0.6874	N/A	N/A
68500	2.758	4.0306e-06	0.7674	N/A	N/A
69000	2.778	3.6951e-06	0.7345	N/A	N/A
69500	2.798	3.3595e-06	0.6404	N/A	N/A
70000	2.819	3.0240e-06	0.6571	N/A	N/A
70500	2.839	2.6884e-06	0.6858	N/A	N/A
71000	2.859	2.3529e-06	0.7133	N/A	N/A
71500	2.879	2.0173e-06	0.7026	N/A	N/A
72000	2.899	1.6818e-06	0.7178	N/A	N/A
72500	2.919	1.3462e-06	0.6567	N/A	N/A
73000	2.939	1.0107e-06	0.6738	N/A	N/A
73500	2.96	6.7512e-07	0.6513	N/A	N/A
74000	2.98	3.3957e-07	0.6676	N/A	N/A
74500	3.0	4.0266e-09	0.7003	N/A	N/A
74505	3.0	N/A	N/A	1.0927	0.6313

Framework Versions

• Transformers: 5.14.0.dev0
• PyTorch: 2.13.0+cu130