Forget num_classes

Change loss function for multi-category because CrossEntropyLoss only accept outputs of (batch, x) and labels with shape of (batch)

Update requirements

First commit

Temporary fix

Temporary fix

Allow LSTM loading on CPU

Remove debugging code

Use BERT vocab size

test

test

Simplify distilling and inference

Simplify distilling

Fix LSTM tokenizer mismatch

Found a critical bug in LSTM inference when distillation using BERT Tokenizer but inference uses custom LSTM tokenizer

debug

Update inference_lstm

Update LSTM gradient clipping

Add layer normalization to LSTM model

test

test

Remove debug

debug

debug

debug

debug

test

test

test

Fix a bug

Use the original training tokenizer instead of building tokenizer from test data

Use the original training tokenizer instead of creating tokenizer from scratch

Test

Debug tokenizer import

Debug tokenizer import

Debug tokenizer import

Update tokenizer class

Test

Debug

Debug

Idk what error

Edit mismatching vocab size leading for mismatching embedding size for training and inference

Edit error parameters

Save vocab size with model_state_dict

Edit default mismatch in LSTM training and inference

Edit some saving and loading details

Edit error parameters

Edit error parameters in inference_lstm.py

Edit error parameters in inference_lstm.py

Edit error parameters in inference_lstm.py