[BUG/Help] <title>使用ptuning微调时报错RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())]

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

Traceback (most recent call last):
  File "main.py", line 429, in <module>
    main()
  File "main.py", line 112, in main
    tokenizer = AutoTokenizer.from_pretrained(model_args.model_name_or_path, trust_remote_code=True)
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 679, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1804, in from_pretrained
    return cls._from_pretrained(
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1958, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/tokenization_chatglm.py", line 205, in __init__
    self.sp_tokenizer = SPTokenizer(vocab_file, num_image_tokens=num_image_tokens)
  File "/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/tokenization_chatglm.py", line 61, in __init__
    self.text_tokenizer = TextTokenizer(vocab_file)
  File "/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/tokenization_chatglm.py", line 22, in __init__
    self.sp.Load(model_path)
  File "/root/miniconda3/lib/python3.8/site-packages/sentencepiece/__init__.py", line 905, in Load
    return self.LoadFromFile(model_file)
  File "/root/miniconda3/lib/python3.8/site-packages/sentencepiece/__init__.py", line 310, in LoadFromFile
    return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)
RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())]

### Expected Behavior

_No response_

### Steps To Reproduce

使用ptuning微调时报错，已经是最新版的模型文件了

### Environment

```markdown
PyTorch  1.11.0
Python  3.8(ubuntu20.04)
Cuda  11.3
```

### Anything else?

_No response_


为什么不能用共享GPU内存呢[Feature] <title>

### Is your feature request related to a problem? Please describe.

为什么不能用共享GPU内存呢
专用6G都满了但是共享GPU内存一点都没动

### Solutions

emm

### Additional context

_No response_


### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

运行python cli_demo.py报错

root@4uot40mdrplpv-0:/yx/ChatGLM-6B# python mycli_demo.py
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Traceback (most recent call last):
  File "/yx/ChatGLM-6B/mycli_demo.py", line 6, in <module>
    tokenizer = AutoTokenizer.from_pretrained("/yx/ChatGLM-6B/THUDM/chatglm-6b", trust_remote_code=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 679, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 1804, in from_pretrained
    return cls._from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 1958, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/tokenization_chatglm.py", line 205, in __init__
    self.sp_tokenizer = SPTokenizer(vocab_file, num_image_tokens=num_image_tokens)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/tokenization_chatglm.py", line 61, in __init__
    self.text_tokenizer = TextTokenizer(vocab_file)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/tokenization_chatglm.py", line 22, in __init__
    self.sp.Load(model_path)
  File "/usr/local/lib/python3.11/site-packages/sentencepiece/__init__.py", line 905, in Load
    return self.LoadFromFile(model_file)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sentencepiece/__init__.py", line 310, in LoadFromFile
    return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())]

我是在docker中运行的,  麻烦看看是怎么回事, 谢谢

### Expected Behavior

_No response_

### Steps To Reproduce

help

### Environment

```markdown
- OS:Red Hat 4.8.5-44
- Python:3.11
- Transformers:4.27.1
- PyTorch:2.0
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :False
```

### Anything else?

_No response_


How to fix error like this: torch.cuda.OutOfMemoryError: CUDA out of memory ?

OS: ubuntu 20.04
The error message said we need to change value of max_split_size_mb, but I search source code and cannot find any file contains max_split_size_mb, would you please provide some guidance about how to fix?
```
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:16<00:00,  2.09s/it]
Traceback (most recent call last):
  File "/home/zhangclb/sandbox/ai_llm/ChatGLM-6B/cli_demo.py", line 6, in <module>
    model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).half().cuda()
  File "/home/zhangclb/.local/env39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 749, in cuda
    return self._apply(lambda t: t.cuda(device))
  File "/home/zhangclb/.local/env39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 641, in _apply
    module._apply(fn)
  File "/home/zhangclb/.local/env39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 641, in _apply
    module._apply(fn)
  File "/home/zhangclb/.local/env39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 641, in _apply
    module._apply(fn)
  [Previous line repeated 2 more times]
  File "/home/zhangclb/.local/env39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 664, in _apply
    param_applied = fn(param)
  File "/home/zhangclb/.local/env39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 749, in <lambda>
    return self._apply(lambda t: t.cuda(device))
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 128.00 MiB (GPU 0; 1.83 GiB total capacity; 1.27 GiB already allocated; 57.19 MiB free; 1.28 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

```


[Help] <title> Oracle Linux 7.9 运行int4模型出错，AttributeError: 'NoneType' object has no attribute 'int4WeightExtractionFloat'

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

/x/home/chatglm_env/lib/python3.7/site-packages/requests/__init__.py:104: RequestsDependencyWarning: urllib3 (1.26.14) or chardet (5.1.0)/charset_normalizer (2.0.12) doesn't match a supported version!
  RequestsDependencyWarning)
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
/x/home/chatglm_env/lib/python3.7/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: libc10_cuda.so: cannot open shared object file: No such file or directory
  warn(f"Failed to load image Python extension: {e}")
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
No compiled kernel found.
Compiling kernels : /x/home/.cache/huggingface/modules/transformers_modules/local/quantization_kernels_parallel.c
Compiling gcc -O3 -fPIC -pthread -fopenmp -std=c99 /x/home/.cache/huggingface/modules/transformers_modules/local/quantization_kernels_parallel.c -shared -o /x/home/.cache/huggingface/modules/transformers_modules/local/quantization_kernels_parallel.so
sh: gcc: command not found
Compile failed, using default cpu kernel code.
Compiling gcc -O3 -fPIC -std=c99 /x/home/.cache/huggingface/modules/transformers_modules/local/quantization_kernels.c -shared -o /x/home/.cache/huggingface/modules/transformers_modules/local/quantization_kernels.so
Kernels compiled : /x/home/.cache/huggingface/modules/transformers_modules/local/quantization_kernels.so
Cannot load cpu kernel, don't use quantized model on cpu.
Using quantization cache
Applying quantization to glm layers
Traceback (most recent call last):
  File "chatglm-int4-demo.py", line 8, in <module>
    response, history = model.chat(tokenizer, '你好', history=[])
  File "/x/home/chatglm_env/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/x/home/.cache/huggingface/modules/transformers_modules/local/modeling_chatglm.py", line 1137, in chat
    outputs = self.generate(**input_ids, **gen_kwargs)
  File "/x/home/chatglm_env/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/x/home/chatglm_env/lib/python3.7/site-packages/transformers/generation/utils.py", line 1447, in generate
    **model_kwargs,
  File "/x/home/chatglm_env/lib/python3.7/site-packages/transformers/generation/utils.py", line 2447, in sample
    output_hidden_states=output_hidden_states,
  File "/x/home/chatglm_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/x/home/.cache/huggingface/modules/transformers_modules/local/modeling_chatglm.py", line 1051, in forward
    return_dict=return_dict,
  File "/x/home/chatglm_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/x/home/.cache/huggingface/modules/transformers_modules/local/modeling_chatglm.py", line 887, in forward
    output_attentions=output_attentions
  File "/x/home/chatglm_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/x/home/.cache/huggingface/modules/transformers_modules/local/modeling_chatglm.py", line 588, in forward
    output_attentions=output_attentions
  File "/x/home/chatglm_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/x/home/.cache/huggingface/modules/transformers_modules/local/modeling_chatglm.py", line 406, in forward
    mixed_raw_layer = self.query_key_value(hidden_states)
  File "/x/home/chatglm_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/x/home/.cache/huggingface/modules/transformers_modules/local/quantization.py", line 334, in forward
    output = W8A16LinearCPU.apply(input, self.weight, self.weight_scale, self.weight_bit_width, self.quantization_cache)
  File "/x/home/.cache/huggingface/modules/transformers_modules/local/quantization.py", line 74, in forward
    weight = extract_weight_to_float(quant_w, scale_w, weight_bit_width, quantization_cache=quantization_cache)
  File "/x/home/.cache/huggingface/modules/transformers_modules/local/quantization.py", line 256, in extract_weight_to_float
    func = cpu_kernels.int4WeightExtractionFloat
AttributeError: 'NoneType' object has no attribute 'int4WeightExtractionFloat'

### Expected Behavior

_No response_

### Steps To Reproduce

from transformers import AutoTokenizer, AutoModel

model_path = '/x/home/chatglm-6b-int4'

tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
model = AutoModel.from_pretrained(model_path, trust_remote_code=True).float()

response, history = model.chat(tokenizer, '你好', history=[])

### Environment

```markdown
- OS: Oracle 7.9
- Python: 3.7.13
- Transformers: 2.6.1
- PyTorch: 1.13.1
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) : no cuda, use cpu
```

### Anything else?

_No response_


可以使用自己的数据微调吗


6GB显卡提示显存不足

显卡：3060laptop 6GB
报错：RuntimeError: CUDA out of memory. Tried to allocate 96.00 MiB (GPU 0; 6.00 GiB total capacity; 5.27 GiB already allocated; 0 bytes free; 5.28 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

{'base_commit': 'c55ecd89a079b86620cc722f2e21a14e3718d0f3', 'files': [{'path': 'web_demo.py', 'Loc': {'(None, None, None)': {'mod': [5]}}, 'status': 'modified'}, {'path': 'cli_demo.py', 'Loc': {'(None, None, None)': {'mod': [6]}}, 'status': 'modified'}]}


[BUG/Help] 这两天更新版本后，chat的微调好像用不了了

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

前几天使用chat微调还是可以用的，那时候output文件是完整的包，而不是增量微调包。
这两天更新后，使用的还是项目自带的train_chat.sh，模型用的是int4。
output文件确实小了，但是却无法运行了，具体形式为运行以下代码
```python
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("/content/ChatGLM-6B/ptuning/output/chattm/checkpoint-50", trust_remote_code=True)
model = AutoModel.from_pretrained("/content/ChatGLM-6B/ptuning/output/chattm/checkpoint-50", trust_remote_code=True).half().cuda()
model = model.eval()
response, history = model.chat(tokenizer, "你好", history=[])
print(response)
```
报以下内容后无反应，至少5分钟。期间显存一直在上升
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
The dtype of attention mask (torch.int64) is not bool
最终报错
2023-04-11 13:51:41.577016: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT

### Expected Behavior

_No response_

### Steps To Reproduce

-

### Environment

```markdown
colab pro 默认环境
```

### Anything else?

_No response_


centos上，2个12g显存的显卡如何配置可以同时使用

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

centos上，2个12g显存的显卡，无论是训练还是web，都始终用0号显卡，如何配置可以同时使用

### Expected Behavior

_No response_

### Steps To Reproduce

Centos7
12G nvida *2

### Environment

```markdown
- OS:Centos7
- Python:3.8
- Transformers:4.26.1
- PyTorch: 1.12
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :True
```

### Anything else?

_No response_


[BUG/Help] ValueError: 150000 is not in list

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

  0%|          | 19/30000 [31:30<828:54:23, 99.53s/it]
  0%|          | 20/30000 [33:09<828:37:17, 99.50s/it]
  0%|          | 21/30000 [34:48<828:09:42, 99.45s/it]Traceback (most recent call last):
  File "/root/projects/ChatGLM-6B/ptuning/main.py", line 393, in <module>
    main()
  File "/root/projects/ChatGLM-6B/ptuning/main.py", line 332, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/root/anaconda3/envs/torch10/lib/python3.9/site-packages/transformers/trainer.py", line 1633, in train
    return inner_training_loop(
  File "/root/anaconda3/envs/torch10/lib/python3.9/site-packages/transformers/trainer.py", line 1902, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs)
  File "/root/anaconda3/envs/torch10/lib/python3.9/site-packages/transformers/trainer.py", line 2645, in training_step
    loss = self.compute_loss(model, inputs)
  File "/root/anaconda3/envs/torch10/lib/python3.9/site-packages/transformers/trainer.py", line 2677, in compute_loss
    outputs = model(**inputs)
  File "/root/anaconda3/envs/torch10/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/.cache/huggingface/modules/transformers_modules/modeling_chatglm.py", line 1160, in forward
    transformer_outputs = self.transformer(
  File "/root/anaconda3/envs/torch10/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/.cache/huggingface/modules/transformers_modules/modeling_chatglm.py", line 928, in forward
    mask_positions = [seq.tolist().index(mask_token) for seq in input_ids]
  File "/root/.cache/huggingface/modules/transformers_modules/modeling_chatglm.py", line 928, in <listcomp>
    mask_positions = [seq.tolist().index(mask_token) for seq in input_ids]
ValueError: 150000 is not in list

### Expected Behavior

_No response_

### Steps To Reproduce

PRE_SEQ_LEN=8
LR=1e-2

CUDA_VISIBLE_DEVICES=0 python3 main.py \
    --do_train \
    --train_file ../data/train.json \
    --validation_file ../data/dev.json \
    --prompt_column instruction \
    --response_column output \
    --overwrite_cache \
    --model_name_or_path ~/projects/zero_nlp/simple_thu_chatglm6b/thuglm/ \
    --output_dir output/adgen-chatglm-6b-pt-$PRE_SEQ_LEN-$LR \
    --overwrite_output_dir \
    --max_source_length 64 \
    --max_target_length 64 \
    --per_device_train_batch_size 100 \
    --per_device_eval_batch_size 100 \
    --gradient_accumulation_steps 16 \
    --predict_with_generate \
    --max_steps 30000 \
    --logging_steps 100 \
    --save_steps 100 \
    --learning_rate $LR \
    --pre_seq_len $PRE_SEQ_LEN \
    # --quantization_bit 4

### Environment

```markdown
- OS: centos8
- Python: 3.9
- Transformers: 4.27.1
- PyTorch:2.0.0
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) : True
```

### Anything else?

_No response_


[BUG/Help] ImportError: cannot import name 'GENERATION_CONFIG_NAME' from 'transformers.utils'

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

>>> model = AutoModel.from_pretrained("THUDM/chatglm-6b-int4",trust_remote_code=True).float()
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\mina_\Anaconda3\envs\ChatGLM-6B\lib\site-packages\transformers\models\auto\auto_factory.py", line 456, in from_pretrained
    logger.warning(
  File "C:\Users\mina_\Anaconda3\envs\ChatGLM-6B\lib\site-packages\transformers\dynamic_module_utils.py", line 374, in get_class_from_dynamic_module

  File "C:\Users\mina_\Anaconda3\envs\ChatGLM-6B\lib\site-packages\transformers\dynamic_module_utils.py", line 147, in get_class_in_module
    def get_class_in_module(class_name, module_path):
  File "C:\Users\mina_\Anaconda3\envs\ChatGLM-6B\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "C:\Users\mina_/.cache\huggingface\modules\transformers_modules\THUDM\chatglm-6b-int4\dac03c3ac833dab2845a569a9b7f6ac4e8c5dc9b\modeling_chatglm.py", line 30, in <module>
    from transformers.generation.utils import LogitsProcessorList, StoppingCriteriaList, GenerationConfig
  File "C:\Users\mina_\Anaconda3\envs\ChatGLM-6B\lib\site-packages\transformers\generation\utils.py", line 39, in <module>
    from .configuration_utils import GenerationConfig
  File "C:\Users\mina_\Anaconda3\envs\ChatGLM-6B\lib\site-packages\transformers\generation\configuration_utils.py", line 24, in <module>
    from ..utils import (
ImportError: cannot import name 'GENERATION_CONFIG_NAME' from 'transformers.utils' (C:\Users\mina_\Anaconda3\envs\ChatGLM-6B\lib\site-packages\transformers\utils\__init__.py)

### Expected Behavior

_No response_

### Steps To Reproduce

1. `conda activate chatglm-6b`
2. `from transformers import AutoTokenizer, AutoModel`
3. `tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)`
4. `model = AutoModel.from_pretrained("THUDM/chatglm-6b-int4",trust_remote_code=True).float()`
5. See this issue.

### Environment

```markdown
- OS: Windows 10
- Python: 3.7.5
- Transformers:
- PyTorch:
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) : False
```

### Anything else?

_No response_


[BUG/Help] <title>KeyError: 'lm_head.weight'

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

报错：KeyError: 'lm_head.weight'

### Expected Behavior

Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.

Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "C:\Users\Administrator\Downloads\ChatGLM-6B-main\cli_demo.py", line 7, in <module>
    model = AutoModel.from_pretrained(r"C:\Users\Administrator\Downloads\ChatGLM-6B-main\model",trust_remote_code=True,ignore_mismatched_sizes=True).half().quantize(4).cuda()
  File "C:\Program Files\Python310\lib\site-packages\transformers\models\auto\auto_factory.py", line 466, in from_pretrained
    return model_class.from_pretrained(
  File "C:\Program Files\Python310\lib\site-packages\transformers\modeling_utils.py", line 2646, in from_pretrained
    ) = cls._load_pretrained_model(
  File "C:\Program Files\Python310\lib\site-packages\transformers\modeling_utils.py", line 2959, in _load_pretrained_model
    mismatched_keys += _find_mismatched_keys(
  File "C:\Program Files\Python310\lib\site-packages\transformers\modeling_utils.py", line 2882, in _find_mismatched_keys
    and state_dict[checkpoint_key].shape != model_state_dict[model_key].shape
KeyError: 'lm_head.weight'

### Steps To Reproduce

报错：KeyError: 'lm_head.weight'

### Environment

```markdown
- OS:windows 10
- Python:3.10
- Transformers:4.27.1
- PyTorch:cu118
- CUDA Support True
```

### Anything else?

_No response_


[BUG/Help] ptuning时，指定PRE_SEQ_LEN=512，训练后，回答的问题仍旧有回答一百字左右就断了，该如何调整？

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

训练参数如下：
PRE_SEQ_LEN=512
LR=2e-2

CUDA_VISIBLE_DEVICES=0 python3 main.py \
    --do_train \
    --train_file ./data/gwddc.json \
    --validation_file ./data/gwddc_test.json \
    --prompt_column instruction \
    --response_column output \
    --overwrite_cache \
    --model_name_or_path THUDM/chatglm-6b \
    --output_dir output/adgen-chatglm-6b-pt-gwddc-v3 \
    --overwrite_output_dir \
    --max_source_length 64 \
    --max_target_length 64 \
    --per_device_train_batch_size 4 \
    --per_device_eval_batch_size 1 \
    --gradient_accumulation_steps 16 \
    --predict_with_generate \
    --max_steps 3000 \
    --logging_steps 10 \
    --save_steps 1000 \
    --learning_rate $LR \
    --pre_seq_len $PRE_SEQ_LEN

训练成功，加载checkpoint模型也成功，输入prompts也能正常回答，可是，回答长度仍旧很短，还会出现回答半截断掉的情况，请问该如何调整训练参数？

### Expected Behavior

_No response_

### Steps To Reproduce

1. ./data/gwddc.json为自备的训练集，prompts只有不到2000条
2. 输入上述参数并运行，训练结果信息如下：
……
{'loss': 0.0371, 'learning_rate': 0.0, 'epoch': 96.77}
Saving PrefixEncoder
{'train_runtime': 21212.1807, 'train_samples_per_second': 9.051, 'train_steps_per_second': 0.141, 'train_loss': 0.2381483610868454, 'epoch': 96.77}
***** train metrics *****
  epoch                    =      96.77
  train_loss               =     0.2381
  train_runtime            = 5:53:32.18
  train_samples            =       1982
  train_samples_per_second =      9.051
  train_steps_per_second   =      0.141
帮看是不是train_loss不行？需要增加迭代次数？

### Environment

```markdown
- OS:centos 7.6
- Python:3.9
- Transformers:4.27.1
- PyTorch:2.0.0+cu117
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :True
```

### Anything else?

另外，特别训练了“你是谁”，部署后，也没生效。

{'base_commit': '8633db1503fc3b0edc1d035f64aa35dce5d97969', 'files': [{'path': 'ptuning/README.md', 'Loc': {'(None, None, 180)': {'mod': [180]}}, 'status': 'modified'}, {'path': 'ptuning/arguments.py', 'Loc': {"('DataTrainingArguments', None, 65)": {'mod': [123]}}, 'status': 'modified'}]}


[Help]  如何支持多显卡

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

公司内部使用，装了2卡，发现默认配置只有1卡在跑，请问如何使用才可以使用多卡

### Expected Behavior

_No response_

### Steps To Reproduce

无

### Environment

```markdown
OS: Ubuntu 20.04
Python: 3.8
Transformers: 4.26.1
PyTorch: 1.12
CUDA Support: True
```

### Anything else?

_No response_


Bert output last hidden state

## ❓ Questions & Help

Hi,

Suppose we have an utterance of length 24 (considering special tokens) and we right-pad it with 0 to max length of 64.
If we use Bert pertained model to get the last hidden states, the output would be of size [1, 64, 768].
Can we use just the first 24 as the hidden states of the utterance? I mean is it right to say that the output[0, :24, :] has all the required information?
I realized that from index 24:64, the outputs has float values as well.

{'base_commit': '34f28b2a1342fd72c2e4d4e5613855bfb9f35d34', 'files': [{'path': 'src/transformers/models/bert/modeling_bert.py', 'Loc': {"('BertSelfAttention', 'forward', 276)": {'mod': [279]}}, 'status': 'modified'}]}


dataset download error in speech recognition examples

### System Info

- `transformers` version: 4.35.0.dev0
- Platform: Linux-5.15.0-43-generic-x86_64-with-glibc2.17
- Python version: 3.8.18
- Huggingface_hub version: 0.17.3
- Safetensors version: 0.4.0
- Accelerate version: 0.24.1
- Accelerate config:    not found
- PyTorch version (GPU?): 1.10.0+cu111 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@stevhliu and @MKhalusova

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

CUDA_VISIBLE_DEVICES=0 python run_speech_recognition_ctc.py \
	--dataset_name="common_voice" \
	--model_name_or_path="facebook/wav2vec2-large-xlsr-53" \
	--dataset_config_name="tr" \
	--output_dir="./wav2vec2-common_voice-tr-demo" \
	--overwrite_output_dir \
	--num_train_epochs="15" \
	--per_device_train_batch_size="16" \
	--gradient_accumulation_steps="2" \
	--learning_rate="3e-4" \
	--warmup_steps="500" \
	--evaluation_strategy="steps" \
	--text_column_name="sentence" \
	--length_column_name="input_length" \
	--save_steps="400" \
	--eval_steps="100" \
	--layerdrop="0.0" \
	--save_total_limit="3" \
	--freeze_feature_encoder \
	--gradient_checkpointing \
	--chars_to_ignore , ? . ! - \; \: \" “ % ‘ ” � \
	--fp16 \
	--group_by_length \
	--push_to_hub \
	--do_train --do_eval

### Expected behavior

When I run the default command, which set `dataset_name` as "common_voice", and I got a warning:
```
/home/xintong/.cache/huggingface/modules/datasets_modules/datasets/common_voice/220833898d6a60c50f621126e51fb22eb2dfe5244392c70dccd8e6e2f055f4bf/common_voice.py:634: FutureWarning:
            This version of the Common Voice dataset is deprecated.
            You can download the latest one with
            >>> load_dataset("mozilla-foundation/common_voice_11_0", "en")

  warnings.warn(
Generating train split:   0%|                                                                                | 0/1831 [00:00<?, ? examples/s]
Traceback (most recent call last):
  File "/home/xintong/miniconda3/envs/test/lib/python3.8/tarfile.py", line 2578, in next
    tarinfo = self.tarinfo.fromtarfile(self)
  File "/home/xintong/miniconda3/envs/test/lib/python3.8/tarfile.py", line 1283, in fromtarfile
    obj = cls.frombuf(buf, tarfile.encoding, tarfile.errors)
  File "/home/xintong/miniconda3/envs/test/lib/python3.8/tarfile.py", line 1221, in frombuf
    raise TruncatedHeaderError("truncated header")
tarfile.TruncatedHeaderError: truncated header
```
I modified this into `mozilla-foundation/common_voice_11_0`, it passed.
```
Downloading builder script: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8.13k/8.13k [00:00<00:00, 30.3MB/s]
Downloading readme: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 14.4k/14.4k [00:00<00:00, 19.2MB/s]
Downloading extra modules: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3.44k/3.44k [00:00<00:00, 19.9MB/s]
Downloading extra modules: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 60.9k/60.9k [00:00<00:00, 304kB/s]
Downloading data: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12.2k/12.2k [00:00<00:00, 25.6MB/s]
Downloading data: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 568M/568M [00:07<00:00, 71.7MB/s]
Downloading data: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 233M/233M [00:02<00:00, 78.6MB/s]
Downloading data: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 285M/285M [00:04<00:00, 67.7MB/s]
Downloading data: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4.86M/4.86M [00:00<00:00, 73.3MB/s]
Downloading data: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 109M/109M [00:01<00:00, 80.4MB/s]
Downloading data files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:21<00:00,  4.24s/it]
Extracting data files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:07<00:00,  1.54s/it]
Downloading data: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5.76M/5.76M [00:00<00:00, 56.0MB/s]
Downloading data: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.17M/2.17M [00:00<00:00, 54.1MB/s]
Downloading data: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.18M/2.18M [00:00<00:00, 64.3MB/s]
Downloading data: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32.8k/32.8k [00:00<00:00, 53.1MB/s]
Downloading data: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 800k/800k [00:00<00:00, 59.8MB/s]
Downloading data files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:05<00:00,  1.01s/it]
Extracting data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 2954.98it/s]
```

{'base_commit': '82c7e879876822864b5ceaf2c99eb01159266bcd', 'files': [{'path': 'examples/pytorch/speech-recognition/README.md', 'Loc': {'(None, None, 69)': {'mod': [69]}}, 'status': 'modified'}]}


GPT2 Flax "TypeError: JAX only supports number and bool dtypes, got dtype object in array"

On GPU

```
>>> from transformers import AutoTokenizer, FlaxAutoModelForCausalLM

>>> tokenizer = AutoTokenizer.from_pretrained("gpt2-medium")
>>> model = FlaxAutoModelForCausalLM.from_pretrained("gpt2-medium")
>>> input_context = "The dog"
>>> # encode input context
>>> input_ids = tokenizer(input_context, return_tensors="jax").input_ids
>>> # generate candidates using sampling
>>> outputs = model.generate(input_ids=input_ids, max_length=20, top_k=30, do_sample=True)

TypeError: JAX only supports number and bool dtypes, got dtype object in array
```

@patrickvonplaten @patil-suraj

{'base_commit': '0e82f0cbc28b41b3d87a5e4069dc0e20bacc2494', 'files': [{'path': 'src/transformers/models/gpt2/modeling_flax_gpt2.py', 'Loc': {"('FlaxGPT2LMHeadModule', None, 553)": {'mod': []}}, 'status': 'modified'}, {'path': 'src/transformers/models/gpt2/tokenization_gpt2_fast.py', 'Loc': {"('GPT2TokenizerFast', None, 70)": {'mod': []}}, 'status': 'modified'}, {'Loc': [6, 7], 'path': None}]}


Unclear error "NotImplementedError:  "while saving tokenizer. How fix it?

Here is my tokenizer code and how I save it to a json file" /content/bert-datas7.json"

````
from tokenizers import normalizers
from tokenizers.normalizers import Lowercase, NFD, StripAccents

bert_tokenizer.pre_tokenizer = Whitespace()

from tokenizers.processors import TemplateProcessing

bert_tokenizer.post_processor = TemplateProcessing(
    single="[CLS] $A [SEP]",
    pair="[CLS] $A [SEP] $B:1 [SEP]:1",
    special_tokens=[
        ("[CLS]", 1),
        ("[SEP]", 2),
        ("[PAD]", 3),
    ],

)
from tokenizers.trainers import WordPieceTrainer

trainer = WordPieceTrainer(
    vocab_size=30522, special_tokens=["[UNK]", "[CLS]", "[SEP]", "[PAD]", "[MASK]"], pad_to_max_length=True
)
files = [f"/content/For_ITMO.txt" for split in ["test", "train", "valid"]]
bert_tokenizer.train(trainer, files)

model_files = bert_tokenizer.model.save("data", "/content/For_ITMO.txt")

bert_tokenizer.model = WordPiece.from_file(*model_files, unk_token="[UNK]",  pad_to_max_length=True)

bert_tokenizer.save("/content/bert-datas7.json")
````

When I output tokenizer name_or_path = nothing is displayed. This is normal?

````
tokenizer = PreTrainedTokenizerFast(tokenizer_file='/content/bert-datas7.json')
tokenizer.add_special_tokens({'pad_token': '[PAD]'})

print(tokenizer)
>>> PreTrainedTokenizerFast(name_or_path='', vocab_size=1435, model_max_len=1000000000000000019884624838656, is_fast=True, padding_side='right', special_tokens={'pad_token': '[PAD]'})
````
Also, when I try to save my tokenizer, I get an error without explanation. How can I rewrite the code so that all this???
#9658
#10039
[For_ITMO.txt-vocab (1) (1).txt](https://github.com/huggingface/transformers/files/5945659/For_ITMO.txt-vocab.1.1.txt)

````
tokenizer.save_pretrained("/content/tokennizerrrr")

NotImplementedError                       Traceback (most recent call last)
<ipython-input-11-efc48254a528> in <module>()
----> 1 tokenizer.save_pretrained("/content/tokennizerrrr")

2 frames
/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils_base.py in save_vocabulary(self, save_directory, filename_prefix)
   2042             :obj:`Tuple(str)`: Paths to the files saved.
   2043         """
-> 2044         raise NotImplementedError
   2045
   2046     def tokenize(self, text: str, pair: Optional[str] = None, add_special_tokens: bool = False, **kwargs) -> List[str]:

NotImplementedError:
````

{'base_commit': '322037e842e5e89080918c824998c17722df6f19', 'files': [{'path': 'src/transformers/tokenization_utils_fast.py', 'Loc': {"('PreTrainedTokenizerFast', '_save_pretrained', 505)": {'mod': [509]}}, 'status': 'modified'}]}


[s2s finetune] huge increase in memory demands with --fp16 native amp

While working on https://github.com/huggingface/transformers/issues/8353 I discovered that `--fp16` causes a 10x+ increase in gpu memory demands.

e.g. I can run bs=12 w/o  `--fp16`

```
cd examples/seq2seq
export BS=12; rm -rf distilbart-cnn-12-6; python finetune.py --learning_rate=3e-5 --gpus 1 \
--do_train --do_predict --val_check_interval 0.25 --n_val 500 --num_train_epochs 2 --freeze_encoder \
--freeze_embeds --data_dir cnn_dm --max_target_length 142 --val_max_target_length=142 \
--train_batch_size=$BS --eval_batch_size=$BS --gradient_accumulation_steps 1 \
--model_name_or_path sshleifer/student_cnn_12_6 --tokenizer_name facebook/bart-large \
--warmup_steps 500 --output_dir distilbart-cnn-12-6

```
But if I add:
```
--fp16
```

(w/ or w/o `--fp16_opt_level O1`)

I get OOM even with bs=1 on a 8GB card and it barely manages on a 24GB card - I think the increase in memory demand is more than 10x.

The OOM either right away when it does the sanity check step, or after just 10-20 batches - so within a few secs

This is with pytorch-1.6. Same goes for pytorch-1.7 and 1.8-nightly.

I wasn't able to test `--fp16` with pytorch-1.5, since I can't build apex on ubuntu-20.04. Without `--fp16` pytorch-1.5 works the same as pytorch-1.6 gpu memory-wise.

I tested with pytorch-1.5 + apex and there is no problem there. Memory consumption is about half.

Here is the table of the batch sizes that fit into a 8gb rtx-1070 (bigger BS leads to an instant OOM):

bs | version
---|--------
12 | pt15
20 | pt15+fp16
12 | pt16
1 | pt16+fp16

If you'd like to reproduce the problem here are the full steps:

```
# prep library
git clone https://github.com/huggingface/transformers
cd transformers
pip install -e .[dev]
pip install -r examples/requirements.txt
cd examples/seq2seq

# prep data
wget https://cdn-datasets.huggingface.co/summarization/cnn_dm_v2.tgz
tar -xzvf cnn_dm_v2.tgz  # empty lines removed
mv cnn_cln cnn_dm

# run
export BS=12;
rm -rf distilbart-cnn-12-6
python finetune.py --learning_rate=3e-5 --gpus 1 \
--do_train --do_predict --val_check_interval 0.25 --n_val 500 --num_train_epochs 2 --freeze_encoder \
--freeze_embeds --data_dir cnn_dm --max_target_length 142 --val_max_target_length=142 \
--train_batch_size=$BS --eval_batch_size=$BS --gradient_accumulation_steps 1 \
--model_name_or_path sshleifer/student_cnn_12_6 --tokenizer_name facebook/bart-large \
--warmup_steps 500 --output_dir distilbart-cnn-12-6
```

This issue is to track the problem and hopefully finding a solution.

@sshleifer

{'base_commit': '57bffc3a8e4fee0cce31e1ff1f662ccf7b16db57', 'files': [{'path': 'aten/src/ATen/autocast_mode.cpp', 'status': 'modified', 'Loc': {"(None, 'cached_cast', 67)": {'mod': [71]}}}, {'path': 'test/test_cuda.py', 'status': 'modified', 'Loc': {"('TestCuda', None, 92)": {'add': [2708]}}}]}


a memory leak in qqp prediction using bart

### System Info

```shell
- `transformers` version: 4.19.0.dev0
- Platform: Linux-5.11.0-43-generic-x86_64-with-glibc2.17
- Python version: 3.8.10
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.10.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No
```

### Who can help?

@sgugger

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I met the same issue #11011. If not using `--eval_accumulation_steps`, it caused CUDA out of memory. If using it, it caused out of RAM and killed by system.

I only did prediction on GLUE QQP dataset using bart without fine-tuning. Considering QQP having a large test set (300k), the prediction got slower and slower, and finally got out of memory.

This is the script to reproduce:
```
CUDA_VISIBLE_DEVICES=0 python run_glue.py   --model_name_or_path facebook/bart-large   --task_name qqp   --output_dir bart-large_qqp --eval_accumulation_steps 100 --do_predict --per_device_eval_batch_size 24
```

### Expected behavior

```shell
Prediction without out memory.
```

{'base_commit': '1a688709b34b10bd372e3e0860c8d39d170ebf53', 'files': [{'path': 'src/transformers/trainer.py', 'Loc': {"('Trainer', 'evaluation_loop', 2549)": {'mod': [2635]}}, 'status': 'modified'}]}


Skip some weights for load_in_8bit and keep them as fp16/32?

### Feature request

Hello,

I am looking for a way to load a checkpoint where I only load some of the weights in 8 bit and keep others in 16/32 bit.

### Motivation

My motivation is for vision-language models like Llava or BLIP2 where I want to load the LLM part in 8 bit but the image encoder should stay in 16 bit because I notice performance degradations with CLIP in 8 bit and also want to be able to train this part without LoRA.

As far as I can see in the documentation, issues and with Google (both here and for bitsandbytes), there is currently no way to do this.

### Your contribution

I can in theory help implement something like this but I don't know where and how in the code this should be done.

{'base_commit': 'cef2e40e0f8eaad13b8d32817a48fdddc32eb2a5', 'files': [{'path': 'src/transformers/modeling_utils.py', 'Loc': {"('PreTrainedModel', 'from_pretrained', 2528)": {'mod': [3524]}}, 'status': 'modified'}, {'path': 'src/transformers/utils/quantization_config.py', 'Loc': {"('BitsAndBytesConfig', None, 151)": {'mod': [176]}}, 'status': 'modified'}]}


Question: Object of type EncoderDecoderConfig is not JSON serializable

Hi.
An error occurred when I used Trainer to train and save EncoderDecoderModel.

```python
  File "/home/jwli/ljw/study/hotpotqa/roberta_seq2seq/roberta_for_seq2seq.py", line 482, in <module>
    run(model_args, data_args, training_args)
  File "/home/jwli/ljw/study/hotpotqa/roberta_seq2seq/roberta_for_seq2seq.py", line 465, in run
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/site-packages/transformers/trainer.py", line 1391, in train
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch, ignore_keys_for_eval)
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/site-packages/transformers/trainer.py", line 1495, in _maybe_log_save_evaluate
    self._save_checkpoint(model, trial, metrics=metrics)
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/site-packages/transformers/trainer.py", line 1557, in _save_checkpoint
    self.save_model(output_dir)
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/site-packages/transformers/trainer.py", line 1961, in save_model
    self._save(output_dir)
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/site-packages/transformers/trainer.py", line 2009, in _save
    self.model.save_pretrained(output_dir, state_dict=state_dict)
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/site-packages/transformers/modeling_utils.py", line 1053, in save_pretrained
    model_to_save.config.save_pretrained(save_directory)
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/site-packages/transformers/configuration_utils.py", line 416, in save_pretrained
    self.to_json_file(output_config_file, use_diff=True)
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/site-packages/transformers/configuration_utils.py", line 739, in to_json_file
    writer.write(self.to_json_string(use_diff=use_diff))
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/site-packages/transformers/configuration_utils.py", line 725, in to_json_string
    return json.dumps(config_dict, indent=2, sort_keys=True) + "\n"
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/json/encoder.py", line 201, in encode
    chunks = list(chunks)
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/json/encoder.py", line 431, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/json/encoder.py", line 438, in _iterencode
    o = _default(o)
  File "/home/jwli/anaconda3/envs/study/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type EncoderDecoderConfig is not JSON serializable
```
My model and Config define the following code.
```python
    tokenizer = RobertaTokenizerFast.from_pretrained(model_args.tokenizer_name)
    encoder_config = RobertaConfig.from_pretrained(model_args.encoder_model_name_or_path)
    decoder_config = RobertaConfig.from_pretrained(model_args.decoder_model_name_or_path)
    encoder_decoder_config = EncoderDecoderConfig.from_encoder_decoder_configs(encoder_config, decoder_config)
    model = RobertaForSeq2Seq.from_encoder_decoder_pretrained(model_args.encoder_model_name_or_path,
                                                              model_args.decoder_model_name_or_path,
                                                              config=encoder_decoder_config, tie_encoder_decoder=True)
    model.config.decoder_start_token_id = tokenizer.bos_token_id
    model.config.eos_token_id = tokenizer.eos_token_id
    model.config.max_length = 64
    model.config.early_stopping = True
    model.config.no_repeat_ngram_size = 3
    model.config.length_penalty = 2.0
    model.config.num_beams = 4
    model.config.pad_token_id = tokenizer.pad_token_id
```
This error occurred because EncoderDecoderConfig cannot be converted to json format. But I don't know how to modify it.
```python
ERROR OCCURRED:

        if use_diff is True:
            config_dict = self.to_diff_dict()
        else:
            config_dict = self.to_dict()
        return json.dumps(config_dict, indent=2, sort_keys=True) + "\n"
```

I look forward to your help! Thanks!
 @jplu  @patrickvonplaten


Different Results from version 0.4.0 to version 0.5.0

Hi, I found the results after training is different from version 0.4.0 to version 0.5.0. I have fixed all initialization to reproduce the results. And I also test version 0.2.0 and 0.3.0, the results are the same to version 0.4.0, but from version 0.5.0 +, the results is different. I am wondering that have you trained a new model, so the weights changed?

{'base_commit': '45d21502f0b67eb8a5ad244d469dcc0dfb7517a7', 'files': [{'path': 'pytorch_pretrained_bert/modeling.py', 'Loc': {"('BertPreTrainedModel', 'init_bert_weights', 515)": {'mod': []}}, 'status': 'modified'}]}


Fast Tokenizers instantiated via vocab/merge files do not respect skip_special_tokens=True

## Environment info
- `transformers` version: 4.3.2
- Platform: macOS-11.2.1-x86_64-i386-64bit
- Python version: 3.9.1
- PyTorch version (GPU?): 1.7.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

## Information

See title; this issue does not reproduce with slow tokenizers. Does not reproduce with serialized tokenizers.

Found while investigating https://github.com/minimaxir/aitextgen/issues/88

## To reproduce

Using [gpt2_merges.txt](https://github.com/minimaxir/aitextgen/blob/master/aitextgen/static/gpt2_merges.txt) and [gpt2_vocab.json](https://github.com/minimaxir/aitextgen/blob/master/aitextgen/static/gpt2_vocab.json) as linked:

```py
from transformers import AutoModelForCausalLM, GPT2Tokenizer, GPT2TokenizerFast

model = AutoModelForCausalLM.from_pretrained("distilgpt2")

outputs = model.generate(max_length=40)

# tensor([[50256,   383,   471,    13,    50,    13,  2732,   286,  4796,   468,
#           587, 10240,   262,  1918,   286,   257,  1966,  5349,  5797,   508,
#           373,  2823,   290,  2923,   416,   257, 23128,   287,   262,   471,
#            13,    50,    13, 13241,   319,  3583,    13,   198,   198,   198]])

tokenizer_fast = GPT2TokenizerFast(vocab_file="gpt2_vocab.json", merges_file="gpt2_merges.txt")
tokenizer_fast.decode(outputs[0], skip_special_tokens=True)

# '<|endoftext|> The U.S. Department of Justice has been investigating the death of a former FBI agent who was shot and killed by a gunman in the U.S. Capitol on Wednesday.\n\n\n'

tokenizer_slow = GPT2Tokenizer(vocab_file="gpt2_vocab.json", merges_file="gpt2_merges.txt")
tokenizer_slow.decode(outputs[0], skip_special_tokens=True)

# ' The U.S. Department of Justice has been investigating the death of a former FBI agent who was shot and killed by a gunman in the U.S. Capitol on Wednesday.\n\n\n'

```

{'base_commit': '1c8c2d9ab34b8c8d326db9e0608f8e54cfccb885', 'files': [{'path': 'src/transformers/tokenization_utils_base.py', 'Loc': {"('SpecialTokensMixin', 'add_special_tokens', 900)": {'mod': []}}, 'status': 'modified'}, {'Loc': [33], 'path': None}]}

Cment指出用户代码问题，给出需要使用的API
自己代码的问题 另一个issue中指出cmit
I think this is happening because when you load it from the vocab and merge files, it doesn't know <|endoftext|> is a special token. For the skip_special_tokens to work, I believe it would be necessary to add them to the tokenizer:
tokenizer_fast.add_special_tokens({
    "additional_special_tokens": "<|endoftext|>"
})


RoBERTa config defaults are inconsistent with fairseq implementation

### System Info

 python 3.12, transformers 4.14, latest mac os

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import RobertaConfig
my_config = RobertaConfig()
roberta_config = RobertaConfig.from_pretrained("roberta-base")

assert (
  my_config.max_position_embeddings == roberta_config.max_position_embeddings
), "%d %d" % (my_config.max_position_embeddings, roberta_config.max_position_embeddings)

### Expected behavior

The config defaults should correspond the the base model?

This is an implementation detail, but it did send me on a debugging spree as it hid as a sticky CUDA assertion error.
```Assertion `srcIndex < srcSelectDimSize` failed```

The problem is that by default if you create the position_ids yourself or if you let transformers roberta_modelling take care of it (it also does it the way fairseq implemented it), it will create indeces that are out of bounds with the default configuration as everything is shifted by pad_token_id.

This is more of a heads up. Do transformers generally provide defaults aligned with the original models, or are the defaults here meant to be agnostic of that?

{'base_commit': '5bcbdff15922b1d0eeb035879630ca61c292122a', 'files': [{'path': 'src/transformers/models/roberta/configuration_roberta.py', 'Loc': {"('RobertaConfig', None, 29)": {'mod': [59]}}, 'status': 'modified'}]}


Not able to run any MetaGPT examples

Referred Issue #1322 , but not able to resolve the issue. I added azure based api endpoint and api key in config2.yaml

│   105 │   │   typer.echo("Missing argument 'IDEA'. Run 'metagpt --help' for more information."   │
│   106 │   │   raise typer.Exit()                                                                 │
│   107 │                                                                                │
│ ❱ 108 │   return generate_repo(                                                                  │
│   109 │   │   idea,                                                                              │
│   110 │   │   investment,                                                                        │
│   111 │   │   n_round,                                                                           │
│                                                                                │
\metagpt\software_company.py:30 in generate_repo          │
│                                                                                │
│    27 │   recover_path=None,                                                                     │
│    28 ) -> ProjectRepo:                                                                          │
│    29 │   """Run the startup logic. Can be called from CLI or other Python scripts."""           │
│ ❱  30 │   from metagpt.config2 import config                                                     │
│    31 │   from metagpt.context import Context                                                    │
│    32 │   from metagpt.roles import (                                                            │
│    33 │   │   Architect,                                                                         │
│                                                                                │
\new_meta_env\Lib\site-packages\metagpt-0.8.1-py3.11.egg\metagpt\ │
│ config2.py:164 in <module>                                                                       │
│                                                                                │
│   161 │   return result                                                                          │
│   162                                                                                │
│   163                                                                                │
│ ❱ 164 config = Config.default()                                                                  │
\new_meta_env\Lib\site-packages\metagpt-0.8.1-py3.11.egg\metagpt\ │
│ config2.py:106 in default                                                                        │
│                                                                                │
│   103 │   │   dicts = [dict(os.environ)]                                                         │
│   104 │   │   dicts += [Config.read_yaml(path) for path in default_config_paths]                 │
│   105 │   │   final = merge_dict(dicts)                                                          │
│ ❱ 106 │   │   return Config(**final)                                                             │
│   107 │                                                                                │
│   108 │   @classmethod                                                                           │
│   109 │   def from_llm_config(cls, llm_config: dict):                                            │
│                                                                                │
\new_meta_env\Lib\site-packages\pydantic\main.py:176 in __init__  │
│                                                                                │
│    173 │   │   """                                                                               │
│    174 │   │   # `__tracebackhide__` tells pytest and some other tools to omit this function fr  │
│    175 │   │   __tracebackhide__ = True                                                          │
│ ❱  176 │   │   self.__pydantic_validator__.validate_python(data, self_instance=self)             │
│    177 │                                                                                │
│    178 │   # The following line sets a flag that we use to determine when `__init__` gets overr  │
│    179 │   __init__.__pydantic_base_init__ = True  # pyright: ignore[reportFunctionMemberAccess  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValidationError: 1 validation error for Config
llm
  Field required [type=missing, input_value={'ALLUSERSPROFILE': 'C:\\..._INIT_AT_FORK': 'FALSE'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.7/v/missing

{'base_commit': 'f0df3144d68ed288f5ccce0c34d3939f8462ba98', 'files': [{'path': 'config/config2.yaml', 'Loc': {}}]}


请问metagpt是否支持向量数据，构建自己的知识库

请问metagpt是否支持向量数据，构建自己的知识库


Rate Limited error

openai.error.RateLimitError: Rate limit reached for 10KTPM-200RPM in organization org-fK5bb25UFhVbebfBtfCejGc4 on tokens per min. Limit: 10000 / min. Please try again in 6ms. Contact us through our help center at help.openai.com if you continue to have issues.

Maybe a way to resume so all the runtime isn't just lost?

{'base_commit': 'be56351e000a0f08562820fb04f6fdbe34d9e655', 'files': [{'path': 'metagpt/provider/openai_api.py', 'Loc': {"('OpenAIGPTAPI', '_achat_completion_stream', 150)": {'mod': []}}, 'status': 'modified'}]}


**Bug description**
<!-- Clearly and directly describe the current bug -->
I using ***MetaGPT ver 0.8.1*** but when use RAG with method **SimpleEngine.from_docs** have error ***ValueError: Creator not registered for key: LLMType.OLLAMA***

<!--  **Bug solved method**  -->
<!-- If you solved the bug, describe the idea or process to solve the current bug. Of course, you can also paste the URL address of your Pull Request. -->
<!-- If not, provide more auxiliary information to facilitate our further positioning and investigation  -->

**Environment information**
<!-- Environment：System version (like ubuntu 22.04), Python version (conda python 3.7), LLM type and model (OpenAI gpt-4-1106-preview) -->

- LLM type and model name: ollama and model: hf.co/hugging-quants/Llama-3.2-3B-Instruct-Q8_0-GGUF
- System version:
- Python version: 3.10
- MetaGPT version or branch: 0.8.1

<!-- Dependent packagess：the packages version cause the bug(like `pydantic 1.10.8`), installation method（like `pip install metagpt` or `pip install from source` or `run in docker`） -->

- packages version:
- installation method:

**Screenshots or logs**
<!-- Screenshots or logs of the bug can help us understand the problem more quickly -->
***config2.yaml***
embedding:
  api_type: "ollama"
  model: "hf.co/hugging-quants/Llama-3.2-3B-Instruct-Q8_0-GGUF"
  base_url: "http://127.0.0.1:11434/api"

llm:
  api_type: "ollama"
  model: "hf.co/hugging-quants/Llama-3.2-3B-Instruct-Q8_0-GGUF"
  base_url: "http://127.0.0.1:11434/api"

***Error Response***
[/usr/local/lib/python3.10/dist-packages/metagpt/rag/factories/base.py](https://localhost:8080/#) in get_instance(self, key, **kwargs)
     27             return creator(**kwargs)
     28
---> 29         raise ValueError(f"Creator not registered for key: {key}")
     30
     31

ValueError: Creator not registered for key: LLMType.OLLAMA


请问如何配置使用Azure上的api？

你好，
我看到文档中需要配置openAI的key，但是我注意到在provider中有azure_api的相关文件,
请问是否在哪个地方可以配置让他使用azure提供的服务？

{'base_commit': 'df8d1124c68b62bb98c71b6071abf5efe6293dba', 'files': [{'path': 'config/config.yaml', 'Loc': {}}]}


GLM4一直报错

2024-02-22 16:50:26.666 | ERROR    | metagpt.utils.common:log_it:476 - Finished call to 'metagpt.actions.action_node.ActionNode._aask_v1' after 80.109(s), this was the 5th time calling it. exp: 1 validation error for PM_NODE_AN
  Value error, Missing fields: {'Full API spec', 'Required Python packages', 'Required Other language third-party packages'} [type=value_error, input_value={'Required JavaScript pac...ation and development.'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/value_error

{'base_commit': 'dfa33fcdaade1e4f8019835bf065d372d76724ae', 'files': [{'path': 'config/config2.yaml', 'Loc': {}}]}


failed to launch chromium browser process errors

get errors on launch of browser process; below is the error from terminal which happens for all browser processes trying to launch.

```
INFO     | metagpt.utils.mermaid:mermaid_to_file:38 - Generating /Users/lopezdp/DevOps/Ai_MetaGPT/workspace/test_app/resources/competitive_analysis.pdf..

Error: Failed to launch the browser process! spawn /usr/bin/chromium ENOENT

TROUBLESHOOTING: https://pptr.dev/troubleshooting

    at ChildProcess.onClose (file:///Users/lopezdp/DevOps/Ai_MetaGPT/node_modules/@puppeteer/browsers/lib/esm/launch.js:253:24)
    at ChildProcess.emit (node:events:513:28)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
    at onErrorNT (node:internal/child_process:485:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)
```

{'base_commit': '80a189ad4a1546f8c1a9dbe00c42725868c35e5e', 'files': [{'path': 'config/puppeteer-config.json', 'Loc': {}}]}


The following error appears on every run

![image](https://github.com/geekan/MetaGPT/assets/115678682/1fb58e0b-47a7-4e1f-a7b7-924ea9adedb0)

2024-03-27 11:15:59.019 | ERROR    | metagpt.utils.common:wrapper:631 - Exception occurs, start to serialize the project, exp:
Traceback (most recent call last):
  File "D:\andconda\envs\metagpt\lib\site-packages\tenacity\__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
  File "d:\下载\metagpt-main\metagpt\utils\repair_llm_raw_output.py", line 296, in retry_parse_json_text
    parsed_data = CustomDecoder(strict=False).decode(output)
json.decoder.JSONDecodeError: Unterminated string starting at: line 13 column 25 (char 3485)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\andconda\envs\metagpt\lib\site-packages\tenacity\_asyncio.py", line 50, in __call__
    result = await fn(*args, **kwargs)
  File "d:\下载\metagpt-main\metagpt\actions\action_node.py", line 425, in _aask_v1
    parsed_data = llm_output_postprocess(
tenacity.RetryError: RetryError[<Future at 0x1f1a7f31d30 state=finished raised JSONDecodeError>]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "d:\下载\metagpt-main\metagpt\utils\common.py", line 640, in wrapper
    return await func(self, *args, **kwargs)
  File "d:\下载\metagpt-main\metagpt\roles\role.py", line 550, in run
    rsp = await self.react()
tenacity.RetryError: RetryError[<Future at 0x1f1a7f31160 state=finished raised RetryError>]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\下载\metagpt-main\metagpt\utils\common.py", line 626, in wrapper
    result = await func(self, *args, **kwargs)
  File "d:\下载\metagpt-main\metagpt\team.py", line 134, in run
    await self.env.run()
Exception: Traceback (most recent call last):
  File "D:\andconda\envs\metagpt\lib\site-packages\tenacity\__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
  File "d:\下载\metagpt-main\metagpt\utils\repair_llm_raw_output.py", line 296, in retry_parse_json_text
    parsed_data = CustomDecoder(strict=False).decode(output)
  File "d:\下载\metagpt-main\metagpt\utils\custom_decoder.py", line 297, in decode
    return super().decode(s)
  File "D:\andconda\envs\metagpt\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "D:\andconda\envs\metagpt\lib\json\decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
  File "d:\下载\metagpt-main\metagpt\utils\custom_decoder.py", line 65, in scan_once
    return _scan_once(string, idx)
  File "d:\下载\metagpt-main\metagpt\utils\custom_decoder.py", line 36, in _scan_once
    return parse_object((string, idx + 1), strict, _scan_once, object_hook, object_pairs_hook, memo)
  File "d:\下载\metagpt-main\metagpt\utils\custom_decoder.py", line 164, in JSONObject
    value, end = scan_once(s, end)
  File "d:\下载\metagpt-main\metagpt\utils\custom_decoder.py", line 34, in _scan_once
    return parse_string(string, idx + 1, strict, delimiter=nextchar)
  File "d:\下载\metagpt-main\metagpt\utils\custom_decoder.py", line 227, in py_scanstring
    raise JSONDecodeError("Unterminated string starting at", s, begin)
json.decoder.JSONDecodeError: Unterminated string starting at: line 13 column 25 (char 3485)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\andconda\envs\metagpt\lib\site-packages\tenacity\_asyncio.py", line 50, in __call__
    result = await fn(*args, **kwargs)
  File "d:\下载\metagpt-main\metagpt\actions\action_node.py", line 425, in _aask_v1
    parsed_data = llm_output_postprocess(
  File "d:\下载\metagpt-main\metagpt\provider\postprocess\llm_output_postprocess.py", line 19, in llm_output_postprocess
    result = postprocess_plugin.run(output=output, schema=schema, req_key=req_key)
  File "d:\下载\metagpt-main\metagpt\provider\postprocess\base_postprocess_plugin.py", line 68, in run
    new_output = self.run_repair_llm_output(output=output, schema=schema, req_key=req_key)
  File "d:\下载\metagpt-main\metagpt\provider\postprocess\base_postprocess_plugin.py", line 32, in run_repair_llm_output
    parsed_data = self.run_retry_parse_json_text(content)
  File "d:\下载\metagpt-main\metagpt\provider\postprocess\base_postprocess_plugin.py", line 47, in run_retry_parse_json_text
    parsed_data = retry_parse_json_text(output=content)  # should use output=content
  File "D:\andconda\envs\metagpt\lib\site-packages\tenacity\__init__.py", line 289, in wrapped_f
    return self(f, *args, **kw)
  File "D:\andconda\envs\metagpt\lib\site-packages\tenacity\__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
  File "D:\andconda\envs\metagpt\lib\site-packages\tenacity\__init__.py", line 326, in iter
    raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x1f1a7f31d30 state=finished raised JSONDecodeError>]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "d:\下载\metagpt-main\metagpt\utils\common.py", line 640, in wrapper
    return await func(self, *args, **kwargs)
  File "d:\下载\metagpt-main\metagpt\roles\role.py", line 550, in run
    rsp = await self.react()
  File "d:\下载\metagpt-main\metagpt\roles\role.py", line 517, in react
    rsp = await self._react()
  File "d:\下载\metagpt-main\metagpt\roles\role.py", line 463, in _react
    rsp = await self._act()
  File "d:\下载\metagpt-main\metagpt\roles\role.py", line 392, in _act
    response = await self.rc.todo.run(self.rc.history)
  File "d:\下载\metagpt-main\metagpt\actions\design_api.py", line 58, in run
    doc = await self._update_system_design(filename=filename)
  File "d:\下载\metagpt-main\metagpt\actions\design_api.py", line 86, in _update_system_design
    system_design = await self._new_system_design(context=prd.content)
  File "d:\下载\metagpt-main\metagpt\actions\design_api.py", line 73, in _new_system_design
    node = await DESIGN_API_NODE.fill(context=context, llm=self.llm)
  File "d:\下载\metagpt-main\metagpt\actions\action_node.py", line 505, in fill
    return await self.simple_fill(schema=schema, mode=mode, images=images, timeout=timeout, exclude=exclude)
  File "d:\下载\metagpt-main\metagpt\actions\action_node.py", line 457, in simple_fill
    content, scontent = await self._aask_v1(
  File "D:\andconda\envs\metagpt\lib\site-packages\tenacity\_asyncio.py", line 88, in async_wrapped
    return await fn(*args, **kwargs)
  File "D:\andconda\envs\metagpt\lib\site-packages\tenacity\_asyncio.py", line 47, in __call__
    do = self.iter(retry_state=retry_state)
  File "D:\andconda\envs\metagpt\lib\site-packages\tenacity\__init__.py", line 326, in iter
    raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x1f1a7f31160 state=finished raised RetryError>]

{'base_commit': '8d98ce34e54eb6250f1f2cf60f5d4dd66d462a5d', 'files': [{'path': 'metagpt/strategy/planner.py', 'Loc': {"('Planner', 'update_plan', 68)": {'mod': [75]}}, 'status': 'modified'}]}


SubscriptionRunner

import asyncio
from metagpt.subscription import SubscriptionRunner
from metagpt.roles import Searcher
from metagpt.schema import Message

async def trigger():
    while True:
        yield Message("the latest news about OpenAI")
        await asyncio.sleep(1)

async def callback(msg: Message):
    print(msg.content)

# async def main():
#     aa = trigger()
#     async for i in aa:
#         await callback(i)
async def main():
    pd = SubscriptionRunner()
    await pd.subscribe(Searcher(), trigger(), callback)
    await pd.run()

asyncio.run(main())
在创建Runner时候报错，0.6.3版本
Traceback (most recent call last):
  File "e:\tmp\metatest\OSSWatcher .py", line 44, in <module>
    asyncio.run(main())
  File "C:\Users\888888\.conda\envs\mp\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\uweih034\.conda\envs\mp\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\888888\.conda\envs\mp\Lib\asyncio\base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "e:\tmp\metatest\OSSWatcher .py", line 40, in main
    pd = SubscriptionRunner()
         ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\888888\.conda\envs\mp\Lib\site-packages\pydantic\main.py", line 164, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\888888\.conda\envs\mp\Lib\site-packages\pydantic\_internal\_mock_val_ser.py", line 47, in __getattr__
    raise PydanticUserError(self._error_message, code=self._code)
pydantic.errors.PydanticUserError: `SubscriptionRunner` is not fully defined; you should define `Environment`, then call `SubscriptionRunner.model_rebuild()`.

For further information visit https://errors.pydantic.dev/2.5/u/class-not-fully-defined

{'base_commit': 'bdf9d224b5a05228897553a29214adc074fbc465', 'files': [{'path': 'metagpt/environment.py', 'Loc': {"('Environment', None, 27)": {'mod': []}}, 'status': 'modified'}, {'Loc': [21], 'path': None}]}


Specify Directory of pdf documents as Knowledge Base

Hi, how can we specify any folder which includes pdf documents as a knowledge base and create a new Role of Document Controller to extract specific information from within the documents in KB?

Any help would be highly appreciated

Thanks much appreciated

{'base_commit': 'f88fa9e2df09c28f867bda54ec24fa25b50be830', 'files': [{'path': 'metagpt/document_store', 'Loc': {}}, {'path': 'tests/metagpt/document_store', 'Loc': {}}, {'path': 'examples/search_kb.py', 'Loc': {}}]}


All custom components throw errors after update to latest version

### Bug Description

```
[01/29/25 00:15:00] ERROR    2025-01-29 00:15:00 - ERROR    - chat - Error building vertices: Error serializing vertex build response: Unable to serialize unknown type:  chat.py:405
                             <class 'pydantic._internal._model_construction.ModelMetaclass'>
```

### Reproduction

1. langflow updated to v1.1.2 from v1.1.1
2. all previously created custom components throwing error:

[01/29/25 00:24:09] ERROR    2025-01-29 00:24:09 - ERROR    - chat - Error building vertices: Error serializing vertex build response: Unable to serialize unknown type:  chat.py:405
                             <class 'pydantic._internal._model_construction.ModelMetaclass'>

### Expected behavior

Langflow should build tool correctly, as on previous version.

Simplified failing code:
```python
from langflow.custom import Component
from langflow.io import Output
from langflow.schema import Data
from langflow.field_typing import Tool
from langchain.tools import StructuredTool
from pydantic import BaseModel, Field

class MinimalSchema(BaseModel):
    input_text: str = Field(..., description="Text Input")

class SimpleToolComponentMinimalSchema(Component):
    display_name = "Simple Tool Minimal Schema Test"
    description = "Component with StructuredTool and minimal schema"
    outputs = [Output(display_name="Tool", name="test_tool", method="build_tool")]

    class MinimalSchema(BaseModel): # Define inner schema
        input_text: str = Field(..., description="Text Input")

    def build_tool(self) -> Tool:
        return StructuredTool.from_function( # Return directly - simplified
            name="minimal_tool",
            description="Minimal tool for testing schema",
            func=self.run_tool,
            args_schema=SimpleToolComponentMinimalSchema.MinimalSchema
        )

    def run_tool(self, input_text: str) -> str:
        return f"Tool received: {input_text}"
```

### Who can help?

_No response_

### Operating System

wsl Ubuntu latest

### Langflow Version

1.1.2

### Python Version

3.12

### Screenshot

_No response_

### Flow File

_No response_


Add pgVector in the building instructions for the PostgreSQL Docker image

### Feature Request

Include the pgVector component with the Docker build instructions. This would provide the use with a fully functional PostgreSQL Vector DB, ready to be used inside LangFlow.

### Motivation

I am not a programmer, neither I do have proper knowledge of SQL, but I liked to play with some RAG ideas and LangFlow seems perfect.
So, after installing the Docker version for development of LangFlow, I noticed that the PostgreSQL server is missing the pgVector component, or at least that is what I understood from the error messages.
Perhaps, it would be useful if the pgVector could be included in the Docker container, so having the user to just activate it on the SQL database. Anyway, I might be wrong, so in that case please forgive me.

### Your Contribution

After looking into the repository and searching around, with the help of AI (of course!), I found that the Docker instructions for the PostgreSQL server are defined inside the file \docker\cdk.Dockerfile (hope it's correct), and these might be the instructions to include pgVector:

```
FROM --platform=linux/amd64 python:3.10-slim

WORKDIR /app

# Install Poetry and build dependencies
RUN apt-get update && apt-get install -y \
    gcc \
    g++ \
    curl \
    build-essential \
    git \
    postgresql-server-dev-all \
    && rm -rf /var/lib/apt/lists/*

# Install Poetry
RUN curl -sSL https://install.python-poetry.org | python3 -

# Add Poetry to PATH
ENV PATH="${PATH}:/root/.local/bin"

# Copy the pyproject.toml and poetry.lock files
COPY poetry.lock pyproject.toml ./

# Copy the rest of the application codes
COPY ./ ./

# Install dependencies
RUN poetry config virtualenvs.create false && poetry install --no-interaction --no-ansi

# Install pgvector extension
RUN git clone https://github.com/pgvector/pgvector.git /tmp/pgvector && \
    cd /tmp/pgvector && \
    make && \
    make install && \
    rm -rf /tmp/pgvector

# Install additional dependencies
RUN poetry add botocore
RUN poetry add pymysql

# Command to run your application
CMD ["sh", "./container-cmd-cdk.sh"]
```

{'base_commit': '19818db68b507332be71f30dd90d16bf4c7d6f83', 'files': [{'path': 'docker_example/docker-compose.yml', 'Loc': {}}]}


Can't access flow directly by link

### Bug Description

When you try to access a flow using it's URL (ex. http://localhost:55650/flow/0b95342f-6ce4-43d0-9d60-c28bf66a3781), the page doesn't load and in the browser's console is shown the following message: ``Uncaught SyntaxError: Unexpected token '<' (at index-DK9323ab.js:1:1)``. I think that this problem is related to #1182 .

Navegating through the main page to access this flow works fine. If I reload the page, it doesn't load as described before.

### Reproduction

1. Run the Docker image langflowui/langflow
2. Open the langflow main page
3. Creates a new flow
4. Copy the flow link into a new tab or just reload the page

### Expected behavior

To open the flow editor page.

### Who can help?

_No response_

### Operating System

Docker image (langflowai/langflow) running in K8s

### Langflow Version

1.0.19

### Python Version

None

### Screenshot

Instead of loading the JS file, is loaded the HTML file as shown in the following picture:

![image](https://github.com/user-attachments/assets/4a192a11-5389-497f-8898-fd0598684ca1)

All requests in this image loads the same HTML.

### Flow File

_No response_


### Bug Description

redis.exceptions.ResponseError: unknown command 'module'

https://github.com/user-attachments/assets/32ea6046-d5f1-4d85-96b5-41d381776986

### Reproduction

Add a redis click run error, see the video

### Expected behavior

ResponseError: unknown command 'MODULE'

### Who can help?

_No response_

### Operating System

windows

### Langflow Version

1.0.18

### Python Version

3.11

### Screenshot

_No response_

### Flow File

_No response_


cannot import name 'DEFAULT_CONNECTION_STRING' in  v0.6.3a5

```
% git branch
* (HEAD detached at v0.6.3a5)
  dev
% cd docker_example
% docker compose up

[+] Running 1/0
 ✔ Container docker_example-langflow-1  Created                                                                    0.0s
Attaching to langflow-1
langflow-1  | Traceback (most recent call last):
langflow-1  |   File "/home/user/.local/bin/langflow", line 5, in <module>
langflow-1  |     from langflow.__main__ import main
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/__init__.py", line 5, in <module>
langflow-1  |     from langflow.processing.process import load_flow_from_json
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/processing/process.py", line 10, in <module>
langflow-1  |     from langflow.graph import Graph
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/graph/__init__.py", line 2, in <module>
langflow-1  |     from langflow.graph.graph.base import Graph
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/graph/graph/base.py", line 7, in <module>
langflow-1  |     from langflow.graph.graph.constants import lazy_load_vertex_dict
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/graph/graph/constants.py", line 1, in <module>
langflow-1  |     from langflow.graph.vertex import types
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/graph/vertex/types.py", line 5, in <module>
langflow-1  |     from langflow.graph.vertex.base import Vertex
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/graph/vertex/base.py", line 9, in <module>
langflow-1  |     from langflow.interface.initialize import loading
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/interface/initialize/loading.py", line 17, in <module>
langflow-1  |     from langflow.interface.custom_lists import CUSTOM_NODES
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/interface/custom_lists.py", line 7, in <module>
langflow-1  |     from langflow.interface.agents.custom import CUSTOM_AGENTS
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/interface/agents/__init__.py", line 1, in <module>
langflow-1  |     from langflow.interface.agents.base import AgentCreator
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/interface/agents/base.py", line 5, in <module>
langflow-1  |     from langflow.custom.customs import get_custom_nodes
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/custom/customs.py", line 1, in <module>
langflow-1  |     from langflow.template import frontend_node
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/template/frontend_node/__init__.py", line 1, in <module>
langflow-1  |     from langflow.template.frontend_node import (
langflow-1  |   File "/home/user/.local/lib/python3.10/site-packages/langflow/template/frontend_node/memories.py", line 7, in <module>
langflow-1  |     from langchain.memory.chat_message_histories.postgres import DEFAULT_CONNECTION_STRING
langflow-1  | ImportError: cannot import name 'DEFAULT_CONNECTION_STRING' from 'langchain.memory.chat_message_histories.postgres' (/home/user/.local/lib/python3.10/site-packages/langchain/memory/chat_message_histories/postgres.py)
langflow-1 exited with code 1
```


Method for Dynamically Manipulating Parameters of a Custom Component

```python
class DynamicConfigCustomComponent(CustomComponent):
    def build_config(self, prev_selection=None):
        config = {
            "param1": {"display_name": "Parameter 1"},
            "param2": {
                "display_name": "Parameter 2",
                "options": [1, 2, 3],
                "value": 1,
            },
        }

        if prev_selection is not None:
            if prev_selection == 2:
                config["param3"] = {"display_name": "Parameter 3", "value": "hello"}

        return config

```
I want to dynamically change different values depending on the type of component that is input or connected when using a custom component, as shown in the attached code. For example, in Langflow's prompt template, when you change the text, the key value input into that component is dynamically displayed in the list. Is there any way to do this?

{'base_commit': '12a46b6936e23829d9956d4d5f1fa51faff76137', 'files': [{'path': 'src/frontend/src/types/components/index.ts', 'Loc': {}}]}


Error with 'python -m autogpt' command. Please set your OpenAI API key in .env or as an environment variable. You can get your key from https://beta.openai.com/account/api-keys

### Duplicates

- [X] I have searched the existing issues

### Steps to reproduce 🕹

Installed the 'stable' version of the program
I run 'python -m autogpt' command and comes up with an error.

![Screenshot 2023-04-16 183147](https://user-images.githubusercontent.com/130889399/232320050-2b495403-55e9-4d43-b588-e53172eba533.jpg)

I have paid Chat GPT and Open AI API accounts.
For Chat GPT I have access to version 4
For Open AI API I do not have access to version 4, I am on the version before this.

### Current behavior 😯

Error message ;Please set your OpenAI API key in .env or as an environment variable.
You can get your key from https://beta.openai.com/account/api-keys'

### Expected behavior 🤔

Should load the program as to start commands

### Your prompt 📝

```yaml
python -m autogpt```


Add logging of user input of the role and goals

### Duplicates

- [X] I have searched the existing issues

### Summary 💡

Now logs reflect only gpt's response but i don't really remember what exactly i input before.  Please log it same as in the console.
Current logging makes it a lot harder to debug

### Examples 🌈
```
All packages are installed.
Welcome back!  Would you like me to return to being sc3?
Continue with the last settings?
Name:  sc3
Role:  warhammer 40k writer
Goals: ['research the theme', 'do a 5000 symbols structurized explanation on wh40k lore', 'terminate']
Continue (y/n): n
Welcome to Auto-GPT!  run with '--help' for more information.
Create an AI-Assistant:  Enter the name of your AI and its role below. Entering nothing will load defaults.
Name your AI:  For example, 'Entrepreneur-GPT'
AI Name: da23eads
da23eads here!  I am at your service.
Describe your AI's role:  For example, 'an AI designed to autonomously develop and run businesses with the sole goal of increasing your net worth.'
da23eads is: wh 40k writer
Enter up to 5 goals for your AI:  For example: Increase net worth, Grow Twitter Account, Develop and manage multiple businesses autonomously'
Enter nothing to load defaults, enter nothing when finished.
Goal 1: research the theme
Goal 2: do a plot esplanation on warhammer 40k universe
Goal 3: terminate
Goal 4:
Using memory of type:  LocalCache
Using Browser:  chrome
- Thinking...
```

### Motivation 🔦

make the world better


AutoGPT v0.3.1 crashes immediately after task given

### Which Operating System are you using?

Windows

### Which version of Auto-GPT are you using?

Latest Release v0.3.1

### GPT-3 or GPT-4?

GPT-3.5

### Steps to reproduce 🕹

Welcome to Auto-GPT!  run with '--help' for more information.
Create an AI-Assistant:  input '--manual' to enter manual mode.
  Asking user via keyboard...
I want Auto-GPT to: Search Big Mac prices in EU countries
Unable to automatically generate AI Config based on user desire.  Falling back to manual mode.
Create an AI-Assistant:  Enter the name of your AI and its role below. Entering nothing will load defaults.
Name your AI:  For example, 'Entrepreneur-GPT'
  Asking user via keyboard...
AI Name: MacGPT
MacGPT here!  I am at your service.
Describe your AI's role:  For example, 'an AI designed to autonomously develop and run businesses with the sole goal of increasing your net worth.'
  Asking user via keyboard...
MacGPT is: Search for Big Mc prices in EU countries
Enter up to 5 goals for your AI:  For example: Increase net worth, Grow Twitter Account, Develop and manage multiple businesses autonomously'
  Enter nothing to load defaults, enter nothing when finished.
  Asking user via keyboard...
Goal 1: Conduct a thorough and accurate search of BigMc prices across EU countries
  Asking user via keyboard...
Goal 2: Provide price per each EU capital
  Asking user via keyboard...
Goal 3: Ensure that the information provided is up-to-date and accurate
  Asking user via keyboard...
Goal 4: Continuously improve the search algorithm to increase the accuracy and efficiency of the search process.
  Asking user via keyboard...
Goal 5: Do not crash ang give error - "openai.error.AuthenticationError: <empty message>"
Enter your budget for API calls:  For example: $1.50
  Enter nothing to let the AI run without monetary limit
  Asking user via keyboard...
Budget: $1
MacGPT  has been created with the following details:
Name:  MacGPT
Role:  Search for Big Mc prices in EU countries
Goals:
-  Conduct a thorough and accurate search of BigMc prices across EU countries
-  Provide price per each EU capital
-  Ensure that the information provided is up-to-date and accurate
-  Continuously improve the search algorithm to increase the accuracy and efficiency of the search process.
-  Do not crash ang give error - "openai.error.AuthenticationError: <empty message>"
Using memory of type:  LocalCache
Using Browser:  chrome
Traceback (most recent call last):
  File "C:\Users\makkolev\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\agpt\autogpt\__main__.py", line 5, in <module>
    autogpt.cli.main()
  File "C:\agpt\autogpt_env\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "C:\agpt\autogpt_env\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\agpt\autogpt_env\lib\site-packages\click\core.py", line 1635, in invoke
    rv = super().invoke(ctx)
  File "C:\agpt\autogpt_env\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\agpt\autogpt_env\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "C:\agpt\autogpt_env\lib\site-packages\click\decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "C:\agpt\autogpt\cli.py", line 90, in main
    run_auto_gpt(
  File "C:\agpt\autogpt\main.py", line 186, in run_auto_gpt
    agent.start_interaction_loop()
  File "C:\agpt\autogpt\agent\agent.py", line 113, in start_interaction_loop
    assistant_reply = chat_with_ai(
  File "C:\agpt\autogpt\llm\chat.py", line 244, in chat_with_ai
    assistant_reply = create_chat_completion(
  File "C:\agpt\autogpt\llm\llm_utils.py", line 166, in create_chat_completion
    response = api_manager.create_chat_completion(
  File "C:\agpt\autogpt\llm\api_manager.py", line 55, in create_chat_completion
    response = openai.ChatCompletion.create
  File "C:\agpt\autogpt_env\lib\site-packages\openai\api_resources\chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "C:\agpt\autogpt_env\lib\site-packages\openai\api_resources\abstract\engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
  File "C:\agpt\autogpt_env\lib\site-packages\openai\api_requestor.py", line 226, in request
    resp, got_stream = self._interpret_response(result, stream)
  File "C:\agpt\autogpt_env\lib\site-packages\openai\api_requestor.py", line 619, in _interpret_response
    self._interpret_response_line(
  File "C:\agpt\autogpt_env\lib\site-packages\openai\api_requestor.py", line 682, in _interpret_response_line
    raise self.handle_error_response(
openai.error.AuthenticationError: <empty message>

### Current behavior 😯

Crashes multiple times. Open_API_key has been provided. Restarted virtual environment a couple of times.
NB! Tried to start AutoGPT both with Windows Python3.10 way and via Docker. In both cases can't start start search and receive immediately error (below) - openai.error.AuthenticationError: <empty message>

### Expected behavior 🤔

Starts correctly

### Your prompt 📝

```AI Name: MacGPT
MacGPT here!  I am at your service.
Describe your AI's role:  For example, 'an AI designed to autonomously develop and run businesses with the sole goal of increasing your net worth.'
  Asking user via keyboard...
MacGPT is: Search for Big Mc prices in EU countries
Enter up to 5 goals for your AI:  For example: Increase net worth, Grow Twitter Account, Develop and manage multiple businesses autonomously'
  Enter nothing to load defaults, enter nothing when finished.
  Asking user via keyboard...
Goal 1: Conduct a thorough and accurate search of BigMc prices across EU countries
  Asking user via keyboard...
Goal 2: Provide price per each EU capital
  Asking user via keyboard...
Goal 3: Ensure that the information provided is up-to-date and accurate
  Asking user via keyboard...
Goal 4: Continuously improve the search algorithm to increase the accuracy and efficiency of the search process.
  Asking user via keyboard...
Goal 5: Do not crash ang give error - "openai.error.AuthenticationError: <empty message>"
Enter your budget for API calls:  For example: $1.50
  Enter nothing to let the AI run without monetary limit
  Asking user via keyboard...
Budget: $1
MacGPT  has been created with the following details:
Name:  MacGPT
Role:  Search for Big Mc prices in EU countries
Goals:
-  Conduct a thorough and accurate search of BigMc prices across EU countries
-  Provide price per each EU capital
-  Ensure that the information provided is up-to-date and accurate
-  Continuously improve the search algorithm to increase the accuracy and efficiency of the search process.
-  Do not crash ang give error - "openai.error.AuthenticationError: <empty message>"
Using memory of type:  LocalCache
Using Browser:  chrome
```

### Your Logs 📒

```log
Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\makkolev\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\agpt\autogpt\__main__.py", line 5, in <module>
    autogpt.cli.main()
  File "C:\agpt\autogpt_env\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "C:\agpt\autogpt_env\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\agpt\autogpt_env\lib\site-packages\click\core.py", line 1635, in invoke
    rv = super().invoke(ctx)
  File "C:\agpt\autogpt_env\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\agpt\autogpt_env\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "C:\agpt\autogpt_env\lib\site-packages\click\decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "C:\agpt\autogpt\cli.py", line 90, in main
    run_auto_gpt(
  File "C:\agpt\autogpt\main.py", line 186, in run_auto_gpt
    agent.start_interaction_loop()
  File "C:\agpt\autogpt\agent\agent.py", line 113, in start_interaction_loop
    assistant_reply = chat_with_ai(
  File "C:\agpt\autogpt\llm\chat.py", line 244, in chat_with_ai
    assistant_reply = create_chat_completion(
  File "C:\agpt\autogpt\llm\llm_utils.py", line 166, in create_chat_completion
    response = api_manager.create_chat_completion(
  File "C:\agpt\autogpt\llm\api_manager.py", line 55, in create_chat_completion
    response = openai.ChatCompletion.create(
  File "C:\agpt\autogpt_env\lib\site-packages\openai\api_resources\chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "C:\agpt\autogpt_env\lib\site-packages\openai\api_resources\abstract\engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
  File "C:\agpt\autogpt_env\lib\site-packages\openai\api_requestor.py", line 226, in request
    resp, got_stream = self._interpret_response(result, stream)
  File "C:\agpt\autogpt_env\lib\site-packages\openai\api_requestor.py", line 619, in _interpret_response
    self._interpret_response_line(
  File "C:\agpt\autogpt_env\lib\site-packages\openai\api_requestor.py", line 682, in _interpret_response_line
    raise self.handle_error_response(
openai.error.AuthenticationError: <empty message>
Press any key to continue . . .
```


How to identify query params with keys only and no value

### First Check

- [X] I added a very descriptive title to this issue.
- [X] I used the GitHub search to find a similar issue and didn't find it.
- [X] I searched the FastAPI documentation, with the integrated search.
- [X] I already searched in Google "How to X in FastAPI" and didn't find any information.
- [X] I already read and followed all the tutorial in the docs and didn't find an answer.
- [X] I already checked if it is not related to FastAPI but to [Pydantic](https://github.com/samuelcolvin/pydantic).
- [X] I already checked if it is not related to FastAPI but to [Swagger UI](https://github.com/swagger-api/swagger-ui).
- [X] I already checked if it is not related to FastAPI but to [ReDoc](https://github.com/Redocly/redoc).

### Commit to Help

- [X] I commit to help with one of those options 👆

### Example Code

```python
@router.get("/events")
def get_alerts(request: Request)
    params = request.query_params
```

### Description

I want to handle a use case where I want to handle a use case where if a query param is passed but no value is set, I would return a specific message. I want a different behavior when to when it is not passed at all.

I tried using request.query_params but it doesn't get the Key in the request as well.

Postman request looks like this:
<img width="805" alt="image" src="https://user-images.githubusercontent.com/104721284/192010955-160c2418-63f3-46ac-9f64-a416b92c03ae.png">

### Operating System

macOS

### Operating System Details

_No response_

### FastAPI Version

0.70.0

### Python Version

3.9

### Additional Context

_No response_