TimeDistributed Wrapper not working with LSTM/GRU

Please make sure that the boxes below are checked before you submit your issue. If your issue is an implementation question, please ask your question on [StackOverflow](http://stackoverflow.com/questions/tagged/keras) or [join the Keras Slack channel](https://keras-slack-autojoin.herokuapp.com/) and ask there instead of filing a GitHub issue.

Thank you!

- [x] Check that you are up-to-date with the master branch of Keras. You can update with:
pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

- [x] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found [here](https://www.tensorflow.org/get_started/os_setup).

- [x] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

- [x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

I am trying to apply word level attention to words in a document after passing the sentences through a GRU. However, the TimeDistributed Wrapper isn't working with GRU/LSTM.

I get the following error

```python
File "/home/##/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 569, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
File "/home/##/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 632, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
File "/home/##/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 164, in create_node
    output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
File "/home/##/.local/lib/python2.7/site-packages/keras/layers/wrappers.py", line 129, in call
    y = self.layer.call(X)  # (nb_samples * timesteps, ...)
File "/home/##/.local/lib/python2.7/site-packages/keras/layers/recurrent.py", line 201, in call
    input_shape = K.int_shape(x)
File "/home/##/.local/lib/python2.7/site-packages/keras/backend/theano_backend.py", line 128, in int_shape
    raise Exception('Not a Keras tensor:', x)
Exception: ('Not a Keras tensor:', Reshape{3}.0)
```
The code snipped is written below

```python
input_layer = Input(shape=(document_size, img_h,), dtype='int32', name='input_layer')
embedding_layer = TimeDistributed(Embedding(len(W2V), img_w, input_length=img_h, weights=[W2V], trainable=True, mask_zero=True))(input_layer)
gru_word = TimeDistributed(GRU(GRU_layers[0], return_sequences=True, activation=conv_non_linear))(embedding_layer)
```


Compiling model in callback causes TypeError

**System information**.
- Have I written custom code (as opposed to using a stock example script provided in Keras): yes
- TensorFlow version (use command below): 2.8.0 (2.4 too)
- Python version: 3.7

**Describe the problem**.

In a fine-tuning case I would like to do transfer-learning phase first (with fine-tuned layers frozen) and after that,  all layers should be unfrozen. I wrote a callback that unfreeze the layers after few epochs. Unfortunately, after changing the layers' `trainable` attribute, the model should be recompiled - and the recompilation causes the `TypeError` (see colab).

I am aware that I can workaround this by compiling and fitting model twice - for both phases separately - but the usage of callback seems more elegant to me.

**Standalone code to reproduce the issue**.

https://colab.research.google.com/drive/1u6VlH6EIQGXSp7vEIngTasp3v2EE42Wi?usp=sharing

**Source code / logs**.

```
TypeError: 'NoneType' object is not callable
```

{'base_commit': '980a6be629610ee58c1eae5a65a4724ce650597b', 'files': [{'path': 'keras/engine/training.py', 'Loc': {"('Model', 'make_train_function', 998)": {'mod': []}}, 'status': 'modified'}]}


Is the cifar10_cnn.py example actually performing data augmentation?

When `datagen.fit(X_train)` is called in the [`cifar10_cnn.py` example](https://github.com/fchollet/keras/blob/master/examples/cifar10_cnn.py#L103), shouldn't it be (when `data_augmentation=True`):

``` python
datagen.fit(X_train, augment=True)
```

as the [default value for `augment` is `False`](https://github.com/fchollet/keras/blob/master/keras/preprocessing/image.py#L410)?

Also, I am right in thinking when using `augment=True` the original (i.e. non-augmented - ignoring any normalisation/standardisation) data is not necessarily trained on? If so, I thought data augmentation is a method of artificially increasing the size of your dataset, so shouldn't we additionally be training on the non-augmented data? Thanks
- [x] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps
- [x] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
  pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps
- [x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).


What happened to WordContextProduct?

``` python
In [1]: import keras

In [2]: keras.__version__
Out[2]: '0.3.2'

In [3]: from keras.layers.embeddings import WordContextProduct
Using Theano backend.
/usr/local/lib/python3.5/site-packages/theano/tensor/signal/downsample.py:5: UserWarning: downsample module has been moved to the pool module.
  warnings.warn("downsample module has been moved to the pool module.")
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-3-65e83b407b3e> in <module>()
----> 1 from keras.layers.embeddings import WordContextProduct

ImportError: cannot import name 'WordContextProduct'
```

This page now returns a 404: https://github.com/fchollet/keras/blob/master/examples/skipgram_word_embeddings.py

Was this code taken out of keras, or just moved somewhere else?

Thanks,

Zach

---

Please make sure that the boxes below are checked before you submit your issue. Thank you!
- [x] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps
- [x] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
  pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps
- [x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).
I'm trying something like this:

``` python
models = []

# Word vectors
model_word = Sequential()
model_word.add(Embedding(1e4, 300, input_length=1))
model_word.add(Reshape(dims=(300,)))
models.append(model_word)

# Context vectors
model_context = Sequential()
model_context.add(Embedding(1e4, 300, input_length=1))
model_context.add(Reshape(dims=(300,)))
models.append(model_context)

# Combined model
model = Sequential()
model.add(Merge(models, mode='dot'))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='mean_squared_error', optimizer=Adam(lr=0.001))
```

Does that look reasonable?  And then as input, I need to provide 2 lists of indexes?
"


Masking the output of a conv layer

Hi,
I am trying to apply a given mask in the output of a conv layer. The simplest form of my problem can be seen in the img

![image](https://cloud.githubusercontent.com/assets/810340/17194147/e8728ad4-542c-11e6-8c60-b2949c288cec.png)

The mask should be considered as an input when training/predicting. I have already tried to use the Merge layer (mode='mul') to apply the input mask as follows:

``` python
main_input= Input(shape=(3, 64, 64))
mask1_input = Input(shape=(1, 64, 64))
mask2_input = Input(shape=(1, 64, 64))

conv1 = Convolution2D(1,7,7, border_mode='same')(main_input)
merged_model1 = Sequential()
merged_model1.add(Merge([conv1, mask1_input], mode='mul'))

conv2 = Convolution2D(1, 7,7, border_mode='same')(main_input)
merged_model2 = Sequential()
merged_model2.add(Merge([conv2, mask2_input], mode='mul'))

model = Sequential()
model.add(Merge([merged_model1,merged_model2],mode='sum'))
```

But it is not working, maybe because I'm trying to merge a layer with a Tensor. But even if I could do that, I don't feel this is the right way to do that. Can someone help?

Please make sure that the boxes below are checked before you submit your issue. Thank you!
- [X] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps
- [X] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
  pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps


model.fit : AttributeError: 'Model' object has no attribute '_compile_metrics'

**System information**
- Have I written custom code (as opposed to using example directory): Yes
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Mint 19.3
- TensorFlow backend (yes / no): yes
- TensorFlow version: 2.0.0b1
- Keras version: 2.2.4-tf
- Python version: 3.6
- CUDA/cuDNN version: /
- GPU model and memory: GTX 940MX, 430.26

**Describe the current behavior**
The model.fit() function throws a `AttributeError: 'Model' object has no attribute '_compile_metrics'` exception.

**Describe the expected behavior**
It should work ?

**Code to reproduce the issue**
```python
import numpy as np
import tensorflow as tf

input_3D = tf.keras.Input(shape=(None, None, None, 1)) # unknown width, length and depth, 1 gray channel
network_3D = tf.keras.layers.Conv3D(
    filters = 128,  # dimensionality of output space
    kernel_size = 5,  # shape of 2D convolution window (5x5)
    strides = 1,  # stride of convolution along all spatial dimensions
    padding = "same", data_format = "channels_last",  # input with shape (batch, height, width, channels)
    activation = tf.keras.layers.LeakyReLU(alpha = 0.2),  # activation function to use
    use_bias = True,
    kernel_initializer = tf.keras.initializers.TruncatedNormal(stddev = 1e-2),
    # initializer for the kernel weights matrix
    bias_initializer = 'zeros',  # initializer for the bias vector
    input_shape = (None, None, None, 1)
)(input_3D)
network_3D = tf.keras.layers.BatchNormalization(
    momentum = 0.1, # momentum + decay = 1.0
    epsilon = 1e-5,
    scale = True
)(network_3D)

model = tf.keras.Model(inputs = input_3D, outputs = network_3D)
model.loss = tf.losses.mean_squared_error
model.optimizer = tf.keras.optimizers.Adam(learning_rate = 0.002)
v = np.zeros((100,100,100,100))
l = np.zeros((100,100,100))
model.fit(v, l, epochs = 20, batch_size = 1)
```

**Other info / logs**
```python
Traceback (most recent call last):
  File ".../venv/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3296, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-14-a0cacfaacdab>", line 1, in <module>
    history = model.fit(v, l, epochs = 20, batch_size = 1)
  File ".../venv/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 643, in fit
    use_multiprocessing=use_multiprocessing)
  File ".../venv/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 632, in fit
    shuffle=shuffle)
  File ".../venv/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 2385, in _standardize_user_data
    metrics=self._compile_metrics,
AttributeError: 'Model' object has no attribute '_compile_metrics'
```

{'base_commit': 'ed07472bc5fc985982db355135d37059a1f887a9', 'files': [{'path': 'keras/engine/training.py', 'Loc': {"('Model', 'compile', 40)": {'mod': []}}, 'status': 'modified'}]}


Improve Keras Documentation User Experience for Long Code Snippets By Removing The Need For Horizontal Slide Bars

**Category**: documentation user-experience
**Comment**: modify highlight.js  to wrap long documentation code snippets
**Why**: eliminates the need for a user to manually click and slide a horizontal slider just to get a quick sense of what available parameters and their default values are

**Context**
While reading the documentation, and coming from a scikit-learn background, I really like how their documentation shows all the class and method parameters ([example page](http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html)). It's very helpful to quickly be able to see the default parameters.

Take [Dense](https://keras.io/layers/core/#dense) for example. If the documentation looked like this (imagine this a code block, not individually highlighted lines):

`keras.layers.Dense(units, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)`

Benefits:
- easy to read
- no scrolling a horizontal slider
- immediately tells me the available parameters and their default values

Compare that experience to the current Keras experience:

```
keras.layers.Dense(units, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)
```

Disadvantages:
- requires scrolling horizontally to see the rest
- easy to lose track of where you are while scrolling
- requires physical action to see everything

The Keras team no-doubt is busy with much bigger concerns than documentation formatting. One could say that the "Arguments" are all listed below or by clicking the "Source". True, however the key point I'm trying to make is usability, and quick readability. Reading through an "Argument"'s verbose description, or having to scroll horizontally is not quick nor an optimal experience.

I'm not going to make a case for why making documentation easy-to-read is important. I think the Keras documentation **content** itself is outstanding.


problem of save/load model

HI,

Thanks for making such a wonderful tool!

I'm using Keras 1.0. I want to save and load the model both the arch and the parameters. So I use the method in FAQ. Here is the code.

```
def save_model(self, model, options):
    json_string = model.to_json()
    open(options['file_arch'], 'w').write(json_string)
    model.save_weights(options['file_weight'])

def load_model(self, options):
    self.model = model_from_json(open(options['file_arch']).read())
    self.model.load_weights(options['file_weight'])
    return self.model
```

When I load model and use model.predict(), there is a error:
AttributeError: 'NoneType' object has no attribute 'predict'

Don't know why. If I don't load the model from file, just train a model and use it, everything seems ok.

I checked the issues, most people just need to load the parameters. Is it possible when I load the architecture, I overwrite the old model and loose the model.predict()?

Thanks again for making Keras!

Ben

{'base_commit': '7a12fd0f8597760cf8e1238a9b021e247693517b', 'files': [{'path': 'keras/src/trainers/trainer.py', 'Loc': {"('Trainer', 'compile', 40)": {'mod': []}}, 'status': 'modified'}]}


Same model but graph gives bad performance

Hello,

I am learning to use Graph as it seems more powerful so I implemented one of my previous model which uses Sequential. Here is the model using sequential (number of dimension set in random):

```
def build_generation_embedding_model(self, dim):
    print "Build model ..."
    input_model = Sequential()
    input_model.add(TimeDistributedDense(dim, input_shape=(10,10)))
    input_model.add(LSTM(dim, return_sequences=False))
    input_model.add(Dense(dim))
    canonical_model = Sequential()
    canonical_model.add(TimeDistributedDense(dim, input_shape=(15,15)))
    canonical_model.add(LSTM(dim, return_sequences=False))
    canonical_model.add(Dense(dim))
    self.model = Sequential()
    self.model.add(Merge([input_model, canonical_model], mode='concat'))
    self.model.add(Dense(15))
    self.model.add(Activation('softmax'))
    self.model.compile(loss='categorical_crossentropy', optimizer='rmsprop')
```

The model works fine and below is my reimplementation using Graph:

```
def build_generation_embedding_model_graph(self, dim):
    self.model = Graph()
    self.model.add_input(name='input1', input_shape=(10,10))
    self.model.add_input(name='canonical', input_shape=(15,15))
    self.model.add_node(TimeDistributedDense(dim), name='Embed_input1', input='input1')
    self.model.add_node(TimeDistributedDense(dim), name='Embed_canonical', input='canonical')
    self.model.add_node(LSTM(dim, return_sequences=False), name='Hidden_input1', input='Embed_input1')
    self.model.add_node(LSTM(dim, return_sequences=False), name='Hidden_canonical', input='Embed_canonical')
    self.model.add_node(Dense(15), name='merge', inputs=['Hidden_input1','Hidden_canonical'], merge_mode='concat')
    self.model.add_node(Activation('softmax'), name='activation', input='merge')
    self.model.add_output(name='output', input='merge')
    self.model.compile('rmsprop', {'output':'categorical_crossentropy'})
```

My impression is that they are exactly the same model (grateful if somebody spotted something wrong there). But the model based on Graph gives a loss of 3.6 while the loss for the other one is around 0.002.

Is there a reason for this please ?

Thank you for your help


Loss Increases after some epochs

I have tried different convolutional neural network codes and I am running into a similar issue. The network starts out training well and decreases the loss but after sometime the loss just starts to increase. I have shown an example below:
Epoch 15/800
1562/1562 [==============================] - 49s - loss: 0.9050 - acc: 0.6827 - val_loss: 0.7667 - val_acc: 0.7323
Epoch 16/800
1562/1562 [==============================] - 49s - loss: 0.8906 - acc: 0.6864 - val_loss: 0.7404 - val_acc: 0.7434
Epoch 380/800
1562/1562 [==============================] - 49s - loss: 1.5519 - acc: 0.4880 - val_loss: 1.4250 - val_acc: 0.5233
Epoch 381/800
1562/1562 [==============================] - 48s - loss: 1.5416 - acc: 0.4897 - val_loss: 1.5032 - val_acc: 0.4868
Epoch 800/800
1562/1562 [==============================] - 49s - loss: 1.8483 - acc: 0.3402 - val_loss: 1.9454 - val_acc: 0.2398

I have tried this on different cifar10 architectures I have found on githubs. I am training this on a GPU Titan-X Pascal. This only happens when I train the network in batches and with data augmentation. I have changed the optimizer, the initial learning rate etc. I have also attached a link to the code. I just want a cifar10 model with good enough accuracy for my tests, so any help will be appreciated. The code is from this:
https://github.com/fchollet/keras/blob/master/examples/cifar10_cnn.py

{'base_commit': 'c2b844ba2fe8d0d597da9ef6a9af3b20d18d0bec', 'files': [{'path': 'examples/cifar10_cnn.py', 'Loc': {'(None, None, None)': {'mod': [65]}}, 'status': 'modified'}]}


Using keras for Distributed training raise RuntimeError("Graph is finalized and cannot be modified.")

I'm using keras for distributed training with following code:

``` python
#!/usr/bin/env python
# -*- coding:utf-8 -*-
# Created by Enigma on 2016/9/26

import numpy as np
import tensorflow as tf

# Define Hyperparameters
FLAGS = tf.app.flags.FLAGS

# For missions
tf.app.flags.DEFINE_string("ps_hosts", "",
                           "Comma-separated list of hostname:port pairs")
tf.app.flags.DEFINE_string("worker_hosts", "",
                           "Comma-separated list of hostname:port pairs")
tf.app.flags.DEFINE_string("job_name", "", "One of 'ps', 'worker'")
tf.app.flags.DEFINE_integer("task_index", 0, "Index of task within the job")

# Hyperparameters

from keras import backend as K
from keras.layers import Input, Dense
from keras.models import Model

def main(_):
    ps_hosts = FLAGS.ps_hosts.split(",")
    worker_hosts = FLAGS.worker_hosts.split(",")
    cluster = tf.train.ClusterSpec({"ps": ps_hosts, "worker": worker_hosts})

    server_config = tf.ConfigProto(
        gpu_options=tf.GPUOptions(allow_growth=True),
        log_device_placement=True)
    server = tf.train.Server(cluster, config=server_config,
                             job_name=FLAGS.job_name, task_index=FLAGS.task_index)

    if FLAGS.job_name == "ps":
        server.join()
    elif FLAGS.job_name == "worker":
        with tf.device(tf.train.replica_device_setter(
                worker_device="/job:worker/task:%d/cpu:0" % FLAGS.task_index,
                cluster=cluster)):
            global_step = tf.Variable(0, name='global_step', trainable=False)
            inputs = Input(shape=[1, ])
            hidden = Dense(10, activation='relu')(inputs)
            output = Dense(1, activation='sigmoid')(hidden)
            model = Model(input=inputs, output=output)

            saver = tf.train.Saver()
            summary_op = tf.merge_all_summaries()

        sv = tf.train.Supervisor(is_chief=(FLAGS.task_index == 0),
                                 logdir="./checkpoint/",
                                 # init_op=init_op,
                                 summary_op=summary_op,
                                 saver=saver,
                                 global_step=global_step,
                                 save_model_secs=60)
        with sv.managed_session(server.target) as sess:
            step = 0
            K.set_session(sess)
            model.compile(optimizer='sgd', loss='mse')
            while step < 1000000:
                train_x = np.random.randn(1)
                train_y = 2 * train_x + np.random.randn(1) * 0.33 + 10
                model.fit(train_x, train_y)
        sv.stop()

if __name__ == "__main__":
    tf.app.run()
```

then I run it with:

```
/opt/anaconda3/bin/python /cache/allenwoods/keras_dis_test.py --ps_hosts=0.0.0.0:48636 --worker_hosts=0.0.0.0:46261 --job_name=ps --task_index=0
/opt/anaconda3/bin/python /cache/allenwoods/keras_dis_test.py --ps_hosts=0.0.0.0:48636 --worker_hosts=0.0.0.0:46261 --job_name=worker --task_index=0
```

it doesn't work and return

```
Traceback (most recent call last):
  File "/cache/allenwoods/keras_dis_test.py", line 73, in <module>
    tf.app.run()
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv[:1] + flags_passthrough))
  File "/cache/allenwoods/keras_dis_test.py", line 69, in main
    model.fit(train_x, train_y)
  File "/opt/anaconda3/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/training/supervisor.py", line 969, in managed_session
    self.stop(close_summary_writer=close_summary_writer)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/training/supervisor.py", line 797, in stop
    stop_grace_period_secs=self._stop_grace_secs)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/training/coordinator.py", line 386, in join
    six.reraise(*self._exc_info_to_raise)
  File "/opt/anaconda3/lib/python3.5/site-packages/six.py", line 686, in reraise
    raise value
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/training/supervisor.py", line 959, in managed_session
    yield sess
  File "/cache/allenwoods/VRLforTraffic/src/missions/keras_dis_test.py", line 65, in main
    model.compile(optimizer='sgd', loss='mse')
  File "/opt/anaconda3/lib/python3.5/site-packages/keras/engine/training.py", line 484, in compile
    self.optimizer = optimizers.get(optimizer)
  File "/opt/anaconda3/lib/python3.5/site-packages/keras/optimizers.py", line 580, in get
    instantiate=True, kwargs=kwargs)
  File "/opt/anaconda3/lib/python3.5/site-packages/keras/utils/generic_utils.py", line 18, in get_from_module
    return res()
  File "/opt/anaconda3/lib/python3.5/site-packages/keras/optimizers.py", line 134, in __init__
    self.iterations = K.variable(0.)
  File "/opt/anaconda3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 149, in variable
    v = tf.Variable(value, dtype=_convert_string_dtype(dtype), name=name)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 215, in __init__
    dtype=dtype)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 327, in _init_from_args
    self._snapshot = array_ops.identity(self._variable, name="read")
  File "/opt/anaconda3/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 4150, in name_scope
    yield scope
  File "/opt/anaconda3/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3645, in get_controller
    yield default
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 4150, in name_scope
    yield scope
  File "/opt/anaconda3/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2891, in name_scope
    yield "" if new_stack is None else new_stack + "/"
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 4150, in name_scope
    yield scope
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 293, in _init_from_args
    initial_value, name="initial_value", dtype=dtype)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 657, in convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 180, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 167, in constant
    attrs={"value": tensor_value, "dtype": dtype_value}, name=name).outputs[0]
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2339, in create_op
    self._check_not_finalized()
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2080, in _check_not_finalized
    raise RuntimeError("Graph is finalized and cannot be modified.")
RuntimeError: Graph is finalized and cannot be modified.
```

I wondering if it happens because keras' model wasn't created as part of the graph used in tf.train.Supervisor, but I have not a clue on how to prove it or fix it. Any idea？

{'base_commit': '530eff62e5463e00d73e72c51cc830b9ac3a14ab', 'files': [{'path': 'keras/engine/training.py', 'Loc': {"('Model', '_make_train_function', 685)": {'mod': []}, "('Model', '_make_test_function', 705)": {'mod': []}, "('Model', '_make_predict_function', 720)": {'mod': []}}, 'status': 'modified'}, {'path': 'keras/backend/tensorflow_backend.py', 'Loc': {"(None, 'manual_variable_initialization', 31)": {'mod': []}}, 'status': 'modified'}]}


Parent module '' not loaded, cannot perform relative import with vgg16.py

just set up my ubuntu and have the python 3.5 installed, together with Keras...the following occurs:

RESTART: /usr/local/lib/python3.5/dist-packages/keras/applications/vgg16.py
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/keras/applications/vgg16.py", line 14, in <module>
    from ..models import Model
SystemError: Parent module '' not loaded, cannot perform relative import


stat:awaiting response from contributor
stale
type:Bug

Problem with framework agnostic KerasVariable slicing with another KerasVariable

I defined a KerasVariable with shape (n,d) in a `keras.Layer()` using `self.add_weight()`. I've also defined another KerasVariable with shape (1) , dtype="int32", and value 0.

```
self.first_variable = self.add_weight(
    initializer="zeros", shape=(self.N,input_shape[-1]), trainable=False
)
self.second_variable = self.add_weight(initializer="zeros",shape=(1), trainable=False, dtype="int32")
```

During a call to this custom layer, I'm trying to retrieve a specific index of the first variable using the 2nd variable with:

`self.first_variable[self.second_variable.value]`

This works as expected in pytorch backend, but throws an error in tensorflow backend.

```
Only integers, slices (`:`), ellipsis (`...`), tf.newaxis (`None`) and scalar tf.int32/tf.int64 tensors are valid indices, got <tf.Variable 'custom_layer/variable_1:0' shape=(1,) dtype=int32>

Arguments received by CustomLayer.call():
  • x=tf.Tensor(shape=(None, 1600), dtype=float32)
  • training=True
```

{'base_commit': 'da86250e5a95a7adccabd8821b0d51508c82bddc', 'files': [{'path': 'keras/src/ops/core.py', 'Loc': {"(None, 'slice', 388)": {'mod': []}}, 'status': 'modified'}]}


Shell history polution

I haven't used this, but I just thought maybe this is not such a good idea because it's going to make traversing shell history really irritating. Does this do anything to get around that, or are there any workarounds?

If not, I know in zsh you can just populate the command line with whatever you want using LBUFFER and RBUFFER. What if you made it an option to type "fuck" then hit ctrl-F (for "fuck"), and it would just replace your command line with the correction, and if there's multiple candidates cycle through them by hitting ctrl-F again. That also lets you edit the correction however you want as well.


Thinks 'sl' is install python softlayer

![image](https://user-images.githubusercontent.com/13007697/81414970-66971080-910d-11ea-8a44-da5ab9ca77f9.png)
Ah, yes. This wasn't a mis-spelling of ls at all, but me installing Python-Softlayer.

The output of `thefuck --version` (something like `The Fuck 3.1 using Python
3.5.0 and Bash 4.4.12(1)-release`):

 The Fuck 3.30 using Python 3.8.2 and Bash 5.0.16(1)-release

Your system (Debian 7, ArchLinux, Windows, etc.):

Manjaro

How to reproduce the bug:

Type sl in the terminal, then fuck

<!-- It's only with enough information that we can do something to fix the problem. -->


api调用时，回答的内容中出现莫名其妙的自动问答

使用的baichuan-13b模型
使用的scr/api_demo.py
提问内容为：你好
回答会如图
![image](https://github.com/hiyouga/LLaMA-Efficient-Tuning/assets/26214176/0d2beb92-e3b4-4126-a84f-d30bde97a194)

不明白为什么会出现自动的多轮自我问答


Batch Inference Error for qwen2vl Model After Full Fine-Tuning

### Reminder

- [X] I have read the README and searched the existing issues.

### System Info

- `llamafactory` version: 0.9.2.dev0
- Python version: 3.8.20
- PyTorch version: 2.4.1+cu121 (GPU)
- Transformers version: 4.46.1
- Datasets version: 3.1.0
- Accelerate version: 1.0.1
- PEFT version: 0.12.0
- TRL version: 0.9.6

### Reproduction

I have fine-tuned the qwen2vl model using the command:

```python
llamafactory-cli train examples/train_full/qwen2vl_full_sft.yaml
```
After saving the model in the "saves" directory, I attempted to perform batch inference using the provided script:

```python
python scripts/vllm_infer.py --model_name_or_path path_to_merged_model --dataset alpaca_en_demo
```
However, I encountered the following error:

```python
ValueError: This model does not support image input.
```

1.The model_path I used points to the model saved after running the full fine-tuning script.
2.I have successfully used the LoRA fine-tuned model (trained with the lora_sft script and merged with merge_lora script), which allows for inference using the method provided in the qwen2vl documentation.
3.However, the model saved after full fine-tuning does not seem to support direct inference in the same way.

### Others

_No response_

{'base_commit': '984b202f835d6f3f4869cbb1f0460bb2d9163fc1', 'files': [{'path': 'scripts/vllm_infer.py', 'Loc': {"(None, 'vllm_infer', 38)": {'mod': [43]}}, 'status': 'modified'}]}


Can we set default_system in yaml file when training?

### Reminder

- [X] I have read the README and searched the existing issues.

### System Info

- `llamafactory` version: 0.8.4.dev0
- Platform: Linux-5.4.0-125-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyTorch version: 2.4.0+cu121 (GPU)
- Transformers version: 4.44.2
- Datasets version: 2.21.0
- Accelerate version: 0.33.0
- PEFT version: 0.12.0
- TRL version: 0.9.6
- GPU type: NVIDIA A800-SXM4-80GB
- DeepSpeed version: 0.15.0

### Reproduction

 llamafactory-cli train

### Expected behavior

We do not need the `default_system` in `template.py`.
Set `default_system` in training yaml file to overwrite so we do not need to modify the source code in `template.py`.

### Others

_No response_


llama'response repeat input then the answer

### Reminder

- [ ] I have read the README and searched the existing issues.

### Reproduction

    input_ids = tokenizer(["[INST] " +{text}" + " [/INST]"], return_tensors="pt", add_special_tokens=False).input_ids.to('cuda')

    generate_input = {
        "input_ids": input_ids,
        "max_new_tokens": 512,
        "do_sample": True,
         "top_k": 10,
        "top_p": 0.95,
        "temperature": 0.01,
        "repetition_penalty": 1.3,
        "eos_token_id": tokenizer.eos_token_id,
        "bos_token_id": tokenizer.bos_token_id,
        "pad_token_id": tokenizer.pad_token_id
    }

    generate_ids = model.generate(**generate_input)
    response = tokenizer.decode(generate_ids[0], skip_special_tokens=True)
    print(response)

### Expected behavior

I expect that llama just response the answer. for example,
input is "[INST] how are you [/INST]", output "**I am fine**"
but it repeat the input, the output is "**[INST] how are you [/INST] I am fine**"

### System Info

_No response_

### Others

Do you have any suggestions? This behaviour will limit the speed of the output and I wonder why this happen?

{'base_commit': '18c6e6fea9dcc77c03b36301efe2025a87e177d5', 'files': [{'path': 'src/llmtuner/chat/chat_model.py', 'Loc': {"('ChatModel', 'chat', 88)": {'mod': [102]}}, 'status': 'modified'}]}


deepspeed进行一机八卡断点续训时，直接配置--checkpoint_dir参数，仅加载model权重，无法加载optimizer权重

在配置baichuan2模型在固定 `step` 进行断点续训时，希望同时加载mp_rank_00_model_states.pt以及zero_pp_rank_*_mp_rank_00_optim_states.pt

然而，在使用如下命令 `--checkpoint_dir` 启动断点续训时，并没有载入优化器zero_pp_rank_*_mp_rank_00_optim_states.pt
`
deepspeed  --num_gpus ${NUM_GPUS_PER_WORKER}  src/train_bash.py \
    --stage sft \
    --model_name_or_path /xxxxxxxxxx/model_weight \
    --deepspeed ./ds_config.json \
    --do_train \
    --dataset alpaca_gpt4_en \
    --template default \
    --checkpoint_dir /xxxxxxxxxxxxxxxxx/output_sft/checkpoint-1 \
    --finetuning_type full \
    --output_dir ./output_sft \
    --overwrite_cache \
    --overwrite_output_dir \
    --per_device_train_batch_size 2 \
    --gradient_accumulation_steps 16 \
    --lr_scheduler_type cosine \
    --logging_steps 1 \
    --save_steps 10000 \
    --learning_rate 5e-5 \
    --num_train_epochs 10.0 \
    --plot_loss \
    --fp16 | tee logs/train_g16_lr5e.log
`

请问如何才能顺利加载所有权重与状态？

{'base_commit': '13eb365eb768f30d46967dd5ba302ab1106a96b6', 'files': [{'path': 'src/llmtuner/tuner/sft/workflow.py', 'Loc': {"(None, 'run_sft', 19)": {'mod': [67]}}, 'status': 'modified'}]}


评估指标除了BLEU 分数和汉语 ROUGE 分数还能使用其他的评估指标吗？

我想把模型用于意图词槽的提取，一般这个任务的评价指标是准确率和F1 score等，请问在这个项目里能使用准确率和F1 score作为评价指标吗？应该怎么做呢？谢谢大佬解答~

{'base_commit': '5377d0bf95f2fc79b75b253e956a7945f3030ad3', 'files': [{'path': 'src/llmtuner/tuner/sft/metric.py', 'Loc': {}}]}


系统提示词

想请教下大佬，“系统提示词（非必填）“框传入的内容怎么输入给模型的，怎么和”输入。。“框传入的内容拼接的？对应的代码在哪里？

感谢感谢

{'base_commit': '93809d1c3b73898a89cbdd99061eeeed5fd4f6a7', 'files': [{'path': 'src/llmtuner/extras/template.py', 'Loc': {"('Template', '_encode', 93)": {'mod': [109]}}, 'status': 'modified'}]}


请教llama pro应该怎么用？是可以用来微调吗？

### Reminder

- [X] I have read the README and searched the existing issues.

### Reproduction

请教llama pro应该怎么用？是可以用来做pt和SFT吗？

### Expected behavior

_No response_

### System Info

_No response_

### Others

_No response_


Compute_Accuracy

### Reminder

- [X] I have read the README and searched the existing issues.

### System Info

![image](https://github.com/user-attachments/assets/4847743e-e25b-4136-a3f4-43a3e7335f80)

I'm curious about this metrics for and how could i use this? and when? ( ComputeAccuracy )

![image](https://github.com/user-attachments/assets/672f14bb-c812-45fe-ad77-d3c66f660ce5)
and I saw llama-factory paper's metrics ( multi-choice ) and I  wonder if this metrics are match with ComputeAccuracy

anyone can answer me ?

please tell me how can i use this metrics, give me some example commands

thank you!

### Reproduction

.

### Expected behavior

_No response_

### Others

_No response_

{'base_commit': 'e678c1ccb2583e7b3e9e5bf68b58affc1a71411c', 'files': [{'path': 'examples/train_lora/llama3_lora_eval.yaml', 'Loc': {}}]}


全量微调BaiChuan2-7B-Chat的yaml文件中如何修改超参数能在三张A6000上运行

### Reminder

- [X] I have read the README and searched the existing issues.

### System Info

- `llamafactory` version: 0.8.2.dev0
- Platform: Linux-3.10.0-1160.88.1.el7.x86_64-x86_64-with-glibc2.17
- Python version: 3.8.19
- PyTorch version: 2.3.0+cu121 (GPU)
- Transformers version: 4.41.2
- Datasets version: 2.20.0
- Accelerate version: 0.31.0
- PEFT version: 0.11.1
- TRL version: 0.9.4
- GPU type: NVIDIA RTX A6000
- DeepSpeed version: 0.14.0
- vLLM version: 0.4.3

### Reproduction
```yaml
### model
model_name_or_path: /data/Baichuan2-7B-Chat

### method
stage: sft
do_train: true
finetuning_type: full

### ddp
ddp_timeout: 180000000
deepspeed: examples/deepspeed/ds_z3_config.json

### dataset
dataset: entity
template: baichuan2
cutoff_len: 1024
max_samples: 1000
overwrite_cache: true
preprocessing_num_workers: 16

### output
output_dir: saves/baichuan2-7b/full/sft
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true

### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 2
learning_rate: 1.0e-4
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
pure_bf16: true

### eval
val_size: 0.1
per_device_eval_batch_size: 1
eval_strategy: steps
eval_steps: 500
```
### Expected behavior

您的项目中给出7B模型能在120G的显存上运行，现在我在3张A6000上运行会出现OOM，希望您能告诉我怎么修改超参数能让它跑起来。我也参考了之前的issue，设置了pure_bf16，仍然不能运行。

### Others

_No response_

{'base_commit': '955e01c038ccc708def77f392b0e342f2f51dc9b', 'files': [{'path': 'examples/deepspeed/ds_z3_offload_config.json', 'Loc': {}}]}


predict_oom

### Reminder

- [X] I have read the README and searched the existing issues.

### System Info

model_name_or_path: llm/Qwen2-72B-Instruct
# adapter_name_or_path: saves/qwen2_7b_errata_0705/lora_ace04_instruction_v1_savesteps_10/sft

### method
stage: sft
do_predict: true
finetuning_type: lora

### dataset
dataset: prompt_to_get_cot_normal
template: qwen
cutoff_len: 2048
max_samples: 1000
overwrite_cache: true
preprocessing_num_workers: 16

### output
output_dir: saves/qwen2_72b_errata_0712/lora/predict
overwrite_output_dir: true

### eval
per_device_eval_batch_size: 1
predict_with_generate: true
ddp_timeout: 180000000

### Reproduction

8卡A100 80G 在 72b 的基座 predict 1k的数据显示oom, 所有的显卡同时加载整个模型参数, 导致oom
据官方 160G 即可, 我这80*8 都不够, 请问是bug还是需要设置什么参数;

### Expected behavior

_No response_

### Others

_No response_

{'base_commit': '955e01c038ccc708def77f392b0e342f2f51dc9b', 'files': [{'path': 'Examples/train_lora/', 'Loc': {}}]}


glm-4-9b-chat-1m do_predict得到的generated_predictions.jsonl中的label出现了\n和一些非数据集中的结果。

### Reminder

- [X] I have read the README and searched the existing issues.

### System Info

llamafactory              0.7.2.dev0
Python 3.10.14
ubuntu 20.04

### Reproduction

$llamafactory-cli train glm_predict.yaml

generated_predictions.jsonl 输出
{"label": "\n[S,137.0]", "predict": "\n[S,137.0]"}
{"label": "\n", "predict": "\n"}
{"label": "\n", "predict": "\n"}
{"label": "\n[S,593", "predict": "\n[S,593"}
{"label": "\n[H,593", "predict": "\n[S,593"}

glm_predict.yaml 内容
### model
model_name_or_path: ./THUDM_glm-4-9b-chat-1m
adapter_name_or_path: saves/glm/lora/sft

### method
stage: sft
do_predict: true
finetuning_type: lora

### dataset
dataset: data_v0.1
template: glm4
overwrite_cache: true
preprocessing_num_workers: 16

### output
output_dir: saves/glm/lora/predict

### eval
per_device_eval_batch_size: 4
predict_with_generate: true

### Expected behavior

期望输出
generated_predictions.jsonl 输出
{"label": "[S,137.0]", "predict": "[S,137.0]"}
{"label": "[S,593]", "predict": "[S,593]"}
{"label": "[H,593]", "predict": "[S,593]"}

只包含"\n"的结果都没有出现在数据集中。

### Others

_No response_

{'base_commit': '3f11ab800f7dcf4b61a7c72ead4e051db11a8091', 'files': [{'path': 'src/llamafactory/data/template.py', 'Loc': {'(None, None, None)': {'mod': [663, 664]}}, 'status': 'modified'}]}


使用本项目训练baichuan-13b之后，如何在baichuan-13b中加载训练完的模型

训练完成后如何应该如何在baichuan-13b的项目中修改加载训练完成后的模型？


Phi-1.5跑RM lora 出现'NoneType' object is not subcriptable

### Reminder

- [X] I have read the README and searched the existing issues.

### Reproduction

sh脚本：
```
deepspeed --num_gpus 8 --master_port=9901 src/train_bash.py \
    --stage rm \
    --model_name_or_path Phi-1.5 \
    --deepspeed ds_config.json \
    --adapter_name_or_path sft_lora \
    --create_new_adapter \
    --do_train \
    --dataset comparision_gpt4_en \
    --template default \
    --finetuning_type lora \
    --lora_target Wqkv \
    --overwrite_ouput_dir \
    --output_dir rm_lora \
    --per_device_train_batch_size 2 \
    --gradient_accumulation_steps 4 \
    --lr_scheduler_type cosine \
    --logging_steps 1 \
    --save_steps 200 \
    --learning_rate 1e-6 \
    --num_train_epochs 1.0 \
    --max_steps 200 \
    --fp16 > rm.log 2>&1 &
wait

```

### Expected behavior

期望结果：成功读取权重并进入训练

### System Info

设备：NPU
包版本：
```
transformers==4.36.1
deepspeed==0.12.4
peft==0.7.1
trl==0.7.4
torch==2.1.0
accelerate==0.25.0
```

### Others

报错信息：
Traceback
 File "src/train_bash.py" line 14
    main()
File "src/train_bash.py" line 5
    run_exp()
File "LLaMA-Factory/src/llmtuner/train/tuner.py", line 28 in run_exp
    run_rm(model_args, data_args, training_args, finetuning_args, callbacks)
File "LLaMA-Factory/src/llmtuner/train/rm/workflow.py", line 50, in run_rm
    train_result = trainer.train(resume_from_checkpoint=training_args.resume_from_checkpoint)
File "transformers/trainer.py" line 2728
    loss = self.compute_loss(model, inputs)
File "LLaMA-Factory/src/llmtuner/train/rm/trainer.py" in line 41, in compute_loss
    _, _, values = model(**inputs, output_hidden_states=True, return_dict=True)
File ".../trl/models/modeling_value_head.py", in line 175. in forward
    last_hidden_state = base_model_output.hidden_state[-1]
TypeError: 'NoneType' object is not subscriptable

我一开始怀疑是权重问题，重新下载了权重依然报该错误，尝试将Phi-1.5换成Phi-2同样报错


请问项目中对多轮对话语料的处理方式

是用多个历史对话拼接后作为input来预测最后一轮的回答吗？还是把历史对话拆分成多个轮次的训练语料比如5轮次对话可以拆分成1 2 3 4 5轮次对话样本。关于具体的处理过程代码 能否请作者指出一下 我想学习学习。谢谢。

{'base_commit': 'd46c136c0e104c50999df18a88c42658b819f71f', 'files': [{'path': 'src/llmtuner/dsets/preprocess.py', 'Loc': {"(None, 'preprocess_supervised_dataset', 50)": {'mod': []}}, 'status': 'modified'}]}


After I ingest a file, run privateGPT and try to ask anything, I get following error:

```
Traceback (most recent call last):
  File "C:\Stable_Diffusion\privateGPT\privateGPT.py", line 75, in <module>
    main()
  File "C:\Stable_Diffusion\privateGPT\privateGPT.py", line 47, in main
    res = qa(query)
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\chains\base.py", line 140, in __call__
    raise e
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\chains\base.py", line 134, in __call__
    self._call(inputs, run_manager=run_manager)
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\chains\retrieval_qa\base.py", line 120, in _call
    answer = self.combine_documents_chain.run(
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\chains\base.py", line 239, in run
    return self(kwargs, callbacks=callbacks)[self.output_keys[0]]
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\chains\base.py", line 140, in __call__
    raise e
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\chains\base.py", line 134, in __call__
    self._call(inputs, run_manager=run_manager)
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\chains\combine_documents\base.py", line 84, in _call
    output, extra_return_dict = self.combine_docs(
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\chains\combine_documents\stuff.py", line 87, in combine_docs
    return self.llm_chain.predict(callbacks=callbacks, **inputs), {}
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\chains\llm.py", line 213, in predict
    return self(kwargs, callbacks=callbacks)[self.output_key]
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\chains\base.py", line 140, in __call__
    raise e
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\chains\base.py", line 134, in __call__
    self._call(inputs, run_manager=run_manager)
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\chains\llm.py", line 69, in _call
    response = self.generate([inputs], run_manager=run_manager)
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\chains\llm.py", line 79, in generate
    return self.llm.generate_prompt(
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\llms\base.py", line 134, in generate_prompt
    return self.generate(prompt_strings, stop=stop, callbacks=callbacks)
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\llms\base.py", line 191, in generate
    raise e
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\llms\base.py", line 185, in generate
    self._generate(prompts, stop=stop, run_manager=run_manager)
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\llms\base.py", line 405, in _generate
    self._call(prompt, stop=stop, run_manager=run_manager)
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\llms\llamacpp.py", line 225, in _call
    for token in self.stream(prompt=prompt, stop=stop, run_manager=run_manager):
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain\llms\llamacpp.py", line 274, in stream
    for chunk in result:
  File "C:\Users\olegt\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_cpp\llama.py", line 618, in _create_completion
    raise ValueError(
ValueError: Requested tokens exceed context window of 1000
```

I tried it with docx and pdf, used models are ggml-vic13b-q5_1.bin and stable-vicuna-13B.ggml.q4_0.bin.
During ingestion or loading privateGPT I get no error.

OS: Windows 10
CPU: Ryzen 7 3700
RAM: 32gb

{'base_commit': '2f3aab9cfdc139f399387dbb90300d5a8bf8d2f1', 'files': [{'path': 'ingest.py', 'Loc': {"(None, 'process_documents', 114)": {'mod': [124]}}, 'status': 'modified'}]}


LLM mock donot gives any output

i have downloaded the llm models

used this
poetry install --with local
poetry run python scripts/setup

still i get this output
![Screenshot from 2023-10-27 23-50-34](https://github.com/imartinez/privateGPT/assets/148402457/201b18f9-c269-40e4-99c5-a22fd3b9366d)

{'base_commit': '24cfddd60f74aadd2dade4c63f6012a2489938a1', 'files': [{'path': 'settings.yaml', 'Loc': {}}]}


privateGPT中文提问显示token超出限制，英文提问不存在这个问题


Error in Model Download and Tokenizer Fetching During Setup Script Execution

### Environment
Operating System: Macbook Pro M1
Python Version: 3.11

Description
I'm encountering an issue when running the setup script for my project. The script is supposed to download an embedding model and an LLM model from Hugging Face, followed by their respective tokenizers. While the script successfully downloads the embedding and LLM models, it fails when attempting to download the tokenizer with a 404 Client Error.

### Steps to Reproduce
Run `poetry run python scripts/setup`
Embedding model (BAAI/bge-small-en-v1.5) and the LLM model (mistral-7b-instruct-v0.2.Q4_K_M.gguf) are downloaded successfully.
The script then attempts to download a tokenizer and fails.

### Actual Behavior
The script throws an error when trying to download the tokenizer. The error message indicates a 404 Client Error: Not Found for url: https://huggingface.co/None/resolve/main/tokenizer_config.json. This suggests that either the tokenizer's name is not being correctly passed (as indicated by the 'None' in the URL) or there's an issue with the tokenizer's availability on Hugging Face.

### Logs
```bash
22:02:47.207 [INFO    ] private_gpt.settings.settings_loader - Starting application with profiles=['default']
Downloading embedding BAAI/bge-small-en-v1.5
Fetching 14 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 14/14 [00:00<00:00, 14.81it/s]
Embedding model downloaded!
Downloading LLM mistral-7b-instruct-v0.2.Q4_K_M.gguf
LLM model downloaded!
Downloading tokenizer None
Traceback (most recent call last):
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 270, in hf_raise_for_status
    response.raise_for_status()
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/None/resolve/main/tokenizer_config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/transformers/utils/hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
                    ^^^^^^^^^^^^^^^^
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1374, in hf_hub_download
    raise head_call_error
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1247, in hf_hub_download
    metadata = get_hf_file_metadata(
               ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1624, in get_hf_file_metadata
    r = _request_wrapper(
        ^^^^^^^^^^^^^^^^^
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 402, in _request_wrapper
    response = _request_wrapper(
               ^^^^^^^^^^^^^^^^^
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 426, in _request_wrapper
    hf_raise_for_status(response)
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 320, in hf_raise_for_status
    raise RepositoryNotFoundError(message, response) from e
huggingface_hub.utils._errors.RepositoryNotFoundError: 404 Client Error. (Request ID: Root=1-65f21479-09e39977255bdb72502d4b8c;66371627-2d02-44c6-8f25-d115820c1986)

Repository Not Found for url: https://huggingface.co/None/resolve/main/tokenizer_config.json.
Please make sure you specified the correct `repo_id` and `repo_type`.
If you are trying to access a private or gated repo, make sure you are authenticated.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/User/Projects/Tests/privateGPT/scripts/setup", line 43, in <module>
    AutoTokenizer.from_pretrained(
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 767, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 600, in get_tokenizer_config
    resolved_config_file = cached_file(
                           ^^^^^^^^^^^^
  File "/Users/User/Projects/Tests/privateGPT/.venv/lib/python3.11/site-packages/transformers/utils/hub.py", line 421, in cached_file
    raise EnvironmentError(
OSError: None is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`
```
### Additional Information
It seems like the script is not correctly fetching the name or identifier for the tokenizer.
The issue might be related to how the tokenizer's name is being resolved or passed in the script (None).
I also tried with docker compose, yielding same results. Maybe it is just some setting that I am missing from the docs?

Thank you


gpt_tokenize: unknown token '?'

gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
[1]    32658 killed     python3 privateGPT.py


LLM Chat only returns "#" characters

No matter the prompt, privateGPT only returns hashes as the response. This doesn't occur when not using CUBLAS.

Set up info:

NVIDIA GeForce RTX 4080
Windows 11

accelerate==0.25.0
aiofiles==23.2.1
aiohttp==3.9.1
aiosignal==1.3.1
aiostream==0.5.2
altair==5.2.0
annotated-types==0.6.0
anyio==3.7.1
attrs==23.1.0
beautifulsoup4==4.12.2
black==22.12.0
boto3==1.34.2
botocore==1.34.2
build==1.0.3
CacheControl==0.13.1
certifi==2023.11.17
cfgv==3.4.0
charset-normalizer==3.3.2
cleo==2.1.0
click==8.1.7
colorama==0.4.6
coloredlogs==15.0.1
contourpy==1.2.0
coverage==7.3.3
crashtest==0.4.1
cycler==0.12.1
dataclasses-json==0.5.14
datasets==2.14.4
Deprecated==1.2.14
dill==0.3.7
diskcache==5.6.3
distlib==0.3.8
distro==1.8.0
dnspython==2.4.2
dulwich==0.21.7
email-validator==2.1.0.post1
evaluate==0.4.1
fastapi==0.103.2
fastjsonschema==2.19.1
ffmpy==0.3.1
filelock==3.13.1
flatbuffers==23.5.26
fonttools==4.46.0
frozenlist==1.4.1
fsspec==2023.12.2
gradio==4.10.0
gradio_client==0.7.3
greenlet==3.0.2
grpcio==1.60.0
grpcio-tools==1.60.0
h11==0.14.0
h2==4.1.0
hpack==4.0.0
httpcore==1.0.2
httptools==0.6.1
httpx==0.25.2
huggingface-hub==0.19.4
humanfriendly==10.0
hyperframe==6.0.1
identify==2.5.33
idna==3.6
importlib-resources==6.1.1
iniconfig==2.0.0
injector==0.21.0
installer==0.7.0
itsdangerous==2.1.2
jaraco.classes==3.3.0
Jinja2==3.1.2
jmespath==1.0.1
joblib==1.3.2
jsonschema==4.20.0
jsonschema-specifications==2023.11.2
keyring==24.3.0
kiwisolver==1.4.5
llama-index==0.9.3
llama_cpp_python==0.2.29
markdown-it-py==3.0.0
MarkupSafe==2.1.3
marshmallow==3.20.1
matplotlib==3.8.2
mdurl==0.1.2
more-itertools==10.2.0
mpmath==1.3.0
msgpack==1.0.7
multidict==6.0.4
multiprocess==0.70.15
mypy==1.7.1
mypy-extensions==1.0.0
nest-asyncio==1.5.8
networkx==3.2.1
nltk==3.8.1
nodeenv==1.8.0
numpy==1.26.3
onnx==1.15.0
onnxruntime==1.16.3
openai==1.5.0
optimum==1.16.1
orjson==3.9.10
packaging==23.2
pandas==2.1.4
pathspec==0.12.1
pexpect==4.9.0
Pillow==10.1.0
pkginfo==1.9.6
platformdirs==4.1.0
pluggy==1.3.0
poetry==1.7.1
poetry-core==1.8.1
poetry-plugin-export==1.6.0
portalocker==2.8.2
pre-commit==2.21.0
-e git+https://github.com/imartinez/privateGPT@d3acd85fe34030f8cfd7daf50b30c534087bdf2b#egg=private_gpt
protobuf==4.25.1
psutil==5.9.6
ptyprocess==0.7.0
pyarrow==14.0.1
pydantic==2.5.2
pydantic-extra-types==2.2.0
pydantic-settings==2.1.0
pydantic_core==2.14.5
pydub==0.25.1
Pygments==2.17.2
pyparsing==3.1.1
pypdf==3.17.2
pyproject_hooks==1.0.0
pyreadline3==3.4.1
pytest==7.4.3
pytest-asyncio==0.21.1
pytest-cov==3.0.0
python-dateutil==2.8.2
python-dotenv==1.0.0
python-multipart==0.0.6
pytz==2023.3.post1
pywin32==306
pywin32-ctypes==0.2.2
PyYAML==6.0.1
qdrant-client==1.7.0
rapidfuzz==3.6.1
referencing==0.32.0
regex==2023.10.3
requests==2.31.0
requests-toolbelt==1.0.0
responses==0.18.0
rich==13.7.0
rpds-py==0.14.1
ruff==0.1.8
s3transfer==0.9.0
safetensors==0.4.1
scikit-learn==1.3.2
scipy==1.11.4
semantic-version==2.10.0
sentence-transformers==2.2.2
sentencepiece==0.1.99
shellingham==1.5.4
six==1.16.0
sniffio==1.3.0
soupsieve==2.5
SQLAlchemy==2.0.23
starlette==0.27.0
sympy==1.12
tenacity==8.2.3
threadpoolctl==3.2.0
tiktoken==0.5.2
tokenizers==0.15.0
tomlkit==0.12.0
toolz==0.12.0
torch==2.1.2+cu121
torchaudio==2.1.2+cu121
torchvision==0.16.2+cu121
tqdm==4.66.1
transformers==4.36.1
trove-classifiers==2024.1.8
typer==0.9.0
types-PyYAML==6.0.12.12
typing-inspect==0.9.0
typing_extensions==4.9.0
tzdata==2023.3
ujson==5.9.0
urllib3==1.26.18
uvicorn==0.24.0.post1
virtualenv==20.25.0
watchdog==3.0.0
watchfiles==0.21.0
websockets==11.0.3
wrapt==1.16.0
xxhash==3.4.1
yarl==1.9.4

{'base_commit': 'd3acd85fe34030f8cfd7daf50b30c534087bdf2b', 'files': [{'path': 'private_gpt/components/llm/llm_component.py', 'Loc': {"('LLMComponent', '__init__', 21)": {'mod': [45]}}, 'status': 'modified'}]}


CPU utilization

CPU utilization appears to be capped at 20%
Is there a way to increase CPU utilization and thereby enhance performance?


Segfault : not enough space in the context's memory pool

ggml_new_tensor_impl: not enough space in the context's memory pool (needed 3779301744, available 3745676000)
zsh: segmentation fault  python3.11 privateGPT.py

Whats context memory pool? can i configure it? i actually have a lot of excess memory


Use falcon model in privategpt

Hi how can i use Falcon model in privategpt?

https://huggingface.co/tiiuae/falcon-40b-instruct

Thanks


Need help on in some errors

 File "F:\privateGPT\Lib\site-packages\langchain\embeddings\llamacpp.py", line 79, in validate_environment
    values["client"] = Llama(
                             ^^^^^^
 File "F:\privateGPT\Lib\site-packages\llama_cpp\llama.py", line 155, in __init__
    self.ctx = llama_cpp.llama_init_from_file(
                  ^^^^^^^^^^^^^^^^^^^^^

  File "F:\privateGPT\Lib\site-packages\llama_cpp\llama_cpp.py", line 182, in llama_init_from_file
    return _lib.llama_init_from_file(path_model, params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [WinError -1073741795] Windows Error 0xc000001d
During handling of the above exception, another exception occurred:

 File "F:\privateGPT\ingest.py", line 62, in <module>
    main()
  File "F:\privateGPT\ingest.py", line 53, in main
    llama = LlamaCppEmbeddings(model_path=llama_embeddings_model, n_ctx=model_n_ctx)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pydantic\main.py", line 339, in pydantic.main.BaseModel.__init__
  File "pydantic\main.py", line 1102, in pydantic.main.validate_model
  File "F:\privateGPT\Lib\site-packages\langchain\embeddings\llamacpp.py", line 99, in validate_environment
    raise NameError(f"Could not load Llama model from path: {model_path}")
NameError: Could not load Llama model from path: F:/privateGPT/models/ggml-model-q4_0.bin
Exception ignored in: <function Llama.__del__ at 0x000002307F085E40>
Traceback (most recent call last):
  File "F:\privateGPT\Lib\site-packages\llama_cpp\llama.py", line 978, in __del__
    if self.ctx is not None:
       ^^^^
AttributeError: 'Llama' object has no attribute 'ctx'


Installed on Ubuntu 20.04 with Python3.11-venv

Error on line 38:
https://github.com/imartinez/privateGPT/blob/b1057afdf8f65fdb10e4160adbd8462be0c08271/privateGPT.py#L38C7-L38C7

Error:

Using embedded DuckDB with persistence: data will be stored in: db
Found model file at  models/ggml-gpt4all-j-v1.3-groovy.bin
Invalid model file
Traceback (most recent call last):
  File "/home/kk/Documents/privateGPT/privateGPT.py", line 83, in <module>
    main()
  File "/home/kk/Documents/privateGPT/privateGPT.py", line 38, in main
    llm = GPT4All(model=model_path, n_ctx=model_n_ctx, backend='gptj', n_batch=model_n_batch, callbacks=callbacks, verbose=False)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All
__root__
  Unable to instantiate model (type=value_error)


Enter a query，
It show:

ERROR: The prompt size exceeds the context window size and cannot be processed.GPT-J ERROR: The prompt is2614tokens and the context window is2048!

ERROR: The prompt size exceeds the context window size and cannot be processed.


Performance bottleneck using GPU

Hi Guys,

I am running the default Mistral model, and when running queries I am seeing 100% CPU usage (so single core), and up to 29% GPU usage which drops to have 15% mid answer.

I am using a MacBook Pro with M3 Max. I have set: model_kwargs={"n_gpu_layers": -1, "offload_kqv": True},

I am curious as LM studio runs the same model with low CPU usage and 80%+ GPU

{'base_commit': '6bbec79583b7f28d9bea4b39c099ebef149db843', 'files': [{'path': 'private_gpt/ui/ui.py', 'Loc': {"('PrivateGptUi', 'yield_deltas', 81)": {'mod': []}}, 'status': 'modified'}]}


How to limit YouTube Music search to tracks only?

### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I remove or skip any mandatory\* field

### Checklist

- [X] I'm asking a question and **not** reporting a bug or requesting a feature
- [X] I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme)
- [X] I've verified that I'm running yt-dlp version **2023.01.06** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)

### Please make sure the question is worded well enough to be understood

Is there a way to return only tracks in a YTmusic search? Sometimes music videos have sound effects, while I'm only interested in the original song.

I'm using this command:
`yt-dlp -f bestaudio --playlist-items 1 --default-search "https://music.youtube.com/search?q=" -a list-of-tracks.txt`

### Provide verbose output that clearly demonstrates the problem

- [ ] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [ ] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

_No response_

{'base_commit': '87ebab0615b1bf9b14b478b055e7059d630b4833', 'files': [{'path': 'yt_dlp/extractor/youtube.py', 'Loc': {"('YoutubeMusicSearchURLIE', None, 6647)": {'mod': [6676]}}, 'status': 'modified'}, {'path': 'yt_dlp/extractor/youtube.py', 'Loc': {"('YoutubeMusicSearchURLIE', None, 6647)": {'mod': [6659]}}, 'status': 'modified'}]}


Is YT-DLP capacity of downloading/displaying Automatic caption?

### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [X] I'm asking a question and **not** reporting a bug or requesting a feature
- [X] I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme)
- [X] I've verified that I'm running yt-dlp version **2023.06.22** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)

### Please make sure the question is worded well enough to be understood

**Similar Issue:**
- #5733

---
I may have missed one or 2 that could answer my question. These discussions and answers are not clear to my understanding, so here i am

**Details**
There are many videos that have "auto-generated subtitles | automatic captions" and no non-generated subtitles. I've ran `yt-dlp --list-subs URL` and discover that it said `URL has no subtitles`.

**QUESTION:**
1. Is it possible for yt-dlp to display the automatic caption while I am streaming the video to MPV?
2. Does yt-dlp preferred "non auto-generated caption"?

I'm not sure if this is intentional or not due to one discussion via issues that a guy mentioned that yt-dlp preferred non-autogenerated subtitles.

**Command for using MPV with yt-dlp**
the command was `mpv "https://youtu.be/i6kccBc-FBQ" --ytdl-raw-options=write-auto-subs=,write-subs=,sub-lang=en`

EDIT: added the double quote to the URL in the command line

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [ ] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['-vU', '--list-subs', 'https://youtu.be/i6kccBc-FBQ']
[debug] Portable config "C:\Program Scoop\apps\yt-dlp\current\yt-dlp.conf": []
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.06.22 [812cdfa06] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: none
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1851 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.06.22, Current version: stable@2023.06.22
Current Build Hash: 37e7ffe204309357cfd1388b0e2c782a30e293ddd0f2761a9a8f6afa185b3566
yt-dlp is up to date (stable@2023.06.22)
[youtube] Extracting URL: https://youtu.be/i6kccBc-FBQ
[youtube] i6kccBc-FBQ: Downloading webpage
[youtube] i6kccBc-FBQ: Downloading ios player API JSON
[youtube] i6kccBc-FBQ: Downloading android player API JSON
[debug] Loading youtube-nsig.b7910ca8 from cache
[debug] [youtube] Decrypted nsig ftRL4j1AuTut8ZV => WMPfJf_eWd71gQ
[youtube] i6kccBc-FBQ: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[info] Available automatic captions for i6kccBc-FBQ:
Language Name                  Formats
af       Afrikaans             vtt, ttml, srv3, srv2, srv1, json3
ak       Akan                  vtt, ttml, srv3, srv2, srv1, json3
sq       Albanian              vtt, ttml, srv3, srv2, srv1, json3
am       Amharic               vtt, ttml, srv3, srv2, srv1, json3
ar       Arabic                vtt, ttml, srv3, srv2, srv1, json3
hy       Armenian              vtt, ttml, srv3, srv2, srv1, json3
as       Assamese              vtt, ttml, srv3, srv2, srv1, json3
ay       Aymara                vtt, ttml, srv3, srv2, srv1, json3
az       Azerbaijani           vtt, ttml, srv3, srv2, srv1, json3
bn       Bangla                vtt, ttml, srv3, srv2, srv1, json3
eu       Basque                vtt, ttml, srv3, srv2, srv1, json3
be       Belarusian            vtt, ttml, srv3, srv2, srv1, json3
bho      Bhojpuri              vtt, ttml, srv3, srv2, srv1, json3
bs       Bosnian               vtt, ttml, srv3, srv2, srv1, json3
bg       Bulgarian             vtt, ttml, srv3, srv2, srv1, json3
my       Burmese               vtt, ttml, srv3, srv2, srv1, json3
ca       Catalan               vtt, ttml, srv3, srv2, srv1, json3
ceb      Cebuano               vtt, ttml, srv3, srv2, srv1, json3
zh-Hans  Chinese (Simplified)  vtt, ttml, srv3, srv2, srv1, json3
zh-Hant  Chinese (Traditional) vtt, ttml, srv3, srv2, srv1, json3
co       Corsican              vtt, ttml, srv3, srv2, srv1, json3
hr       Croatian              vtt, ttml, srv3, srv2, srv1, json3
cs       Czech                 vtt, ttml, srv3, srv2, srv1, json3
da       Danish                vtt, ttml, srv3, srv2, srv1, json3
dv       Divehi                vtt, ttml, srv3, srv2, srv1, json3
nl       Dutch                 vtt, ttml, srv3, srv2, srv1, json3
en-orig  English (Original)    vtt, ttml, srv3, srv2, srv1, json3
en       English               vtt, ttml, srv3, srv2, srv1, json3
eo       Esperanto             vtt, ttml, srv3, srv2, srv1, json3
et       Estonian              vtt, ttml, srv3, srv2, srv1, json3
ee       Ewe                   vtt, ttml, srv3, srv2, srv1, json3
fil      Filipino              vtt, ttml, srv3, srv2, srv1, json3
fi       Finnish               vtt, ttml, srv3, srv2, srv1, json3
fr       French                vtt, ttml, srv3, srv2, srv1, json3
gl       Galician              vtt, ttml, srv3, srv2, srv1, json3
lg       Ganda                 vtt, ttml, srv3, srv2, srv1, json3
ka       Georgian              vtt, ttml, srv3, srv2, srv1, json3
de       German                vtt, ttml, srv3, srv2, srv1, json3
el       Greek                 vtt, ttml, srv3, srv2, srv1, json3
gn       Guarani               vtt, ttml, srv3, srv2, srv1, json3
gu       Gujarati              vtt, ttml, srv3, srv2, srv1, json3
ht       Haitian Creole        vtt, ttml, srv3, srv2, srv1, json3
ha       Hausa                 vtt, ttml, srv3, srv2, srv1, json3
haw      Hawaiian              vtt, ttml, srv3, srv2, srv1, json3
iw       Hebrew                vtt, ttml, srv3, srv2, srv1, json3
hi       Hindi                 vtt, ttml, srv3, srv2, srv1, json3
hmn      Hmong                 vtt, ttml, srv3, srv2, srv1, json3
hu       Hungarian             vtt, ttml, srv3, srv2, srv1, json3
is       Icelandic             vtt, ttml, srv3, srv2, srv1, json3
ig       Igbo                  vtt, ttml, srv3, srv2, srv1, json3
id       Indonesian            vtt, ttml, srv3, srv2, srv1, json3
ga       Irish                 vtt, ttml, srv3, srv2, srv1, json3
it       Italian               vtt, ttml, srv3, srv2, srv1, json3
ja       Japanese              vtt, ttml, srv3, srv2, srv1, json3
jv       Javanese              vtt, ttml, srv3, srv2, srv1, json3
kn       Kannada               vtt, ttml, srv3, srv2, srv1, json3
kk       Kazakh                vtt, ttml, srv3, srv2, srv1, json3
km       Khmer                 vtt, ttml, srv3, srv2, srv1, json3
rw       Kinyarwanda           vtt, ttml, srv3, srv2, srv1, json3
ko       Korean                vtt, ttml, srv3, srv2, srv1, json3
kri      Krio                  vtt, ttml, srv3, srv2, srv1, json3
ku       Kurdish               vtt, ttml, srv3, srv2, srv1, json3
ky       Kyrgyz                vtt, ttml, srv3, srv2, srv1, json3
lo       Lao                   vtt, ttml, srv3, srv2, srv1, json3
la       Latin                 vtt, ttml, srv3, srv2, srv1, json3
lv       Latvian               vtt, ttml, srv3, srv2, srv1, json3
ln       Lingala               vtt, ttml, srv3, srv2, srv1, json3
lt       Lithuanian            vtt, ttml, srv3, srv2, srv1, json3
lb       Luxembourgish         vtt, ttml, srv3, srv2, srv1, json3
mk       Macedonian            vtt, ttml, srv3, srv2, srv1, json3
mg       Malagasy              vtt, ttml, srv3, srv2, srv1, json3
ms       Malay                 vtt, ttml, srv3, srv2, srv1, json3
ml       Malayalam             vtt, ttml, srv3, srv2, srv1, json3
mt       Maltese               vtt, ttml, srv3, srv2, srv1, json3
mi       Māori                 vtt, ttml, srv3, srv2, srv1, json3
mr       Marathi               vtt, ttml, srv3, srv2, srv1, json3
mn       Mongolian             vtt, ttml, srv3, srv2, srv1, json3
ne       Nepali                vtt, ttml, srv3, srv2, srv1, json3
nso      Northern Sotho        vtt, ttml, srv3, srv2, srv1, json3
no       Norwegian             vtt, ttml, srv3, srv2, srv1, json3
ny       Nyanja                vtt, ttml, srv3, srv2, srv1, json3
or       Odia                  vtt, ttml, srv3, srv2, srv1, json3
om       Oromo                 vtt, ttml, srv3, srv2, srv1, json3
ps       Pashto                vtt, ttml, srv3, srv2, srv1, json3
fa       Persian               vtt, ttml, srv3, srv2, srv1, json3
pl       Polish                vtt, ttml, srv3, srv2, srv1, json3
pt       Portuguese            vtt, ttml, srv3, srv2, srv1, json3
pa       Punjabi               vtt, ttml, srv3, srv2, srv1, json3
qu       Quechua               vtt, ttml, srv3, srv2, srv1, json3
ro       Romanian              vtt, ttml, srv3, srv2, srv1, json3
ru       Russian               vtt, ttml, srv3, srv2, srv1, json3
sm       Samoan                vtt, ttml, srv3, srv2, srv1, json3
sa       Sanskrit              vtt, ttml, srv3, srv2, srv1, json3
gd       Scottish Gaelic       vtt, ttml, srv3, srv2, srv1, json3
sr       Serbian               vtt, ttml, srv3, srv2, srv1, json3
sn       Shona                 vtt, ttml, srv3, srv2, srv1, json3
sd       Sindhi                vtt, ttml, srv3, srv2, srv1, json3
si       Sinhala               vtt, ttml, srv3, srv2, srv1, json3
sk       Slovak                vtt, ttml, srv3, srv2, srv1, json3
sl       Slovenian             vtt, ttml, srv3, srv2, srv1, json3
so       Somali                vtt, ttml, srv3, srv2, srv1, json3
st       Southern Sotho        vtt, ttml, srv3, srv2, srv1, json3
es       Spanish               vtt, ttml, srv3, srv2, srv1, json3
su       Sundanese             vtt, ttml, srv3, srv2, srv1, json3
sw       Swahili               vtt, ttml, srv3, srv2, srv1, json3
sv       Swedish               vtt, ttml, srv3, srv2, srv1, json3
tg       Tajik                 vtt, ttml, srv3, srv2, srv1, json3
ta       Tamil                 vtt, ttml, srv3, srv2, srv1, json3
tt       Tatar                 vtt, ttml, srv3, srv2, srv1, json3
te       Telugu                vtt, ttml, srv3, srv2, srv1, json3
th       Thai                  vtt, ttml, srv3, srv2, srv1, json3
ti       Tigrinya              vtt, ttml, srv3, srv2, srv1, json3
ts       Tsonga                vtt, ttml, srv3, srv2, srv1, json3
tr       Turkish               vtt, ttml, srv3, srv2, srv1, json3
tk       Turkmen               vtt, ttml, srv3, srv2, srv1, json3
uk       Ukrainian             vtt, ttml, srv3, srv2, srv1, json3
ur       Urdu                  vtt, ttml, srv3, srv2, srv1, json3
ug       Uyghur                vtt, ttml, srv3, srv2, srv1, json3
uz       Uzbek                 vtt, ttml, srv3, srv2, srv1, json3
vi       Vietnamese            vtt, ttml, srv3, srv2, srv1, json3
cy       Welsh                 vtt, ttml, srv3, srv2, srv1, json3
fy       Western Frisian       vtt, ttml, srv3, srv2, srv1, json3
xh       Xhosa                 vtt, ttml, srv3, srv2, srv1, json3
yi       Yiddish               vtt, ttml, srv3, srv2, srv1, json3
yo       Yoruba                vtt, ttml, srv3, srv2, srv1, json3
zu       Zulu                  vtt, ttml, srv3, srv2, srv1, json3
i6kccBc-FBQ has no subtitles
```

0
这个可不算，因为user知道命令只是引号问题


Unable to install curl_cffi

### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [X] I'm asking a question and **not** reporting a bug or requesting a feature
- [X] I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme)
- [X] I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels))
- [X] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)

### Please make sure the question is worded well enough to be understood

I am trying to install `curl_cffi` in order to get around Vimeo's new TLS fingerprinting anti-bot protection. I have run the command `pipx install 'yt-dlp[default,curl_cffi]' --force'`, which gives the output:

```
Installing to existing venv 'yt-dlp'
⚠  Note: yt-dlp was already on your PATH at /opt/homebrew/bin/yt-dlp
  installed package yt-dlp 2024.7.2, installed using Python 3.12.4
  These apps are now globally available
    - yt-dlp
  These manual pages are now globally available
    - man1/yt-dlp.1
⚠  Note: '/Users/username-hidden/.local/bin' is not on your PATH environment variable. These apps will not be globally accessible until your PATH is updated. Run `pipx ensurepath` to automatically add it,
    or manually modify your PATH in your shell's config file (e.g. ~/.bashrc).
done! ✨ 🌟 ✨
```

From this output, I understand that `curl_cffi` would have been installed. However, running `yt-dlp --list-impersonate-targets -vU` does not show it.

I intend to use `--impersonate chrome` but I am stuck at `curl_cffi` installation. Any help would be **greatly** appreciated. Thank you.

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [X] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['--list-impersonate-targets', '-vU']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.07.02 from yt-dlp/yt-dlp [93d33cb29] (pip)
[debug] Python 3.12.4 (CPython arm64 64bit) - macOS-14.5-arm64-arm-64bit (OpenSSL 3.3.1 4 Jun 2024)
[debug] exe versions: ffmpeg 7.0.1 (setts), ffprobe 7.0.1
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.06.02, mutagen-1.47.0, requests-2.32.3, sqlite3-3.46.0, urllib3-2.2.2, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1831 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.07.02 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2024.07.02 from yt-dlp/yt-dlp)
[info] Available impersonate targets
Client   OS   Source
---------------------------------------
Chrome   -    curl_cffi (not available)
Edge     -    curl_cffi (not available)
Safari   -    curl_cffi (not available)
```