Instructions to use sanchit-gandhi/parler-tts-600M-cross-attention-decayed with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use sanchit-gandhi/parler-tts-600M-cross-attention-decayed with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sanchit-gandhi/parler-tts-600M-cross-attention-decayed")

# Load model directly
from transformers import AutoModelForSeq2SeqLM
model = AutoModelForSeq2SeqLM.from_pretrained("sanchit-gandhi/parler-tts-600M-cross-attention-decayed", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use sanchit-gandhi/parler-tts-600M-cross-attention-decayed with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sanchit-gandhi/parler-tts-600M-cross-attention-decayed"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sanchit-gandhi/parler-tts-600M-cross-attention-decayed",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/sanchit-gandhi/parler-tts-600M-cross-attention-decayed

SGLang

How to use sanchit-gandhi/parler-tts-600M-cross-attention-decayed with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "sanchit-gandhi/parler-tts-600M-cross-attention-decayed" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sanchit-gandhi/parler-tts-600M-cross-attention-decayed",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "sanchit-gandhi/parler-tts-600M-cross-attention-decayed" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sanchit-gandhi/parler-tts-600M-cross-attention-decayed",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use sanchit-gandhi/parler-tts-600M-cross-attention-decayed with Docker Model Runner:
```
docker model run hf.co/sanchit-gandhi/parler-tts-600M-cross-attention-decayed
```

parler-tts-600M-cross-attention-decayed / starting_point_0.01.json

sanchit-gandhi

Saving train state of step 90000

e05457b verified over 2 years ago

Raw

History Blame Contribute Delete

2.54 kB

	{
	"model_name_or_path": "parler-tts/parler-tts-untrained-600M-cross-attention",
	"save_to_disk": "/fsx/sanchit/10k_hours_processed_punctuated",
	"temporary_save_to_disk": "/scratch/tmp_dataset_audio/",
	"push_to_hub": true,


	"feature_extractor_name":"ylacombe/dac_44khZ_8kbps",
	"description_tokenizer_name":"google/flan-t5-base",
	"prompt_tokenizer_name":"google/flan-t5-base",

	"report_to": ["wandb"],
	"wandb_run_name": "parler-tts-600M-cross-attention-decayed",
	"overwrite_output_dir": false,
	"output_dir": "./",
	"save_total_limit": 2,

	"train_dataset_name": "blabble-io/libritts_r+blabble-io/libritts_r+blabble-io/libritts_r+parler-tts/mls_eng_10k",
	"train_metadata_dataset_name": "parler-tts/libritts_r_tags_tagged_10k_generated+parler-tts/libritts_r_tags_tagged_10k_generated+parler-tts/libritts_r_tags_tagged_10k_generated+parler-tts/mls-eng-10k-tags_tagged_10k_generated",
	"train_dataset_config_name": "clean+clean+other+default",
	"train_split_name": "train.clean.360+train.clean.100+train.other.500+train",

	"eval_dataset_name": "blabble-io/libritts_r+parler-tts/mls_eng_10k",
	"eval_metadata_dataset_name": "parler-tts/libritts_r_tags_tagged_10k_generated+parler-tts/mls-eng-10k-tags_tagged_10k_generated",
	"eval_dataset_config_name": "other+default",
	"eval_split_name": "test.other+test",

	"target_audio_column_name": "audio",
	"description_column_name": "text_description",
	"prompt_column_name": "text",

	"max_eval_samples": 96,

	"max_duration_in_seconds": 30,
	"min_duration_in_seconds": 2.0,
	"max_text_length": 400,

	"group_by_length": true,

	"add_audio_samples_to_wandb": true,
	"id_column_name": "id",

	"preprocessing_num_workers": 8,

	"do_train": true,
	"num_train_epochs": 15,
	"gradient_accumulation_steps": 8,
	"gradient_checkpointing": false,
	"per_device_train_batch_size": 3,
	"learning_rate": 0.00095,
	"adam_beta1": 0.9,
	"adam_beta2": 0.99,
	"weight_decay": 0.01,

	"lr_scheduler_type": "cosine",
	"warmup_steps": 20000,


	"logging_steps": 1000,
	"freeze_text_encoder": true,


	"do_eval": true,
	"predict_with_generate": true,
	"include_inputs_for_metrics": true,
	"evaluation_strategy": "steps",
	"eval_steps": 10000,
	"save_steps": 10000,

	"per_device_eval_batch_size": 12,

	"audio_encoder_per_device_batch_size":20,
	"dtype": "bfloat16",
	"seed": 456,
	"ddp_timeout": 7200,

	"dataloader_num_workers":8
	}