Cohere Transcribe OpenVINO

This repository contains an OpenVINO™ IR version of Cohere Transcribe optimized for efficient inference with the OpenVINO Runtime. The model has been exported into separate encoder, decoder, and decoder-with-past graphs to enable fast autoregressive decoding using KV-cache.

The repository includes everything required to run inference with OpenVINO, including the model IR files, tokenizer, processor configuration, and metadata.

Model Details

Task: Automatic Speech Recognition (ASR)
Framework: OpenVINO Runtime
Model Format: OpenVINO IR (.xml + .bin)
Supported Devices: CPU, GPU, AUTO
Language: English,

Repository Contents

.
├── encoder.xml
├── encoder.bin
├── decoder.xml
├── decoder.bin
├── decoder_with_past.xml
├── decoder_with_past.bin
├── config.json
├── generation_config.json
├── preprocessor_config.json
├── tokenizer.json
├── tokenizer_config.json
├── special_tokens_map.json
├── meta.json
└── README.md

Installation

We recommend using uv to create an isolated Python environment.

1. Install uv

pip install uv

2. Create a virtual environment

uv venv

Activate the environment.

Linux / macOS

source .venv/bin/activate

Windows (PowerShell)

.venv\Scripts\Activate.ps1

Windows (CMD)

.venv\Scripts\activate.bat

Install Dependencies

uv pip install \
    openvino \
    transformers \
    numpy \
    librosa \
    datasets

or using pip

pip install \
    openvino \
    transformers \
    numpy \
    librosa \
    datasets

Quick Start

python inference.py

Supported Devices

The model can be executed on any OpenVINO-supported device.

Device	Supported
CPU	✅
GPU	✅
AUTO	✅
AUTO:CPU,GPU	✅

Performance Notes

The model uses:

OpenVINO Runtime
Separate encoder and decoder graphs
KV-cache based decoding
Decoder-with-past for efficient autoregressive generation

These optimizations significantly reduce decoding latency compared to running the full decoder at every generation step.

Acknowledgements

This model is an OpenVINO conversion of the original Cohere transcribe model.

Special thanks to:

OpenVINO™ Toolkit for providing the runtime and model optimization tools.
Hugging Face Transformers for preprocessing, tokenization, and model configuration support.
Cohere Labs for the original cohere-transcribe-03-2026 model.

Useful resources:

OpenVINO Documentation: https://docs.openvino.ai
OpenVINO GitHub: https://github.com/openvinotoolkit/openvino
Hugging Face OpenVINO Documentation: https://huggingface.co/docs/optimum/intel/openvino

License

This repository contains only an OpenVINO IR conversion of the original model.

Please refer to the original Cohere transcribe model repository for the model license and usage terms. The conversion itself does not modify the licensing of the original model.

Downloads last month: -; Downloads are not tracked for this model. How to track