--- tags: - ColBERT - PyLate - sentence-transformers - sentence-similarity - feature-extraction - medical-ai - clinical-nlp - medical - clinical - biomedical - xlm-roberta - cc-by-nc-4.0 pipeline_tag: sentence-similarity library_name: PyLate license: cc-by-nc-4.0 language: - en - es - fr - de - en base_model: - BAAI/bge-m3 --- | ⚖️ To use commercially, [please obtain a license](https://licensing.parallia.eu). 🙏 Thank you for supporting my research! 🤗 | |-----------------------------------------------------------------------------------------------------------------------------------| # ClinicalEncoder26AM: The First Multilingual Diagnosable ColBERT for Clinical Reasoning **ClinicalEncoder26AM** is a breakthrough in AI for healthcare—a **non-generative, interpretable reasoning model** that understands clinical text at millisecond speed, with token-level precision. Built on the new **Diagnosable ColBERT** architecture, it maps every word to a semantic clinical graph, enabling real-time reasoning, retrieval, and debugging. It is trained to support medical texts written in English, Spanish, French, German, and Dutch. It also provides good results for all 100+ languages supported by its base model. * 📖 **Read the full announcement:** [ClinicalEncoder26: The First Multilingual Diagnosable ColBERT](https://licensing.parallia.eu/blogs/news/clinicalencoder25-introducing-a-diagnosable-colbert-for-medical-retrieval) * 🧪 **Try the live demo:** [Hover over any word for more details!](http://demo26.parallia.eu/) --- ## **Why ClinicalEncoder26?** Most AI models today focus on generation, but **understanding comes first**. ClinicalEncoder26AM is designed for **deep, interpretable reasoning** in clinical and medical texts, with: - **Millisecond-latency document encoding** - **Token-level semantic mapping** to medical ontologies (UMLS, SnomedCT, ICD-10, etc.) - **Hallucination-free, non-generative reasoning** - **Live debugging and interpretability** via the Diagnosable ColBERT architecture It’s the first model to combine **late-interaction retrieval**, **clinical coding**, and **topic extraction** in a single, unified representation. --- ## **Model Details** ### **Model Description** - **Model Type:** XLM-Roberta - **Base Model:** [Bge-M3](https://huggingface.co/BAAI/bge-m3) - **Document Length:** up to 8194 tokens - **Query Length:** up to 8194 tokens - **Output Dimensionality:** 1024 - **Similarity Function:** MaxSim - **Language:** English, Spanish, French, German, Dutch, and 100 others - **License:** [CC-BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) ### **Key Features** - **Diagnosable ColBERT:** Every token is interpretable and mapped to a clinical concept. - **ClinicalMap25 Integration:** Use without the dense projection layer to map tokens directly to medical concepts at the L2 level. - **Efficient Retrieval:** Uses a late-interaction vector database for fast, scalable similarity search. ## Usage ### Token-to-Concept mapping To use the model in its Token-to-Concept mapping, the model should be used directly from the `transformers` library. To map Tokens to Concepts: - Encode your text using ClinicalEncoder26AM to obtain token embeddings. - Load the ClinicalMap25 embeddings for the target ontology, and compute cosine similarity between your token embeddings and the concept embeddings. - Retrieve the top-matching concepts for each token. See more details in the [Parallia/ClinicalMap25-for-SnomedCT](https://huggingface.co/datasets/Parallia/ClinicalMap25-for-SnomedCT) repository. For extracting local concept mappings, an adapter is required. For extracting a concept-likelihood score, an adapter is also required. If this is relevant to your use case, please contact us. ### Finetuning for BERT medical tasks This model is not a typical ColBERT model, and it can also be used as a finetuning base for many different medical tasks, including medical NER and NEL. To do so, just load the model using the `AutoModel.from_pretrained` method of the `transformers` library, and finetune the model for your favorite task. This model should perform amazingly well for tasks that require deep understanding of medical concepts. NOTE: Loading only the Transformer model with `transformers` will discard the dense layer. This is usually what you want but, for some tasks, the dense layer might provide value. It's advisable to try both approaches for best results. ### Retrieval #### Indexing documents with a single vector To index documents in a more tradional one-vector-per-document fashion, the usage of [MUVERA](https://research.google/blog/muvera-making-multi-vector-retrieval-as-fast-as-single-vector-search/) is recommended. Implementations exist in the [TxtAI library](https://medium.com/neuml/whats-new-in-txtai-9-0-d522bb150afa), and in [Weaviate](https://weaviate.io/blog/muvera). #### Indexing documents with multi-vector late interactions To use the model in its ColBERT mode, one of the easier approaches to install the PyLate library: ```bash pip install -U pylate ``` Other libraries you might want to consider: [TxtAI](https://github.com/neuml/txtai), [Weaviate](https://weaviate.io/). If you want to continue with PyLate, start by loading the ColBERT model and initializing the [FastPLAID](https://github.com/lightonai/fast-plaid) index, then encode and index your documents: ```python from pylate import indexes, models, retrieve # Step 1: Load the ColBERT model model = models.ColBERT( model_name_or_path="Parallia/ClinicalEncoder26AM-Diagnosable-Colbert-L2-for-multilingual-medical-texts", ) # TODO: check in which cases query epxansion helps or hurts retrieval model.do_query_expansion = False # If True, will pad the query to the `query_length` with mask tokens. Defaults to True. model.attend_to_expansion_tokens = False # If False, the original tokens will not only attend to the expansion tokens. Defaults to False. # Step 2: Initialize the PLAID index index = indexes.PLAID( index_folder="pylate-index", index_name="index", override=True, # This overwrites the existing index if any ) # Step 3: Encode the documents documents_ids = ["1", "2", "3"] documents = ["the patient is cold", "the weather is cold", "hypothermia"] documents_embeddings = model.encode( documents, batch_size=32, is_query=False, # Ensure that it is set to False to indicate that these are documents, not queries show_progress_bar=True, ) # Step 4: Add document embeddings to the index by providing embeddings and corresponding ids index.add_documents( documents_ids=documents_ids, documents_embeddings=documents_embeddings, ) ``` Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it: ```python # To load an index, simply instantiate it with the correct folder/name and without overriding it index = indexes.PLAID( index_folder="pylate-index", index_name="index", ) ``` #### Retrieving top-k documents for queries Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries. To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores: ```python # Step 1: Initialize the ColBERT retriever retriever = retrieve.ColBERT(index=index) # Step 2: Encode the queries queries_embeddings = model.encode( ["low body temperature", "it is snowing"], batch_size=32, is_query=True, # Ensure that it is set to True to indicate that these are queries, not documents show_progress_bar=True, ) # Step 3: Retrieve top-k documents scores = retriever.retrieve( queries_embeddings=queries_embeddings, k=10, # Retrieve the top 10 matches for each query ) ``` ### Reranking If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank: ```python from pylate import rank, models queries = [ "low body temperature", "it is snowing", ] documents = [ ["document A", "document B"], ["document 1", "document C", "document B"], ] documents_ids = [ [1, 2], [1, 3, 2], ] model = models.ColBERT( model_name_or_path="Parallia/ClinicalEncoder26AM-Diagnosable-Colbert-L2-for-multilingual-medical-texts", ) # TODO: check in which cases query epxansion helps or hurts retrieval model.do_query_expansion = False # If True, will pad the query to the `query_length` with mask tokens. Defaults to True. model.attend_to_expansion_tokens = False # If False, the original tokens will not only attend to the expansion tokens. Defaults to False. queries_embeddings = model.encode( queries, is_query=True, ) documents_embeddings = model.encode( documents, is_query=False, ) reranked_documents = rank.rerank( documents_ids=documents_ids, queries_embeddings=queries_embeddings, documents_embeddings=documents_embeddings, ) ``` --- ## **License** This model is released under the [CC-BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) license. For commercial use, [please obtain a license](https://licensing.parallia.eu).