Instructions to use GiacomoSignorile/PatentBert-FineTuned with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use GiacomoSignorile/PatentBert-FineTuned with sentence-transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("GiacomoSignorile/PatentBert-FineTuned")

sentences = [
    "COMPOSITIONS COMPRISING OR CONSISTING OF POLYDATIN FOR USE IN TREATMENT OF BONE DISEAS. Title: COMPOSITIONS COMPRISING OR CONSISTING OF POLYDATIN FOR USE IN TREATMENT OF BONE DISEAS\n\nAbstract: \n\nBusiness Description: The patent concerns the possible use of polydatin for the treatment of pathologies characterized by reduced bone mass. Polydatin is an oligostilbene that can be extracted in abundant quantities from the roots of the Polygonum Cuspidatum plant and, according to our studies about osteoblastic proliferation and differentiation, is able to increase osteogenic activity.\n\nTech Features: Skeletal pathologies are widespread in the world population, more than 200 million people are osteoporosis patients and have a high probability of fracture. Polydatin is a substance of natural origin, free of the side effects related to the use of synthetic drugs, currently administered for the treatment of bone loss diseases . Our experiments carried out in two-dimensional cultures (2D) and in cells grown on Scaffold (3D), have shown that polydatin is able to increase the activity of osteoblasts (ALP) and the deposition of Bone Matrix. The compositions comprising or consisting of polydatin may be administered orally (for example capsules or tablets, solutions, emulsions) or topically (oils, creams, ointments), or even through an appropriate enrichment of the active ingredient in functional foods\n\nApplications: Patients with traumatic-degenerative pathologies of the skeleton and oral cavity; Osteoporosis patients;\n\nAdvantages: Molecule of natural origin; Antioxidant activity on the whole organism; Possibility of topical use;",
    "Inhibitory compounds neurodegeneration and tumors. Title: Inhibitory compounds neurodegeneration and tumors\n\nAbstract: \n\nBusiness Description: The inhibition of the monoacylglycerol lipase enzyme (MAGL), naturally present in many brain cells and involved in physio-pathological processes, has a high therapeutic potential: neurodegenerative inflammation pathologies and tumors could be treated with new reversible inhibitory compounds, which would reduce the side effects of the irreversible inhibitors tested so far.\n\nTech Features: Monoacylglycerol lipase (MAGL) is a human enzyme of the endocannabinoid system involved in numerous physio-pathological processes (regulation of inflammation, anxiety, immune modulation, motor coordination ...), yet its overexpression/upregulation can cause neuroinflammatory diseases and tumors. The inhibition of MAGL for therapeutic purposes has been studied so far with irreversible inhibitors, which however nullify the enzyme activity, leading to a progressive loss of the therapeutic effect and to addiction phenomena. On the contrary, the new-patented compounds based on a strong non-covalent reversible mechanism of action avoid the side effects mentioned. Effective in laboratory on various tumor cell lines (e.g. colorectal, breast and ovarian cancer), they could also treat other MAGL-mediated pathologies (neuroinflammation/degeneration, pain, amyotrophic multiple/lateral sclerosis, Alzheimer's disease, Parkinson's disease). Link to scientific publication and information on Ca' Foscari website\n\nApplications: Innovative and less harmful pharmaceutical compositions for the treatment of serious neurodegenerative pathological conditions; Innovative and less harmful pharmaceutical compositions for cancer treatment.\n\nAdvantages: Temporary nature and reversibility; Drastic reduction of side effects; High efficacy tested on tumor cell lines; Exploitable to treat numerous neurodegenerative diseases; One of the few non-covalent reversible MAGL inhibitors with high efficacy.",
    "New computational method for the prognosis of amyotrophic lateral sclerosis. Title: New computational method for the prognosis of amyotrophic lateral sclerosis\n\nAbstract: A method is described for determining a disease progression and survival prognosis, at a succession of prediction times, for patients suffering from amyotrophic lateral sclerosis (ALS). The method comprises a step of defining a set of variables associated with the onset and progression of amyotrophic lateral sclerosis, comprising a first group of variables associated with the onset of amyotrophic lateral sclerosis (comprising at least the variables “patient sex”, “disease onset age”, “disease onset site”), a second group of dynamic time variables (comprising at least the variable “time elapsed since disease onset”), a third group of dynamic functional variables (comprising at least one of the variables breathing, swallowing, communicating, walking/self-care or at least one variable of a functional progression and/or severity scale of amyotrophic lateral sclerosis), and further at least one variable associated with survival. The method further provides for encoding by means of a Dynamic Bayesian Network, using at least one trained algorithm, a plurality of probabilistic conditional dependence relationships, in which each relationship is a probabilistic conditional dependence relationship between two of the aforesaid variables. The aforesaid prediction times are defined so that each prediction time belongs to a respective time interval in which the conditional dependence relationships between the variables are stationary. The method further involves describing the Dynamic Bayesian Network, using at least one trained algorithm, by means of a corresponding graph, comprising said variables as nodes and comprising topological connections oriented between nodes corresponding to variables among which a probabilistic conditional dependence is identified. In the graph, given a node, the connections entering it show a conditional probability of the value assumed by the variable associated with such node, in a given prediction time, depending on the values assumed, in a prior prediction time, from the variables associated with the nodes from which such connections originate. The method further comprises the steps of entering, for each of the defined variables, data acquired at a given acquisition time relating to the situation of a specific patient; and calculating, by electronic processing and/or calculating means, on the basis of the Dynamic Bayesian Network and the graph, and starting from the aforesaid acquired data, the values of each of the defined variables, at one or more prediction times following the acquisition time. Finally, the method involves obtaining, in a given prediction time, disease progression prognosis results on the basis of the values of one or more of the variables of the third group calculated in such prediction time; and the survival prognosis results on the basis of the value of at least one variable associated with survival, calculated at such prediction time.\n\nBusiness Description: The invention relates to a new probabilistic model based on dynamic Bayesian networks: by introducing specific clinical data of patients suffering from amyotrophic lateral sclerosis (ALS), the model object of the invention allows to predict the course of the disease, the loss of autonomy in specific functional domains and the survival, and to identify risk factors of the disease.\n\nTech Features: Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disease, characterized by progressive muscle paralysis caused by motor neuron degeneration. The onset and progression of ALS are very heterogeneous, making the development of personalised approaches extremely difficult. The invention consists of a probabilistic predictor of ALS progression, based on a dynamic Bayesian network built using a database of over 4000 patients treated at specialised centres in Italy and Israel. By intoducing the clinical information of a new ALS subject, the invention allows to simulate the clinical course and to probabilistically predict the risk of impairment of the functional domains typical of ALS and the survival, representing an important tool to support clinical decision-making and the drafting of the treatment plan.\n\nApplications: Online platform (SaaS)/software for predicting ALS progression; Support to clinical decision and drafting of the treatment plan; Institutional stakeholder decision support; In silico generation of patient populations with specific characteristics.\n\nAdvantages: Prediction of ALS progression; Personalized medicine approach; Clinical decision support; Identification of new prognostic markers; Possible alternative to placebo cohorts in clinical trials.",
    "COMPOSITIONS COMPRISING OR CONSISTING OF POLYDATIN FOR USE IN TREATMENT OF BONE DISEAS. Title: COMPOSITIONS COMPRISING OR CONSISTING OF POLYDATIN FOR USE IN TREATMENT OF BONE DISEAS\n\nAbstract: \n\nBusiness Description: The patent concerns the possible use of polydatin for the treatment of pathologies characterized by reduced bone mass. Polydatin is an oligostilbene that can be extracted in abundant quantities from the roots of the Polygonum Cuspidatum plant and, according to our studies about osteoblastic proliferation and differentiation, is able to increase osteogenic activity.\n\nTech Features: Skeletal pathologies are widespread in the world population, more than 200 million people are osteoporosis patients and have a high probability of fracture. Polydatin is a substance of natural origin, free of the side effects related to the use of synthetic drugs, currently administered for the treatment of bone loss diseases . Our experiments carried out in two-dimensional cultures (2D) and in cells grown on Scaffold (3D), have shown that polydatin is able to increase the activity of osteoblasts (ALP) and the deposition of Bone Matrix. The compositions comprising or consisting of polydatin may be administered orally (for example capsules or tablets, solutions, emulsions) or topically (oils, creams, ointments), or even through an appropriate enrichment of the active ingredient in functional foods\n\nApplications: Patients with traumatic-degenerative pathologies of the skeleton and oral cavity; Osteoporosis patients;\n\nAdvantages: Molecule of natural origin; Antioxidant activity on the whole organism; Possibility of topical use;"
]
embeddings = model.encode(sentences)

similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]

Notebooks
Google Colab
Kaggle

GiacomoSignorile commited on Mar 13

Commit

8e441ee

verified ·

1 Parent(s): b74754c

Initial upload of fine-tuned PatentSBERTa Specialist (eHealth)

Browse files

Files changed (4) hide show

README.md +50 -47
model.safetensors +1 -1
sentence_bert_config.json +1 -1
tokenizer.json +1 -1

README.md CHANGED Viewed

@@ -5,38 +5,36 @@ tags:
 - feature-extraction
 - dense
 - generated_from_trainer
-- dataset_size:3907
 - loss:MultipleNegativesRankingLoss
 base_model: AI-Growth-Lab/PatentSBERTa
 widget:
-- source_sentence: 'TMEK2: DEVICE FOR THE QUANTIFICATION OF BIOLOGICAL COMPONENTS
-    IN A FLUID'
   sentences:
-  - 'Technical Classification: C09D_11'
-  - 'Technical Classification: G01M'
-  - 'Technical Classification: G01N_33'
-- source_sentence: Portable optical analyzer for agri-food sector
   sentences:
-  - 'Technical Classification: A61L_27'
-  - 'Technical Classification: H01L_21'
-  - 'Technical Classification: G01N_21'
-- source_sentence: Determining the antioxidant power of biological fluids via palladium
-    nanoparticles
   sentences:
-  - 'Technical Classification: E01C_23'
-  - 'Technical Classification: A61K_31'
-  - 'Technical Classification: G01N_33'
-- source_sentence: Helical scanning system for curved tubes
   sentences:
-  - 'Technical Classification: G01N_29'
-  - 'Technical Classification: B01D_53'
-  - 'Technical Classification: A61P_25'
-- source_sentence: MICROSCOPY METHOD AND APPARATUS FOR OPTICAL TRACKING OF EMITTER
-    OBJECTS
   sentences:
-  - 'Technical Classification: H04N_13'
-  - 'Technical Classification: E04G_21'
-  - 'Technical Classification: B64D_27'
 pipeline_tag: sentence-similarity
 library_name: sentence-transformers
 ---
@@ -50,7 +48,7 @@ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [A
 ### Model Description
 - **Model Type:** Sentence Transformer
 - **Base model:** [AI-Growth-Lab/PatentSBERTa](https://huggingface.co/AI-Growth-Lab/PatentSBERTa) <!-- at revision 3ff1d553c861d8f5bfd902333d97fc95eb6b4c8f -->
-- **Maximum Sequence Length:** 512 tokens
 - **Output Dimensionality:** 768 dimensions
 - **Similarity Function:** Cosine Similarity
 <!-- - **Training Dataset:** Unknown -->
@@ -67,7 +65,7 @@ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [A
 ```
 SentenceTransformer(
-  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'MPNetModel'})
   (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
 )
 ```
@@ -90,9 +88,9 @@ from sentence_transformers import SentenceTransformer
 model = SentenceTransformer("sentence_transformers_model_id")
 # Run inference
 sentences = [
-    'MICROSCOPY METHOD AND APPARATUS FOR OPTICAL TRACKING OF EMITTER OBJECTS',
-    'Technical Classification: H04N_13',
-    'Technical Classification: B64D_27',
 ]
 embeddings = model.encode(sentences)
 print(embeddings.shape)
@@ -101,9 +99,9 @@ print(embeddings.shape)
 # Get the similarity scores for the embeddings
 similarities = model.similarity(embeddings, embeddings)
 print(similarities)
-# tensor([[1.0000, 0.4365, 0.3320],
-#         [0.4365, 1.0000, 0.4859],
-#         [0.3320, 0.4859, 1.0000]])
 ```
 <!--
@@ -148,19 +146,19 @@ You can finetune this model on your own dataset.
 #### Unnamed Dataset
-* Size: 3,907 training samples
 * Columns: <code>sentence_0</code> and <code>sentence_1</code>
 * Approximate statistics based on the first 1000 samples:
-  |         | sentence_0                                                                        | sentence_1                                                                       |
-  |:--------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
-  | type    | string                                                                            | string                                                                           |
-  | details | <ul><li>min: 4 tokens</li><li>mean: 12.62 tokens</li><li>max: 38 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 9.89 tokens</li><li>max: 15 tokens</li></ul> |
 * Samples:
-  | sentence_0                                                          | sentence_1                                          |
-  |:--------------------------------------------------------------------|:----------------------------------------------------|
-  | <code>INNOVATIVE PLATFORM FOR THE STUDY OF AGING PATHOLOGIES</code> | <code>Technical Classification: G16B_25</code>      |
-  | <code>AGENT FOR THE TREATMENT OF LEIOMYOMAS</code>                  | <code>Technical Classification: A61K_31_7105</code> |
-  | <code>POLY DIVINYLBENZENE FOR POLYPEPTIDE SYNTHESIS</code>          | <code>Technical Classification: C08F_12</code>      |
 * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
   ```json
   {
@@ -173,14 +171,14 @@ You can finetune this model on your own dataset.
 ### Training Hyperparameters
 #### Non-Default Hyperparameters
-- `per_device_train_batch_size`: 16
-- `per_device_eval_batch_size`: 16
 - `multi_dataset_batch_sampler`: round_robin
 #### All Hyperparameters
 <details><summary>Click to expand</summary>
-- `per_device_train_batch_size`: 16
 - `num_train_epochs`: 3
 - `max_steps`: -1
 - `learning_rate`: 5e-05
@@ -223,7 +221,7 @@ You can finetune this model on your own dataset.
 - `project`: huggingface
 - `trackio_space_id`: trackio
 - `eval_strategy`: no
-- `per_device_eval_batch_size`: 16
 - `prediction_loss_only`: True
 - `eval_on_start`: False
 - `eval_do_concat_batches`: True
@@ -283,7 +281,12 @@ You can finetune this model on your own dataset.
 ### Training Logs
 | Epoch  | Step | Training Loss |
 |:------:|:----:|:-------------:|
-| 2.0408 | 500  | 2.0066        |
 ### Framework Versions

 - feature-extraction
 - dense
 - generated_from_trainer
+- dataset_size:2633
 - loss:MultipleNegativesRankingLoss
+- dataset_size:7861
 base_model: AI-Growth-Lab/PatentSBERTa
 widget:
+- source_sentence: CONSORTIUM OF PROBIOTICS.
   sentences:
+  - NANOSTRUCTURED FILTERING MEMBRANE FOR ANALYTICAL APPLICATIONS.
+  - CONSORTIUM OF PROBIOTICS.
+  - Real-time communications over Bluetooth Low Energy.
+- source_sentence: Device for the collection and analysis of a biological fluid.
   sentences:
+  - Device for the collection and analysis of a biological fluid.
+  - STED MICROSCOPY BASED ON SYNCHRONOUS DETECTION OF FLUORESCENCE EMISSION.
+  - Robotic apparatus for minimally invasive surgery.
+- source_sentence: Method for obtaining a porous semiconductor.
   sentences:
+  - 'Racetrack Logic: smart in-memory computing.'
+  - Method for obtaining a porous semiconductor.
+  - Salts of benzimidazole compounds, their use and synthetic preparation.
+- source_sentence: Enthalpy exchangers with polymeric membrane.
   sentences:
+  - Closure system for engine connecting rods.
+  - Enthalpy exchangers with polymeric membrane.
+  - Soluble protein with high angiogenic activity.
+- source_sentence: FLEXIBLE CAPACITIVE KEYBOARD FOR TEXTILE USE.
   sentences:
+  - 'GreenValve III: System for spherical segment valves.'
+  - FLEXIBLE CAPACITIVE KEYBOARD FOR TEXTILE USE.
+  - Mandibular advancement device.
 pipeline_tag: sentence-similarity
 library_name: sentence-transformers
 ---
 ### Model Description
 - **Model Type:** Sentence Transformer
 - **Base model:** [AI-Growth-Lab/PatentSBERTa](https://huggingface.co/AI-Growth-Lab/PatentSBERTa) <!-- at revision 3ff1d553c861d8f5bfd902333d97fc95eb6b4c8f -->
+- **Maximum Sequence Length:** 384 tokens
 - **Output Dimensionality:** 768 dimensions
 - **Similarity Function:** Cosine Similarity
 <!-- - **Training Dataset:** Unknown -->
 ```
 SentenceTransformer(
+  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False, 'architecture': 'MPNetModel'})
   (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
 )
 ```
 model = SentenceTransformer("sentence_transformers_model_id")
 # Run inference
 sentences = [
+    'FLEXIBLE CAPACITIVE KEYBOARD FOR TEXTILE USE.',
+    'FLEXIBLE CAPACITIVE KEYBOARD FOR TEXTILE USE.',
+    'GreenValve III: System for spherical segment valves.',
 ]
 embeddings = model.encode(sentences)
 print(embeddings.shape)
 # Get the similarity scores for the embeddings
 similarities = model.similarity(embeddings, embeddings)
 print(similarities)
+# tensor([[1.0000, 1.0000, 0.3686],
+#         [1.0000, 1.0000, 0.3686],
+#         [0.3686, 0.3686, 1.0000]])
 ```
 <!--
 #### Unnamed Dataset
+* Size: 7,861 training samples
 * Columns: <code>sentence_0</code> and <code>sentence_1</code>
 * Approximate statistics based on the first 1000 samples:
+  |         | sentence_0                                                                        | sentence_1                                                                        |
+  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                            |
+  | details | <ul><li>min: 4 tokens</li><li>mean: 12.32 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 8 tokens</li><li>mean: 12.01 tokens</li><li>max: 36 tokens</li></ul> |
 * Samples:
+  | sentence_0                                                                                                    | sentence_1                                                      |
+  |:--------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------|
+  | <code>Genetic diagnostic technique for SCA1-3,6,7</code>                                                      | <code>Applicant/Organization: UNIV DEGLI STUDI DI TORINO</code> |
+  | <code>Method for detecting Macrophomina phaseolina</code>                                                     | <code>Technical Classification: C12Q</code>                     |
+  | <code>System for controlled administration of a substance from a human -body-implanted infusion device</code> | <code>Applicant/Organization: Stefanini Cesare</code>           |
 * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
   ```json
   {
 ### Training Hyperparameters
 #### Non-Default Hyperparameters
+- `per_device_train_batch_size`: 12
+- `per_device_eval_batch_size`: 12
 - `multi_dataset_batch_sampler`: round_robin
 #### All Hyperparameters
 <details><summary>Click to expand</summary>
+- `per_device_train_batch_size`: 12
 - `num_train_epochs`: 3
 - `max_steps`: -1
 - `learning_rate`: 5e-05
 - `project`: huggingface
 - `trackio_space_id`: trackio
 - `eval_strategy`: no
+- `per_device_eval_batch_size`: 12
 - `prediction_loss_only`: True
 - `eval_on_start`: False
 - `eval_do_concat_batches`: True
 ### Training Logs
 | Epoch  | Step | Training Loss |
 |:------:|:----:|:-------------:|
+| 1.5152 | 500  | 0.0002        |
+| 3.0303 | 1000 | 0.0000        |
+| 4.5455 | 1500 | 0.0000        |
+| 0.7622 | 500  | 2.3552        |
+| 1.5244 | 1000 | 1.9437        |
+| 2.2866 | 1500 | 1.8059        |
 ### Framework Versions

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3c205fd35ee2f54179664085aee58168bd124d632f7a65423e0ffbf59b3342cb
 size 437967648

 version https://git-lfs.github.com/spec/v1
+oid sha256:7124e7a341b5a27bb2703ada4533fd5cbeaab50189f7313fa499b4f2c132263f
 size 437967648

sentence_bert_config.json CHANGED Viewed

@@ -1,4 +1,4 @@
 {
-    "max_seq_length": 512,
     "do_lower_case": false
 }

 {
+    "max_seq_length": 384,
     "do_lower_case": false
 }

tokenizer.json CHANGED Viewed

@@ -2,7 +2,7 @@
   "version": "1.0",
   "truncation": {
     "direction": "Right",
-    "max_length": 512,
     "strategy": "LongestFirst",
     "stride": 0
   },

   "version": "1.0",
   "truncation": {
     "direction": "Right",
+    "max_length": 384,
     "strategy": "LongestFirst",
     "stride": 0
   },