Benchmarks
Test results of the LiqFit approach on various datasets using various models.
The models were trained and given different amounts of examples per single class. The F1 score is reported in the table below.
| Model & examples per label | Emotion | AgNews | SST5 |
|---|---|---|---|
Comprehend-it/0 | 56.60 | 79.82 | 37.9 |
Comprehend-it/8 | 63.38 | 85.9 | 46.67 |
Comprehend-it/64 | 80.7 | 88 | 47 |
SetFit/0 | 57.54 | 56.36 | 24.11 |
SetFit/8 | 56.81 | 64.93 | 33.61 |
SetFit/64 | 79.03 | 88 | 45.38 |
LiqFit used knowledgator/comprehend_it-base model, while for SetFit, we utilzed BAAI/bge-base-en-v1.5.
Last updated