End of training

README.md

@@ -19,9 +19,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.5893
-- F1: 0.7943
-- Accuracy: 0.7939
+- Loss: 0.9380
+- F1: 0.8584
+- Accuracy: 0.8571
 
 ## Model description
 
@@ -40,23 +40,22 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 2e-05
-- train_batch_size: 64
+- learning_rate: 8e-05
+- train_batch_size: 32
 - eval_batch_size: 32
 - seed: 42
-- optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- optimizer: Use adamw_torch with betas=(0.9,0.98) and epsilon=1e-06 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
-- num_epochs: 5
+- num_epochs: 4
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss | F1     | Accuracy |
 |:-------------:|:-----:|:----:|:---------------:|:------:|:--------:|
-| No log        | 1.0   | 69   | 1.0553          | 0.6382 | 0.6449   |
-| 1.3836        | 2.0   | 138  | 0.9276          | 0.6992 | 0.7061   |
-| 0.6653        | 3.0   | 207  | 0.6671          | 0.7802 | 0.7816   |
-| 0.6653        | 4.0   | 276  | 0.6117          | 0.8122 | 0.8122   |
-| 0.3915        | 5.0   | 345  | 0.5893          | 0.7943 | 0.7939   |
+| 1.2496        | 1.0   | 146  | 0.8292          | 0.7104 | 0.7061   |
+| 0.4307        | 2.0   | 292  | 0.6250          | 0.8275 | 0.8245   |
+| 0.1552        | 3.0   | 438  | 0.6152          | 0.8503 | 0.8490   |
+| 0.0236        | 4.0   | 584  | 0.9380          | 0.8584 | 0.8571   |
 
 
 ### Framework versions

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1ab235771a6861404c88c0a671990646c11dbcc3f7de9296532c015e6e624476
+oid sha256:a4c073eb7314341909963d3f18ad27590dace8da090318d6b74bf7f238d4d515
 size 598455164

runs/Jan23_01-30-42_ultramarine/events.out.tfevents.1737585042.ultramarine.3365544.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dd390ddeb13616340691c7e0ee912dab805affb94b37d3a1692e17371dc7b769
-size 7666
+oid sha256:824561eceb73e9dae64ed9d51a8ffb57212a32468ba0a66f940a000ed4ad1807
+size 8600

tokenizer_config.json

@@ -937,7 +937,7 @@
     "input_ids",
     "attention_mask"
   ],
-  "model_max_length": 512,
+  "model_max_length": 8192,
   "pad_token": "[PAD]",
   "sep_token": "[SEP]",
   "tokenizer_class": "PreTrainedTokenizerFast",