End of training

README.md

@@ -1,11 +1,9 @@
 ---
 library_name: transformers
 license: apache-2.0
-base_model: t5-small
+base_model: t5-base
 tags:
 - generated_from_trainer
-metrics:
-- rouge
 model-index:
 - name: tmp_test
   results: []
@@ -16,14 +14,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 # tmp_test
 
-This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on the None dataset.
-It achieves the following results on the evaluation set:
-- Loss: 3.4968
-- Rouge1: 0.1438
-- Rouge2: 0.0337
-- Rougel: 0.1049
-- Rougelsum: 0.1052
-- Gen Len: 20.0
+This model is a fine-tuned version of [t5-base](https://huggingface.co/t5-base) on the None dataset.
 
 ## Model description
 
@@ -42,31 +33,16 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 1e-05
-- train_batch_size: 16
-- eval_batch_size: 16
+- learning_rate: 2e-05
+- train_batch_size: 1
+- eval_batch_size: 1
 - seed: 42
-- gradient_accumulation_steps: 2
-- total_train_batch_size: 32
 - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
-- num_epochs: 10
-- mixed_precision_training: Native AMP
+- training_steps: 2
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
-|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|
-| No log        | 1.0   | 25   | 4.3210          | 0.1303 | 0.0263 | 0.0973 | 0.0974    | 20.0    |
-| No log        | 2.0   | 50   | 3.8167          | 0.1344 | 0.0288 | 0.1005 | 0.1005    | 20.0    |
-| No log        | 3.0   | 75   | 3.6506          | 0.1386 | 0.0305 | 0.1023 | 0.1024    | 20.0    |
-| No log        | 4.0   | 100  | 3.5756          | 0.1426 | 0.0317 | 0.1044 | 0.1045    | 20.0    |
-| No log        | 5.0   | 125  | 3.5415          | 0.1433 | 0.0323 | 0.1044 | 0.1045    | 20.0    |
-| No log        | 6.0   | 150  | 3.5215          | 0.1425 | 0.0321 | 0.1039 | 0.104     | 20.0    |
-| No log        | 7.0   | 175  | 3.5101          | 0.1436 | 0.0325 | 0.1047 | 0.1048    | 20.0    |
-| 3.9618        | 8.0   | 200  | 3.5025          | 0.1435 | 0.0331 | 0.105  | 0.1051    | 20.0    |
-| 3.9618        | 9.0   | 225  | 3.4982          | 0.1438 | 0.0337 | 0.1049 | 0.1052    | 20.0    |
-| 3.9618        | 10.0  | 250  | 3.4968          | 0.1438 | 0.0337 | 0.1049 | 0.1052    | 20.0    |
 
 
 ### Framework versions

config.json

@@ -1,12 +1,12 @@
 {
-  "_name_or_path": "t5-small",
+  "_name_or_path": "t5-base",
   "architectures": [
     "T5ForConditionalGeneration"
   ],
   "classifier_dropout": 0.0,
-  "d_ff": 2048,
+  "d_ff": 3072,
   "d_kv": 64,
-  "d_model": 512,
+  "d_model": 768,
   "decoder_start_token_id": 0,
   "dense_act_fn": "relu",
   "dropout_rate": 0.1,
@@ -18,9 +18,9 @@
   "layer_norm_epsilon": 1e-06,
   "model_type": "t5",
   "n_positions": 512,
-  "num_decoder_layers": 6,
-  "num_heads": 8,
-  "num_layers": 6,
+  "num_decoder_layers": 12,
+  "num_heads": 12,
+  "num_layers": 12,
   "output_past": true,
   "pad_token_id": 0,
   "relative_attention_max_distance": 128,

generation_config.json

@@ -1,4 +1,5 @@
 {
+  "_from_model_config": true,
   "decoder_start_token_id": 0,
   "eos_token_id": 1,
   "pad_token_id": 0,

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:53d76fde505168d38d623c71361304830a91672cb31c9dd86ee6a55e341e126d
-size 242041896
+oid sha256:e6c61ba918d54f6e8236dd1d3959f028639a08df803ed8cfe096fe1e161f5b67
+size 891644712

runs/Mar02_16-22-29_eb24f8b67485/events.out.tfevents.1740932560.eb24f8b67485.1021.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c6b99cd70b41e7ed341fbada85871c1a45296ef728863887809a148c7b92d4b
+size 6016

runs/Mar02_17-13-56_eb24f8b67485/events.out.tfevents.1740935648.eb24f8b67485.7761.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57b74b1ac0d3c26630919ba35437b2fc624c8435fcb5bd71e7e1dbb3819fd44a
+size 6363

special_tokens_map.json

@@ -101,25 +101,7 @@
     "<extra_id_98>",
     "<extra_id_99>"
   ],
-  "eos_token": {
-    "content": "</s>",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "pad_token": {
-    "content": "<pad>",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "unk_token": {
-    "content": "<unk>",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  }
+  "eos_token": "</s>",
+  "pad_token": "<pad>",
+  "unk_token": "<unk>"
 }

tokenizer_config.json

@@ -928,11 +928,11 @@
     "<extra_id_98>",
     "<extra_id_99>"
   ],
-  "clean_up_tokenization_spaces": true,
+  "clean_up_tokenization_spaces": false,
   "eos_token": "</s>",
   "extra_ids": 100,
   "extra_special_tokens": {},
-  "model_max_length": 512,
+  "model_max_length": 1000000000000000019884624838656,
   "pad_token": "<pad>",
   "tokenizer_class": "T5Tokenizer",
   "unk_token": "<unk>"

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bd6474827b93beaa728259a765b19d4b5448d7a9ec2a1099e78cd17db25fa03b
-size 5496
+oid sha256:9cf672a9f77b2d035bec7412af2fa989a56b759cf903b61d32921a9ffb402039
+size 5432