End of training

README.md

@@ -1,39 +1,18 @@
 ---
-license: other
+license: apache-2.0
+base_model: meta-math/MetaMath-Mistral-7B
 tags:
-- math
-- alpaca
-- synthetic data
-- instruct
 - axolotl
-- finetune
-- gpt4
-datasets:
-- TIGER-Lab/MathInstruct
-- microsoft/orca-math-word-problems-200k
-language:
-- en
-base_model: meta-math/MetaMath-Mistral-7B
+- generated_from_trainer
+model-index:
+- name: EulerMath-Mistral-7B
+  results: []
 ---
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/6468ce47e134d050a58aa89c/jsw9mC64I69A_KwX0c6oi.png)
-
-<center><h1>📝 Note 📝</h1></center>
-
-📢 This model is currently in 1 epoch and this is a pre release. Main release will be available in 12 hours. 
-
--------------
-
-# 🔢 Einstein-v6-7B
 
-This model is a full fine-tuned version of [meta-math/MetaMath-Mistral-7B](meta-math/MetaMath-Mistral-7B) on the following datasets:
-
-- 🧮 [TIGER-Lab/MathInstruct](https://huggingface.co/datasets/TIGER-Lab/MathInstruct)
-- 📐 [microsoft/orca-math-word-problems-200k](https://huggingface.co/datasets/microsoft/orca-math-word-problems-200k) 
-
-This model is finetuned using `8xRTX3090` + `1xRTXA6000` using [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl).
-
-This model's training was sponsored by [sablo.ai](https://sablo.ai). 
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
 
+[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
 <details><summary>See axolotl config</summary>
 
 axolotl version: `0.4.0`
@@ -113,67 +92,66 @@ special_tokens:
   bos_token: "<s>"
   eos_token: "</s>"
   unk_token: "<unk>"
-```
-
-</details><br>
-
-# 💬 Prompt Template
-
-You can use this prompt template while using the model:
-
-### Alpaca
-
-```
-Below is an instruction that describes a task. Write a response that appropriately completes the request.
-
-### Instruction:
-{instruction}
-
-### Response:
 
 ```
 
-This prompt template is available as a [chat template](https://huggingface.co/docs/transformers/main/chat_templating), which means you can format messages using the
-`tokenizer.apply_chat_template()` method:
-
-```python
-messages = [
-    {"role": "system", "content": "You are helpful AI asistant."},
-    {"role": "user", "content": "Hello!"}
-]
-gen_input = tokenizer.apply_chat_template(message, return_tensors="pt")
-model.generate(**gen_input)
-```
-
-# 🔄 Quantizationed versions
+</details><br>
 
-Quantizationed versions of this model is currently not available. It will be available soon :)
+# EulerMath-Mistral-7B
 
-# 🎯 [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
+This model is a fine-tuned version of [meta-math/MetaMath-Mistral-7B](https://huggingface.co/meta-math/MetaMath-Mistral-7B) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.1956
 
-# 🤖 Additional information about training
+## Model description
 
-This model is full fine-tuned for 2 epoch. 
+More information needed
 
-Total number of steps was 544.
+## Intended uses & limitations
 
-<details><summary>Loss graph</summary>
+More information needed
 
+## Training and evaluation data
 
-</details><br>
+More information needed
 
-# 🤝 Acknowledgments
+## Training procedure
 
-Thanks to [sablo.ai](https://sablo.ai) for sponsoring this model.
+### Training hyperparameters
 
-Thanks to all the dataset authors mentioned in the datasets section.
+The following hyperparameters were used during training:
+- learning_rate: 5e-06
+- train_batch_size: 2
+- eval_batch_size: 2
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 9
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 72
+- total_eval_batch_size: 18
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 10
+- num_epochs: 2
 
-Thanks to [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) for making the repository I used to make this model.
+### Training results
 
-Thanks to all open source AI community.
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 0.707         | 0.0   | 1    | 0.9061          |
+| 0.3011        | 0.25  | 68   | 0.3263          |
+| 0.2585        | 0.5   | 136  | 0.2836          |
+| 0.2352        | 0.75  | 204  | 0.2544          |
+| 0.2192        | 1.0   | 272  | 0.2268          |
+| 0.1527        | 1.23  | 340  | 0.2144          |
+| 0.1452        | 1.48  | 408  | 0.2032          |
+| 0.144         | 1.73  | 476  | 0.1970          |
+| 0.1441        | 1.98  | 544  | 0.1956          |
 
-[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
 
-If you would like to support me:
+### Framework versions
 
-[☕ Buy Me a Coffee](https://www.buymeacoffee.com/weyaxi)
+- Transformers 4.38.2
+- Pytorch 2.1.2+cu118
+- Datasets 2.18.0
+- Tokenizers 0.15.0

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "do_sample": true,
+  "eos_token_id": 2,
+  "transformers_version": "4.38.2"
+}

model-00001-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fb7ddd132c950151879ee704033773a1c08f22fedfbe2459a71cf1304378ddad
+oid sha256:d3e6645954961b8991f249065609b6491bf175453e49211f0ca8ee2fbf8ffeb7
 size 4943170528

model-00002-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:254fae62a9850c1250d558ce0c0a152cbf3843311738cf4ef96d0b9eb71c8ba0
+oid sha256:445c2dd56bda6dbe8914dcc5f16947ac46290e9d906f8566f9c0867481212964
 size 4999819336

model-00003-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e5b4497b7b6358ed1de5f189caf947738698ebcf00c3dec230c973c0552e5d86
+oid sha256:be5900b554d420f18e739a39543dc322439881329fbd19177f398f008c1e3a31
 size 4540524536

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e322fb0a41c22afa338151a84fd9ec7c850cb8bbaf07519a6d94ef22b0f3b433
+size 539576