Update README.md

README.md

@@ -11,112 +11,141 @@ license: apache-2.0
 short_description: Place for AI Models
 ---
 
-Model Card: LLM Brain Rot Demonstration
-Model/Dataset Name
+# Model Card: LLM Brain Rot Demonstration
+
+## Model/Dataset Name
 LLM Brain Rot Demonstration: Qwen2.5 0.5B Comparison
-1. Overview
+
+## Overview
 This demonstration showcases the "Brain Rot" effect in Large Language Models (LLMs) as described in the research paper "LLMs Can Get Brain Rot" by Xing et al. (2024). The demonstration compares two Qwen2.5 0.5B Instruct models: one trained on control data and one trained on 100% M1 junk data, illustrating how exposure to low-quality web content can degrade an LLM's cognitive capabilities.
+
 The original research was conducted by a team from Texas A&M University, University of Texas at Austin, and Purdue University. This demonstration is a simplified implementation of their findings, focusing on the most extreme case (100% junk data) to clearly illustrate the phenomenon.
-2. Intended Use
-Primary Tasks
-⦁	Educational demonstration of data quality effects on LLM performance
-⦁	Comparison of reasoning capabilities between models trained on different data quality
-⦁	Illustration of "thought-skipping" phenomenon in LLMs
-Intended Users
-⦁	Students learning about LLM training and data quality
-⦁	Researchers studying model robustness and data effects
-⦁	Educators demonstrating AI concepts
-⦁	Anyone interested in understanding how training data affects model behavior
-Inappropriate Uses
-⦁	Production deployment or real-world applications
-⦁	Making generalized claims about all LLMs based on this limited comparison
-⦁	Evaluating the overall quality of the base Qwen2.5 model family
-⦁	Drawing conclusions about the effects of content beyond what is demonstrated
-3. Dataset/Model Details
-Models
-⦁	Base Model: Qwen2.5 0.5B Instruct
-⦁	Comparison Models:
-1.	Qwen2.5 0.5B trained on control data (0% junk)
-2.	Qwen2.5 0.5B trained on 100% M1 junk data
-Dataset
-⦁	ARC Challenge questions (small sample from main repository)
-⦁	Safety questions (small sample from main repository)
-⦁	RULER (3 custom sets based on RULER repository sub tests; Needle in Haystack, Variable Tracking and Question Answering)
-⦁	TRAIT (custom set based on original TRAIT repository)
-Model Variants and Datasets in Original Research
+
+## Intended Use
+
+### Primary Tasks
+- Educational demonstration of data quality effects on LLM performance
+- Comparison of reasoning capabilities between models trained on different data quality
+- Illustration of "thought-skipping" phenomenon in LLMs
+
+### Intended Users
+- Students learning about LLM training and data quality
+- Researchers studying model robustness and data effects
+- Educators demonstrating AI concepts
+- Anyone interested in understanding how training data affects model behavior
+
+### Inappropriate Uses
+- Production deployment or real-world applications
+- Making generalized claims about all LLMs based on this limited comparison
+- Evaluating the overall quality of the base Qwen2.5 model family
+- Drawing conclusions about the effects of content beyond what is demonstrated
+
+## Dataset/Model Details
+
+### Models
+- Base Model: Qwen2.5 0.5B Instruct
+- Comparison Models:
+  - Qwen2.5 0.5B trained on control data (0% junk)
+  - Qwen2.5 0.5B trained on 100% M1 junk data
+
+### Dataset
+- ARC Challenge questions (small sample from main repository)
+- Safety questions (small sample from main repository)
+- RULER (3 custom sets based on RULER repository sub tests; Needle in Haystack, Variable Tracking and Question Answering)
+- TRAIT (custom set based on original TRAIT repository)
+
+### Model Variants and Datasets in Original Research
 The original research included 40 model variants:
-⦁	4 base models: Llama3 8B, Qwen2.5 7B, Qwen2.5 0.5B, Qwen3 4B
-⦁	2 junk metrics: M1 (engagement degree) and M2 (semantic quality)
-⦁	5 training ratios: 0%, 20%, 50%, 80%, 100% junk data
-⦁	4 base models x 5 training ratios x 2 junk metrics = 40 total model variants
+- 4 base models: Llama3 8B, Qwen2.5 7B, Qwen2.5 0.5B, Qwen3 4B
+- 2 junk metrics: M1 (engagement degree) and M2 (semantic quality)
+- 5 training ratios: 0%, 20%, 50%, 80%, 100% junk data
+- 4 base models x 5 training ratios x 2 junk metrics = 40 total model variants
+
 The original Dataset was:
-⦁	Source: Twitter/X posts from 2010
-⦁	Filtering: M1 metric (engagement degree) - short but highly popular posts
-⦁	Processing: Control data consists of longer, less popular posts
-⦁	Language: Primarily English
-4. Ethical Considerations
-Possible Biases
-⦁	The Twitter dataset may contain demographic, cultural, and ideological biases present on the platform
-⦁	The M1 metric (based on popularity) may amplify content that is attention-grabbing rather than accurate or thoughtful
-⦁	The models may reproduce stereotypes or problematic content present in the training data
-Risks of Misuse
-⦁	The junk-trained model may generate lower-quality, less reliable, or potentially problematic responses
-⦁	Users might overgeneralize from this specific demonstration to make broader claims about LLMs
-⦁	The demonstration might be misinterpreted as a definitive statement about all social media content
-Privacy/Consent Issues
-⦁	The models were trained on public Twitter posts, but individual tweets may contain personal information
-⦁	Users should be cautious about inputting personal information into either model
-5. Limitations
-Scope Limitations
-⦁	Only demonstrates the effect with one model family (Qwen2.5) and size (0.5B)
-⦁	Only shows the comparison between 0% and 100% junk data, not the "dose-response" relationship
-⦁	Only demonstrates M1 metric effects, not M2 (semantic quality)
-⦁	Only evaluates a limited number of examples per task type for demonstration purposes
-Technical Limitations
-⦁	The smaller model size (0.5B) may show more pronounced effects than larger models
-⦁	The demonstration focuses on reasoning tasks, but the original paper found effects across multiple capabilities
-⦁	The interface may not fully capture all nuances of the "thought-skipping" phenomenon
-Generalizability
-⦁	Results may not apply to all LLM architectures or training methodologies
-⦁	The specific Twitter dataset from 2010 may not represent current web content
-⦁	The demonstration shows correlation, not necessarily causation for all scenarios
-6. Training & Evaluation
-Training Process
+- Source: Twitter/X posts from 2010
+- Filtering: M1 metric (engagement degree) - short but highly popular posts
+- Processing: Control data consists of longer, less popular posts
+- Language: Primarily English
+
+## Ethical Considerations
+
+### Possible Biases
+- The Twitter dataset may contain demographic, cultural, and ideological biases present on the platform
+- The M1 metric (based on popularity) may amplify content that is attention-grabbing rather than accurate or thoughtful
+- The models may reproduce stereotypes or problematic content present in the training data
+
+### Risks of Misuse
+- The junk-trained model may generate lower-quality, less reliable, or potentially problematic responses
+- Users might overgeneralize from this specific demonstration to make broader claims about LLMs
+- The demonstration might be misinterpreted as a definitive statement about all social media content
+
+### Privacy/Consent Issues
+- The models were trained on public Twitter posts, but individual tweets may contain personal information
+- Users should be cautious about inputting personal information into either model
+
+## Limitations
+
+### Scope Limitations
+- Only demonstrates the effect with one model family (Qwen2.5) and size (0.5B)
+- Only shows the comparison between 0% and 100% junk data, not the "dose-response" relationship
+- Only demonstrates M1 metric effects, not M2 (semantic quality)
+- Only evaluates a limited number of examples per task type for demonstration purposes
+
+### Technical Limitations
+- The smaller model size (0.5B) may show more pronounced effects than larger models
+- The demonstration focuses on reasoning tasks, but the original paper found effects across multiple capabilities
+- The interface may not fully capture all nuances of the "thought-skipping" phenomenon
+
+### Generalizability
+- Results may not apply to all LLM architectures or training methodologies
+- The specific Twitter dataset from 2010 may not represent current web content
+- The demonstration shows correlation, not necessarily causation for all scenarios
+
+## Training & Evaluation
+
+### Training Process
 The original models were trained using the following process:
-1.	Base models (Qwen2.5 0.5B Instruct) underwent continual pre-training
-2.	Training parameters: learning rate 1×10^-5, AdamW optimizer, 3 epochs
-3.	Models were trained on either control data or 100% M1 junk data
-4.	After pre-training, models underwent instruction tuning on the Alpaca English dataset
-Evaluation Metrics
+- Base models (Qwen2.5 0.5B Instruct) underwent continual pre-training
+- Training parameters: learning rate 1×10^-5, AdamW optimizer, 3 epochs
+- Models were trained on either control data or 100% M1 junk data
+- After pre-training, models underwent instruction tuning on the Alpaca English dataset
+
+### Evaluation Metrics
 The original research evaluated models on multiple benchmarks:
-⦁	ARC Challenge: Chain-of-thought prompting with accuracy measurement
-⦁	RULER: Sample tasks representing needle-in-haystack, variable tracking, and question answering
-⦁	TRAIT: Sample personality questions with simplified analysis
-⦁	Safety: Subset of harmful behaviors with refusal detection
-⦁	Thought-skipping analysis: Heuristic-based categorization of reasoning failures
-Key Results from Original Research
+- ARC Challenge: Chain-of-thought prompting with accuracy measurement
+- RULER: Sample tasks representing needle-in-haystack, variable tracking, and question answering
+- TRAIT: Sample personality questions with simplified analysis
+- Safety: Subset of harmful behaviors with refusal detection
+- Thought-skipping analysis: Heuristic-based categorization of reasoning failures
+
+### Key Results from Original Research
 For Qwen2.5 0.5B with M1 intervention:
-⦁	ARC Challenge (COT): 74.9 → 57.2 (17.7 point drop)
-⦁	RULER Overall: 93.9 → 71.0 (22.9 point drop)
-⦁	Safety metrics showed increased risk scores
-⦁	Personality traits showed increases in narcissism and psychopathy
-Analysis of Failures
+- ARC Challenge (COT): 74.9 → 57.2 (17.7 point drop)
+- RULER Overall: 93.9 → 71.0 (22.9 point drop)
+- Safety metrics showed increased risk scores
+- Personality traits showed increases in narcissism and psychopathy
+
+### Analysis of Failures
 The primary failure mode identified was "thought-skipping," where models:
-⦁	Skip intermediate reasoning steps
-⦁	Provide answers without showing their thinking process
-⦁	Make logical leaps or factual errors in their reasoning
-7. References
-Primary Research
-⦁	Xing, S., Hong, J., Wang, Y., Chen, R., Zhang, Z., Grama, A., Tu, Z., & Wang, Z. (2024). LLMs Can Get Brain Rot! arXiv preprint arXiv:2510.13928.
-Resources
-⦁	GitHub Repository: https://github.com/llm-brain-rot/llm-brain-rot
-⦁	Project Website: https://llm-brain-rot.github.io/
-⦁	Hugging Face Models:
-1.	Qwen2.5 0.5B trained on control data (0% junk): https://huggingface.co/AmberYifan/qwen2.5-0.5b-instruct-full-pretrain-control-tweet-1m-en-sft
-2.	Qwen2.5 0.5B trained on 100% M1 junk data: https://huggingface.co/AmberYifan/qwen2.5-0.5b-instruct-full-pretrain-junk-tweet-1m-en-sft
-Related Work
-⦁	Qi, X., Zeng, Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., & Henderson, P. (2023). Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693.
-⦁	Shumailov, I., Shumailov, I., Shumailova, Z., Papernot, N., Anderson, A., & Gal, Y. (2023). The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2305.17493.
-⦁	Seddik, M. E., Shumailov, I., Shumailova, Z., & Gal, Y. (2024). How bad is training on synthetic data? A statistical analysis of language model collapse. arXiv preprint arXiv:2404.05094.
+- Skip intermediate reasoning steps
+- Provide answers without showing their thinking process
+- Make logical leaps or factual errors in their reasoning
+
+## References
+
+### Primary Research
+- Xing, S., Hong, J., Wang, Y., Chen, R., Zhang, Z., Grama, A., Tu, Z., & Wang, Z. (2024). LLMs Can Get Brain Rot! arXiv preprint arXiv:2510.13928.
+
+### Resources
+- GitHub Repository: https://github.com/llm-brain-rot/llm-brain-rot
+- Project Website: https://llm-brain-rot.github.io/
+- Hugging Face Models:
+  - Qwen2.5 0.5B trained on control data (0% junk): https://huggingface.co/AmberYifan/qwen2.5-0.5b-instruct-full-pretrain-control-tweet-1m-en-sft
+  - Qwen2.5 0.5B trained on 100% M1 junk data: https://huggingface.co/AmberYifan/qwen2.5-0.5b-instruct-full-pretrain-junk-tweet-1m-en-sft
+
+### Related Work
+- Qi, X., Zeng, Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., & Henderson, P. (2023). Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693.
+- Shumailov, I., Shumailov, I., Shumailova, Z., Papernot, N., Anderson, A., & Gal, Y. (2023). The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2305.17493.
+- Seddik, M. E., Shumailov, I., Shumailova, Z., & Gal, Y. (2024). How bad is training on synthetic data? A statistical analysis of language model collapse. arXiv preprint arXiv:2404.05094.
 
 Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference