---
license: mit
datasets:
- yiyic/oscar_arb_Arab_train
- yiyic/oscar_arb_Arab_test
- SaiedAlshahrani/Arabic_Wikipedia_20230101_bots
- ClusterlabAi/101_billion_arabic_words_dataset
language:
- ar
metrics:
- f1
- exact_match
base_model:
- answerdotai/ModernBERT-base
tags:
- Embedding
- Arabic
- Sentiment_Analysis
- QA
- NER
---
# Model Card: ModernAraBERT

## Summary
- Arabic encoder adapted from `answerdotai/ModernBERT-base` via continued pretraining on Arabic corpora (~9.8GB).
- Strong results across SA, NER (Macro-F1), and QA EM vs. AraBERT/mBERT/MARBERT.
- License: MIT · Paper: LREC 2026 · Hub: gizadatateam/ModernAraBERT

## Intended Uses
- Masked LM, feature extraction, and transfer learning for Arabic tasks.
- Downstream: sentiment analysis, NER, extractive QA, general classification/labeling.

## How to use
```python
from transformers import AutoTokenizer, AutoModelForMaskedLM
name = "gizadatateam/ModernAraBERT"
model = AutoModelForMaskedLM.from_pretrained(name)
tokenizer = AutoTokenizer.from_pretrained(name)
```

## Training data and recipe (brief)
- Corpora: OSIAN, Arabic Billion Words, Arabic Wikipedia, OSCAR Arabic
- Tokenizer: ModernBERT vocab + 80K Arabic tokens
- Objective: MLM (3 epochs; 128→512 seq len)
- Hardware: A100 40GB; framework: PyTorch + Transformers + Accelerate

## Evaluation (from paper)

### Sentiment Analysis — Macro-F1 (%)
| Model             | LABR      | HARD      | AJGT      |
| ----------------- | --------- | --------- | --------- |
| AraBERTv1         | 45.35     | 72.65     | 58.01     |
| AraBERTv2         | 45.79     | 67.10     | 53.59     |
| mBERT             | 44.18     | 71.70     | 61.55     |
| MARBERT           | 45.54     | 67.39     | 60.63     |
| **ModernAraBERT** | **56.45** | **89.37** | **70.54** |

### NER — Macro-F1 (%)
| Model             | Macro-F1  |
| ----------------- | --------- |
| AraBERTv1         | 13.46     |
| AraBERTv2         | 16.77     |
| mBERT             | 12.15     |
| MARBERT           | 7.42      |
| **ModernAraBERT** | **28.23** |

### QA (ARCD test) — EM (%)
| Model             | EM        |
| ----------------- | --------- |
| AraBERT           | 25.36     |
| AraBERTv2         | 26.08     |
| mBERT             | 25.12     |
| MARBERT           | 23.58     |
| **ModernAraBERT** | **27.10** |

## Citation
```bibtex
@inproceedings{<paper_id>,
  title={Efficient Adaptation of English Language Models for Low-Resource and Morphologically Rich Languages: The Case of Arabic},
  author={Maher, Eldamaty, Ashraf, ElShawi, Mostafa},
  booktitle={Proceedings of <conference_name>},
  year={2025},
  organization={<conference_name>}
}
```