EXASPO-3.5-7.8B-Instruct

Introduction

EXASPO-3.5-7.8B-Instruct is a language model specifically optimized for the Korean spoken(colloquial) language.

EXASPO-3.5-7.8B-Instruct is based on the LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct and has undergone continual pre-training and instruction tuning using a Korean spoken-language dataset.

You can find the details of the base model here.

This repository contains the instruction-tuned 7.8B language model with the following features:

• Number of Parameters (without embeddings): 6.98B
• Number of Layers: 32
• Number of Attention Heads: GQA with 32 Q-heads and 8 KV-heads
• Vocab Size: 102,400
• Context Length: 32,768 tokens

Quickstart

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "JunHaHwang/EXASPO-3.5-7.8B-Instruct"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct")

prompt = "최근 겪은 일 중 재밌는 썰좀 풀어줘"

messages = [
    {"role": "system", 
     "content": "You are a kind and helpful assistant."},
    {"role": "user", "content": prompt}
]
input_ids = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
)


output = model.generate(
    input_ids.to("cuda"),
    eos_token_id=tokenizer.eos_token_id,
        max_new_tokens=512,
        temperature=0.7,
        repetition_penalty =1.2
)
print(tokenizer.decode(output[0]))