brimtown/Groupchat-Qwen2.5-0.5B-MLC

Groupchat Model - Qwen2.5 0.5B Fine-tuned

A fine-tuned Qwen2.5-0.5B model trained to mimic individual users in a group chat, converted to MLC format for browser deployment via WebLLM.

Model Details

• Base Model: Qwen2.5-0.5B
• Fine-tuning Method: Unsloth
• Quantization: q4f16_1 (4-bit weights, float16 activations)
• Format: MLC-LLM for WebLLM deployment

Dataset

• Source: Private GroupMe group chat
• Size: ~60,000 messages
• Time Period: 2013-2016
• Participants: 10 users with varying activity levels

Prompt Format

The model was fine-tuned using the ChatML format with the following structure:

You are mimicking users in a group chat. Given the conversation history, respond as the specified user.

Recent messages:
[Speaker1]: [Message1]
[Speaker2]: [Message2]
...

Respond as [TargetSpeaker]:

Example Usage

const prompt = `You are mimicking users in a group chat. Given the conversation history, respond as the specified user.

Recent messages:
Jackson: what's in a bento box
Jackson: That's like a pupu platter right
Nick: What's a pupu platter

Respond as Spencer:`;

Recommended Generation Parameters

For best results, use these settings:

{
  temperature: 0.8,
  top_p: 0.9,
  max_tokens: 32-64,
}

WebLLM Integration

This model is optimized for browser deployment using WebLLM.

Loading in WebLLM

import * as webllm from "@mlc-ai/web-llm";

const appConfig = webllm.prebuiltAppConfig;

// Borrow model_lib from native Qwen2.5-0.5B
const qwenLib = appConfig.model_list.find(m => m.model_id === "Qwen2.5-0.5B-Instruct-q4f16_1-MLC");

appConfig.model_list.push({
    ...qwenLib,
    model_id: "Groupchat-Qwen2.5-0.5B-q4f16_1",
    model: "https://huggingface.co/brimtown/Groupchat-Qwen2.5-0.5B-MLC",
    low_resource_required: true,
    overrides: {
        context_window_size: 512
    }
});

const engine = await webllm.CreateMLCEngine("Groupchat-Qwen2.5-0.5B-q4f16_1", {
    appConfig: appConfig
});

License

This model is released under the same license as the base Qwen2.5 model. The training data is private and not included.

Acknowledgments

• Fine-tuned using Unsloth
• Converted to MLC format using MLC-LLM
• Deployed with WebLLM