LnL-AI
/

dbrx-base-tokenizer

Transformers.js

Model card Files Files and versions

dbrx-base-tokenizer / tokenizer.json

Qubitium's picture

add pad and extra_N special tokens

3a0d911 over 1 year ago

history blame contribute delete

4.24 MB

This file is stored with Xet . It is too big to display, but you can still download it.

Large File Pointer Details

( Raw pointer file )

SHA256:: a974ee932171faced419ceee9a4f8e7de782c4bbae3884a69bb5df59080e92e6
Pointer size:: 132 Bytes
Size of remote file:: 4.24 MB
Xet hash:: 391b067f4ef562e02b040b31049639a45c9a8e4293c7dcb62ca414114ea9248b

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.