./nugetz
/
trending/tags

#tokenizers

10 packages tagged with “tokenizers”

pkg

Tokenizers.DotNet

.NET wrapper of HuggingFace Tokenizers library

v1.4.0↙ 28.2K
tokenizersrusthuggingfacetokenizergpt
pkg

Tokenizers.DotNet.runtime.win-x64

Native(Rust) wrapper of HuggingFace Tokenizers library.

v1.4.0↙ 20.1K
tokenizersrusthuggingfacetokenizergpt
pkg

Tokenizers.DotNet.runtime.linux-x64

Native(Rust) wrapper of HuggingFace Tokenizers library.

v1.4.0↙ 10.5K
tokenizersrusthuggingfacetokenizergpt
pkg

Tokenizers.DotNet.runtime.linux-arm64

Native(Rust) wrapper of HuggingFace Tokenizers library.

v1.4.0↙ 3.1K
tokenizersrusthuggingfacetokenizergpt
pkg

Tokenizers.DotNet.runtime.win

Native(Rust) wrapper of HuggingFace Tokenizers library.

v1.0.5↙ 2.6K
tokenizersrusthuggingfacetokenizergpt
pkg

Tokenizers.DotNet.runtime.win-arm64

Native(Rust) wrapper of HuggingFace Tokenizers library.

v1.4.0↙ 2.7K
tokenizersrusthuggingfacetokenizergpt
pkg

Logos.HfTokenizers✓

C# wrapper for https://github.com/huggingface/tokenizers/tree/main/tokenizers

v0.1.0↙ 1.2K
huggingfacetokenizers

Otterkit.Tokenizers✓

This package contains the Standard COBOL tokenizer for the Otterkit compiler.

v1.0.70↙ 686
OtterkitCOBOLCompilerTokenizers
pkg

Tokenizers.DotNet.runtime.osx-arm64

Native(Rust) wrapper of HuggingFace Tokenizers library.

v1.4.0↙ 76
tokenizersrusthuggingfacetokenizergpt
pkg

Tokenizers.DotNet.runtime.osx-x64

Native(Rust) wrapper of HuggingFace Tokenizers library.

v1.4.0↙ 51
tokenizersrusthuggingfacetokenizergpt