Pure-C# Hunyuan translation, now on your GPU. No llama.cpp — reference one NuGet package and the weights come with it.
dotnet add package Sdcb.HyMT2Sharp
dotnet add package Sdcb.GgufWeights.Hy-MT2-1.8B-2BitLine two is the 2Bit weights (~573 MB). Passing a GGUF file path still works.
Translation race
Each lane eats 83 prompt tokens at its prefill speed, then emits 60 tokens at its decode speed. Speeds are measured on a Ryzen 7 5800X + RTX 3080 Ti (prefill 512 / decode 128) — the model is not running in your browser.
SimdPaddleOCR is officially released today (NuGet: Sdcb.SimdPaddleOCR). It is a complete OCR inference engine written entirely in C#. It does not depend on Paddle Inference or ONNX Runtime, and it does not require shipping OpenCV native libraries.
prefilldecodeSwitch to Q4_K_M+ for a head-to-head — we still lose GPU decode.
A Tencent-specific format: llama.cpp loads it on neither CPU nor GPU, so HyMT2Sharp Vulkan is the only GPU path today.
dotnet add package Sdcb.HyMT2Sharp
dotnet add package Sdcb.GgufWeights.Hy-MT2-1.8B-1.25Bitusing Sdcb.GgufWeights.Hy_MT2_1_8B_1_25Bit;
using Sdcb.HyMT2Sharp.Backends;
using Sdcb.HyMT2Sharp.Model;
// Metal → Vulkan → CPU
using HunyuanDenseModel model = new(Weights.Model(), backend: BackendFactory.Create());A Tencent-specific format: llama.cpp loads it on neither CPU nor GPU, so HyMT2Sharp Vulkan is the only GPU path today. Recommended: just over half the size of Q4_K_M, fastest decode.
dotnet add package Sdcb.HyMT2Sharp
dotnet add package Sdcb.GgufWeights.Hy-MT2-1.8B-2Bitusing Sdcb.GgufWeights.Hy_MT2_1_8B_2Bit;
using Sdcb.HyMT2Sharp.Backends;
using Sdcb.HyMT2Sharp.Model;
// Metal → Vulkan → CPU
using HunyuanDenseModel model = new(Weights.Model(), backend: BackendFactory.Create());vs llama.cpp: CUDA prefill 1.10×, decode 0.82×; CPU prefill 1.72×.
dotnet add package Sdcb.HyMT2Sharp
dotnet add package Sdcb.GgufWeights.Hy-MT2-1.8B-Q4_K_Musing Sdcb.GgufWeights.Hy_MT2_1_8B_Q4_K_M;
using Sdcb.HyMT2Sharp.Backends;
using Sdcb.HyMT2Sharp.Model;
// Metal → Vulkan → CPU
using HunyuanDenseModel model = new(Weights.Model(), backend: BackendFactory.Create());vs llama.cpp: CUDA prefill 1.18×, decode 0.94×; CPU prefill 2.90×.
dotnet add package Sdcb.HyMT2Sharp
dotnet add package Sdcb.GgufWeights.Hy-MT2-1.8B-Q6_Kusing Sdcb.GgufWeights.Hy_MT2_1_8B_Q6_K;
using Sdcb.HyMT2Sharp.Backends;
using Sdcb.HyMT2Sharp.Model;
// Metal → Vulkan → CPU
using HunyuanDenseModel model = new(Weights.Model(), backend: BackendFactory.Create());vs llama.cpp: CUDA prefill 1.04×, decode 0.90×; CPU prefill 2.45×.
dotnet add package Sdcb.HyMT2Sharp
dotnet add package Sdcb.GgufWeights.Hy-MT2-1.8B-Q8_0using Sdcb.GgufWeights.Hy_MT2_1_8B_Q8_0;
using Sdcb.HyMT2Sharp.Backends;
using Sdcb.HyMT2Sharp.Model;
// Metal → Vulkan → CPU
using HunyuanDenseModel model = new(Weights.Model(), backend: BackendFactory.Create());Backends
BackendFactory.Create() tries Metal → Vulkan → CPU; pass "cpu" / "vulkan" / "metal" or set HYMT_BACKEND to force one. CPU kernels then step down by instruction set.
Vulkan / Metal interop, scheduling and memory management are all C# — the repo is still 100% C#.
Intel Arc B580: 2Bit 11,039 / 243 tok/s; Q4_K_M 10,620 / 208 vs llama.cpp Vulkan 7,345 / 194 on the same card. AMD RDNA gets default tile tuning.
One fix worth its own section
The d / dmin scales in GGUF blocks are fp16, and in real weights they are almost all fp16 subnormals. The old fp16 → fp32 path produced an fp32 subnormal on the way, paying ~150 cycles of microcode assist per conversion.
Random synthetic weights almost never land in the subnormal range, so micro-benchmarks never showed it. Computing on the integer mantissa gives bit-identical results.
tok/s · longer is better
Samples
Engine on "Auto" picks Vulkan on a 3080 Ti: 60 decode tokens in 211 ms, about 284 tok/s.
OpenAI-compatible /v1/chat/completions with SSE streaming and a chat page.
The package ships hello-world, cli and minimal-api scripts — they show up in the Samples panel.
Two pitfalls
The 2Bit model likes to split one multi-byte character into several byte tokens. Decoding the first one alone yields half a UTF-8 sequence, so a � flashes in the stream even though the final text is right.
string next = model.Tokenizer
.DecodeVisible(generated)
.TrimEnd('\uFFFD');Dispose only called vkDestroyDevice, assuming it would reclaim everything the device owned. The NVIDIA driver does not — each reload leaked ~4.3 GB. Buffers, pipelines, shader modules, fences, descriptor and command pools are now tracked and freed one by one.
Upgrading from 1.0
The namespace is still Sdcb.HyMT2Sharp.Model; new HunyuanDenseModel(path) and Forward are unchanged, and omitting backend behaves exactly like 1.0.
# 1.x
dotnet remove package Sdcb.HyMT2Sharp.Gguf
dotnet remove package Sdcb.HyMT2Sharp.Kernels
dotnet remove package Sdcb.HyMT2Sharp.Model
# 2.0
dotnet add package Sdcb.HyMT2SharpKnown limitations
Open source under Apache-2.0 — a star is always appreciated. Chat with us in QQ group 495782587 (.NET骚操作).