HardenedVault AI LabHardenedVault AI 实验室

AI infrastructure, built from the metal up.

AI 基础设施,从硬件开始构建。

Most AI products are a thin layer over someone else's runtime. We wrote our own: the kernels, the intermediate representation, the scheduler and the serving stack, with the same discipline we bring to operating systems, compilers and security.

大多数 AI 产品只是在别人的运行时上加一层薄壳。我们自己写了整套:算子、中间表示、调度器与服务栈,用的是我们做操作系统、编译器和安全时的同一套工程纪律。

471.1tok/s

Code editing with speculative decoding

代码编辑,开启推测解码

1.16×

llama.cpp's fastest speculative mode (407.7 tok/s)

对比 llama.cpp 最快的推测模式(407.7 tok/s)

10/ 10

Turns bit-identical with speculation on and off (llama.cpp: 8 of 10)

推测解码开关输出逐位一致的轮次(llama.cpp:8 / 10)

8GB

A desktop RTX 3050. No datacenter GPU needed

一块桌面级 RTX 3050,无需数据中心 GPU

Flagship旗舰项目

SonicBoom: a native tensor runtime.

SonicBoom:原生张量运行时。

Written from the ground up in C++ and CUDA, exposed through a stable C ABI, with GNU Guile as its first-class language. No Python interpreter, pip or wheel tree is involved in inference. One shared library.

从零开始用 C++ 与 CUDA 编写,通过稳定的 C ABI 对外提供接口,以 GNU Guile 为一等语言。推理过程不涉及 Python 解释器、pip 或 wheel 依赖树。只有一个动态库。

FAST

Speed from engineering速度来自工程

Our own kernels and IR, and a warm-up that measures the best kernel choices on your own GPU when a model loads. Speculative decoding is on by default.

自研算子与中间表示,模型加载时在你自己的 GPU 上实测选出最优算子配置。推测解码默认开启。

EXACT

Reproducible by design设计上可复现

The same version on the same machine gives the same output, and speculation on or off never changes a token. Seeded sampling repeats exactly.

同一版本在同一台机器上输出相同,开关推测解码不会改变任何一个 token。带种子的采样可精确重现。

LEAN

An auditable supply chain可审计的供应链

One shared library instead of a Python environment with hundreds of packages. A small bill of materials is a security property, and we treat it as one.

一个动态库,取代动辄数百个包的 Python 环境。精简的物料清单本身就是安全属性,我们也这样对待它。

OPEN

Fits what you run融入现有系统

Call it from C, script it from Guile, or point existing tools at its OpenAI-compatible server. Gemma 4, Qwen2.5 and DeepSeek-R1-Distill-Qwen run today.

可从 C 调用、用 Guile 编写脚本,或让现有工具直接对接其 OpenAI 兼容服务。Gemma 4、Qwen2.5 与 DeepSeek-R1-Distill-Qwen 现已可用。

Figures: Qwen2.5-7B-Instruct Q4_K_M on an RTX 3050 8 GB, SonicBoom v0.0.6. Both runtimes run the same GGUF on the same machine; each figure is the median of three process runs and reproducible with one script. See the benchmark methodology.

数据条件:Qwen2.5-7B-Instruct Q4_K_M,RTX 3050 8 GB,SonicBoom v0.0.6。两个运行时在同一台机器上运行同一个 GGUF,每个数字取三次独立进程运行的中位数,可用一个脚本复现。详见测试方法。

Also from the lab实验室的其他成果

Speech, and a product that uses it all.

语音,以及一个用上全部技术的产品。

The lab's work ships in our own products first, so it is tested on real workloads before it reaches yours.

实验室的成果先用在我们自己的产品里,在真实负载上验证过,才交到你手上。

Services服务

What the AI Lab does for you.

AI 实验室能为你做什么。

From a single workstation to a private cluster, delivered under a license that fits your business.

从一台工作站到私有集群,以适合你业务的许可方式交付。

01

Private LLM deployment私有化大模型部署

Models on your own servers, AI workstations or air-gapped networks, sized and tuned for your hardware. Your data never leaves the building.

在你自己的服务器、AI 一体机或隔离网络中运行模型,按硬件选型与调优,数据不出大楼。

02

Inference performance推理性能工程

When throughput or latency is the bottleneck, we work below the framework: kernels, scheduling, memory and the compiler, measured and reproducible.

当吞吐或延迟成为瓶颈,我们深入框架之下:算子、调度、内存与编译器,一切可测量、可复现。

03

AI security reviewAI 安全审查

Model files, serving stacks and their dependency chains, reviewed by the team behind VED and our security lab.

由 VED 与安全实验室背后的团队,审查模型文件、服务栈及其依赖链。

04

Hardware bring-up新硬件适配

Model inference ported to accelerators and edge platforms the mainstream stacks do not serve well.

把模型推理移植到主流技术栈支持不佳的加速器与边缘平台。

05

Custom runtime work定制运行时开发

New model architectures, custom operators, constrained decoding and integration into your products.

新模型架构、自定义算子、约束解码与产品集成。

06

Edge & in-vehicle AI边缘与车载 AI

Lean, auditable inference for devices where a Python stack is not an option.

为无法承载 Python 技术栈的设备提供精简、可审计的推理。

Research研究方向

Where we are pushing.

我们正在攻坚的方向。

Speed and correctness are not a trade-off we accept.

速度与正确性,我们不接受二选一。

01

Beyond bandwidth超越带宽

Memory bandwidth is not the ceiling. Exact speculative decoding is the first proof: more work per pass over the weights, with output that stays bit-identical.

内存带宽不是天花板。精确推测解码是第一个证明:每次读取权重完成更多工作,输出依然逐位一致。

02

Compiling the whole model编译整个模型

Classic compiler theory applied across the layers of a model, so work that cannot affect the answer is never done.

把经典编译理论应用到模型各层之间,让不影响结果的计算根本不发生。

03

Reproducible inference可复现的推理

Determinism as an engineering guarantee, because audits, regulated industries and serious testing depend on it.

把确定性作为工程保证,因为审计、受监管行业与严肃测试都依赖它。

04

Large models, modest hardware普通硬件跑大模型

Mixture-of-experts models on a single consumer GPU, with CPU and GPU scheduled together.

在单块消费级 GPU 上运行混合专家模型,CPU 与 GPU 协同调度。

Work with us合作

Run AI on your terms.

按你的方式运行 AI。

On your hardware, inside your network, with numbers you can reproduce. Tell us about the workload.

在你的硬件上、你的网络内,用你能复现的数字说话。告诉我们你的需求。