rawmark — real hardware benchmarks

FLEET INTELLIGENCE live from Prometheus · 2026-04-10

100%Local Inference

1Total Requests

nanmsp50 Latency

0Cloud Requests

Band distribution: COMPLEX: 1 · TRIVIAL: 1

BIFROST GPU AMD Radeon RX 9070 XT 16GB RDNA4 · OS Windows 11 Pro · ROCm 7.1 gfx1201

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
qwen3:14b	ollama	25	256	23.5	bifrost
qwen3:14b	ollama	25	256	52.7	bifrost
bifrost-t1b	ollama	43	105	45.7	bifrost
bifrost-t1b	ollama	43	105	46.7	bifrost
bifrost-t1b	ollama	43	106	47.1	bifrost
qwen3:14b		47	256	55.4
qwen3:14b		47	256	8.5
gemma4:e4b		47	256	31.3
gemma4:e4b		47	256	16.6

HEARTH GPU RX 5700 XT 8GB RDNA1 · OS Windows 11 · ROCm N/A

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
bifrost-1a-hearth	ollama	164	28	6.3	hearth

HEARTH VEGA8 GPU Vega 8 iGPU 16GB GTT · OS Windows 11 · ROCm N/A

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
bifrost-1a-overflow	ollama	137	256	11.0	hearth-vega8

FORGE GPU GPU Radeon 8060S 96GB unified · OS Ubuntu 24.04 · ROCm 7.2 gfx1151

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
bifrost-t2-gemma4	ollama	177	256	49.9	forge
bifrost-t2-qwen3	ollama	47	2012	63.1
bifrost-t2-qwen3	ollama	154	4501	60.3
bifrost-t2-qwen3	ollama	47	2094	62.8
bifrost-t2p5-r1	ollama	36	961	5.0
bifrost-t2p5-r1	timed out
bifrost-t2p5-r1	timed out
Qwen3-1.7B-GGUF	lemonade	19	512	131.8
Qwen3-1.7B-GGUF	lemonade	126	512	128.4

FORGE NPU GPU XDNA2 NPU — 0W GPU draw · OS Ubuntu 24.04 · ROCm FLM NPU

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
qwen3:1.7b	ollama	28	256	35.9	forge-npu
qwen3:1.7b	ollama	28	256	36.9	forge-npu