rawmark — real hardware benchmarks

FLEET INTELLIGENCE live from Prometheus · 2026-04-12

86%Local Inference

1Total Requests

30000msp50 Latency

1Cloud Requests

Band distribution: COMPLEX: 3 · FRONTIER: 1 · MODERATE: 1 · TRIVIAL: 2

AMD Ryzen 9 3950X + RX 9070 XT GPU AMD Radeon RX 9070 XT 16GB RDNA4 · OS Windows 11 Pro · ROCm 7.1 gfx1201

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
qwen3:14b	ollama	25	256	23.5	bifrost
qwen3:14b	ollama	25	256	52.7	bifrost
bifrost-t1b	ollama	43	105	45.7	bifrost
bifrost-t1b	ollama	43	105	46.7	bifrost
bifrost-t1b	ollama	43	106	47.1	bifrost
bifrost-t1b	ollama	43	105	46.1	bifrost
qwen3:14b		47	256	55.4
qwen3:14b		47	256	8.5
gemma4:e4b		47	256	31.3
gemma4:e4b		47	256	16.6

AMD Ryzen 7 5700G + RX 5700 XT GPU RX 5700 XT 8GB RDNA1 · OS Windows 11 · ROCm N/A

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
bifrost-1a-hearth	ollama	164	28	6.3	hearth
bifrost-1a-hearth	ollama	164	29	23.8	hearth

AMD Ryzen 7 5700G iGPU Vega 8 GPU Vega 8 iGPU 16GB GTT · OS Windows 11 · ROCm N/A

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
bifrost-1a-overflow	ollama	137	256	11.0	hearth-vega8

AMD Ryzen AI Max+ 395 (Radeon 8060S) GPU Radeon 8060S 96GB unified · OS Ubuntu 24.04 · ROCm 7.2 gfx1151

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
bifrost-t2-gemma4	ollama	177	256	49.9	forge
bifrost-t2-gemma4	ollama	177	256	47.9	forge
bifrost-t2-qwen3	ollama	47	2012	63.1
bifrost-t2-qwen3	ollama	154	4501	60.3
bifrost-t2-qwen3	ollama	47	2094	62.8
bifrost-t2p5-r1	ollama	36	961	5.0
bifrost-t2p5-r1	timed out
bifrost-t2p5-r1	timed out
Qwen3-1.7B-GGUF	lemonade	19	512	131.8
Qwen3-1.7B-GGUF	lemonade	126	512	128.4

AMD Ryzen AI Max+ 395 XDNA2 NPU GPU XDNA2 NPU — 0W GPU draw · OS Ubuntu 24.04 · ROCm FLM NPU

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
qwen3:1.7b	ollama	28	256	35.9	forge-npu
qwen3:1.7b	ollama	28	256	36.9	forge-npu
gemma3:4b	ollama	24	256	16.8	forge-npu