rawmark — real hardware benchmarks

FLEET INTELLIGENCE live from Prometheus · 2026-05-08

100%Local Inference

1Total Requests

nanmsp50 Latency

0Cloud Requests

Band distribution: COMPLEX: 2 · TRIVIAL: 1

AMD Ryzen 9 3950X + RX 9070 XT GPU AMD Radeon RX 9070 XT 16GB RDNA4 · OS Windows 11 Pro · ROCm 7.1 gfx1201

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
qwen3:14b	ollama	25	256	23.5	bifrost
qwen3:14b	ollama	25	256	52.7	bifrost
bifrost-t1b	ollama	43	105	45.7	bifrost
bifrost-t1b	ollama	43	105	46.7	bifrost
bifrost-t1b	ollama	43	106	47.1	bifrost
bifrost-t1b	ollama	43	105	46.1	bifrost
bifrost-t1b	ollama	43	104	9.0	bifrost
gemma4:e4b	ollama	31	300	28.7	bifrost
gemma4:e4b	ollama	30	256	31.0	bifrost
bifrost-t1b	ollama	None	69	43.4	bifrost
bifrost-t1b	ollama	43	105	45.9	bifrost
bifrost-t1b	ollama	43	106	6.9	bifrost
bifrost-t1b	ollama	43	107	40.1	bifrost
qwen3:14b	ollama	25	256	19.4	bifrost
qwen3:14b	ollama	25	256	51.6	bifrost
mistral:7b-instruct-v0.3-q4_0	ollama	21	256	31.0	bifrost
mistral:7b-instruct-v0.3-q4_0	ollama	21	256	101.6	bifrost
gemma4:e4b	ollama	30	256	6.9	bifrost
gemma4:e4b	ollama	30	256	26.9	bifrost
phi4:14b	ollama	25	256	12.9	bifrost
phi4:14b	ollama	25	256	52.6	bifrost
qwen3:14b		47	256	55.4
qwen3:14b		47	256	8.5
gemma4:e4b		47	256	31.3
gemma4:e4b		47	256	16.6

AMD Ryzen 7 5700G + RX 5700 XT GPU RX 5700 XT 8GB RDNA1 · OS Windows 11 · ROCm N/A

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
bifrost-1a-hearth	ollama	164	28	6.3	hearth
bifrost-1a-hearth	ollama	164	29	23.8	hearth
mistral:latest	ollama	21	300	7.9	hearth
mistral:latest	ollama	21	256	56.6	hearth
mistral:latest	ollama	None	108	58.5	hearth
mistral:latest	ollama	21	256	56.9	hearth
bifrost-1a-hearth	ollama	164	27	1.7	hearth
bifrost-1a-hearth	ollama	164	28	3.8	hearth
bifrost-1a-hearth	ollama	164	26	23.4	hearth
bifrost-1a-hearth	ollama	164	26	23.0	hearth
qwen3.5:9b	timed out
qwen3.5:9b	timed out
mistral:7b	ollama	21	256	2.8	hearth
mistral:7b	ollama	21	256	2.8	hearth
mistral:7b	ollama	21	256	56.1	hearth
mistral:7b	ollama	21	249	56.1	hearth
mistral:7b	ollama	21	196	57.3	hearth

AMD Ryzen 7 5700G iGPU Vega 8 GPU Vega 8 iGPU 16GB GTT · OS Windows 11 · ROCm N/A

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
bifrost-1a-overflow	ollama	137	256	11.0	hearth-vega8
bifrost-distill	ollama	847	256	6.1	hearth-vega8
qwen3.5:4b	timed out
qwen3.5:4b	timed out

AMD Ryzen AI Max+ 395 (Radeon 8060S) GPU Radeon 8060S 96GB unified · OS Ubuntu 24.04 · ROCm 7.2 gfx1151

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
bifrost-t2-gemma4	ollama	177	256	49.9	forge
bifrost-t2-gemma4	ollama	177	256	47.9	forge
bifrost-t2-gemma4	ollama	177	256	47.9	forge
bifrost-t2-gemma4	ollama	178	300	45.5	forge
gemma4:e4b	ollama	31	300	52.6	forge
bifrost-t2-gemma4	ollama	None	200	49.8	forge
bifrost-forge-t1	ollama	None	102	58.0	forge
bifrost-t2-gemma4	ollama	178	256	48.4	forge
bifrost-forge-t1	ollama	32	256	46.3	forge
bifrost-t2-gemma4	ollama	178	256	48.9	forge
gpt-oss:120b	ollama	82	256	8.6	forge
gpt-oss:120b	ollama	82	256	35.0	forge
gpt-oss:120b	ollama	82	256	8.6	forge
gpt-oss:120b	ollama	82	256	35.1	forge
bifrost-t2-qwen3	ollama	47	2012	63.1
bifrost-t2-qwen3	ollama	154	4501	60.3
bifrost-t2-qwen3	ollama	47	2094	62.8
bifrost-t2p5-r1	ollama	36	961	5.0
bifrost-t2p5-r1	timed out
bifrost-t2p5-r1	timed out
Qwen3-1.7B-GGUF	lemonade	19	512	131.8
Qwen3-1.7B-GGUF	lemonade	126	512	128.4

AMD Ryzen AI Max+ 395 XDNA2 NPU GPU XDNA2 NPU — 0W GPU draw · OS Ubuntu 24.04 · ROCm FLM NPU

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
qwen3:1.7b	ollama	28	256	35.9	forge-npu
qwen3:1.7b	ollama	28	256	36.9	forge-npu
gemma3:4b	ollama	24	256	16.8	forge-npu
qwen3:1.7b	ollama	28	300	36.7	forge-npu
llama3.2:1b	flm	None	134	49.0	forge-npu
llama3.2:1b	ollama	50	256	52.5	forge-npu
llama3.2:1b	ollama	50	256	52.9	forge-npu
gpt-oss:20b	ollama	82	256	7.3	forge-npu
gpt-oss:20b	ollama	82	256	15.1	forge-npu

AMD Ryzen AI Max+ 395 (Radeon 8060S) [llama-server T2.5] GPU Radeon 8060S 96GB unified · OS Ubuntu 24.04 · ROCm 7.2 gfx1151 (llamacpp-rocm)

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
llama3.3:70b	llama_server	50	256	5.0	forge-t25
llama3.3:70b-Q4_K_M	llama_server	None	118	5.0	forge-t25
llama3.3:70b	llama_server	50	256	5.0	forge-t25
llama3.3:70b	llama_server	50	256	5.0	forge-t25
llama3.3:70b	llama_server	50	256	5.0	forge-t25
llama3.3:70b	llama_server	50	256	5.0	forge-t25
Llama-3.3-70B-Instruct-Q4_K_M.gguf	llama_server	50	256	5.0	forge-t25
Llama-3.3-70B-Instruct-Q4_K_M.gguf	llama_server	50	256	5.0	forge-t25