rawmark — real hardware benchmarks

AMD Ryzen 9 3950X + RX 9070 XT GPU AMD Radeon RX 9070 XT 16GB RDNA4 · OS Windows 11 Pro · ROCm 7.1 gfx1201

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
mistral:7b-instruct-v0.3-q4_0	ollama	21	256	101.6	bifrost
Granite-4.1-8B (Q4_K_M)	ollama	45	256	76.6	bifrost
Qwen3.5-9B (Q4_K_M)	ollama	25	256	62.7	bifrost
qwen3:14b	ollama	47	256	55.4	bifrost
phi4:14b	ollama	25	256	52.6	bifrost
bifrost-t1b	ollama	43	106	47.1	bifrost
gemma4:e4b	ollama	47	256	31.3	bifrost

AMD Ryzen 7 5700G + RX 5700 XT GPU RX 5700 XT 8GB RDNA1 · OS Windows 11 · ROCm N/A

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
mistral:latest	ollama	21	256	62.3	hearth
mistral:7b	ollama	21	196	57.3	hearth
Ministral-8B-2410 (Q4_K_M)	ollama	18	256	50.5	hearth
bifrost-1a-hearth	ollama	164	29	23.8	hearth

AMD Ryzen 7 5700G iGPU Vega 8 GPU Vega 8 iGPU 16GB GTT · OS Windows 11 · ROCm N/A

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
bifrost-1a-overflow	ollama	137	256	11.0	hearth-vega8
bifrost-distill	ollama	847	256	6.1	hearth-vega8

AMD Ryzen AI Max+ 395 (Radeon 8060S) GPU Radeon 8060S 96GB unified · OS Ubuntu 24.04 · ROCm 7.2 gfx1151

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
Qwen3-1.7B-GGUF	lemonade	19	512	131.8	forge
bifrost-t2-qwen3	ollama	47	2012	63.1	forge
bifrost-forge-t1	ollama		102	58.0	forge
Nemotron-3-Nano-30B-A3B (UD-Q4_K_XL)	ollama	32	256	56.5	forge
gemma4:e4b	ollama	31	300	52.6	forge
bifrost-t2-gemma4	ollama	177	256	49.9	forge
gpt-oss:120b	ollama	82	256	35.1	forge
Llama-4-Scout-17B-16E (UD-IQ1_M)	ollama	25	256	19.8	forge
Qwen3-VL-235B-A22B (UD-TQ1_0)	lemonade	23	256	17.0	forge
bifrost-t2p5-r1	ollama	36	961	5.0	forge

AMD Ryzen AI Max+ 395 XDNA2 NPU GPU XDNA2 NPU — 0W GPU draw · OS Ubuntu 24.04 · ROCm FLM NPU

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
llama3.2:1b	flm		134	49.0	forge-npu
gpt-oss:20b	flm	82	256	9.1	forge-npu

AMD Ryzen AI Max+ 395 (Radeon 8060S) [llama-server T2.5] GPU Radeon 8060S 96GB unified · OS Ubuntu 24.04 · ROCm 7.2 gfx1151 (llamacpp-rocm)

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
llama3.3:70b	llama_server	50	256	5.0	forge-t25
llama3.3:70b-Q4_K_M	llama_server		118	5.0	forge-t25
Llama-3.3-70B-Instruct-Q4_K_M.gguf	llama_server	50	256	5.0	forge-t25