I'm working on a benchmark tool for Goverlay. It can compare hardware, but the current chart has only two measured reference systems and the other results are estimates. What would make the benchmark and its comparisons more useful?
How can I make a Linux gaming benchmark more useful?
Answers
Make the distinction between measured and estimated results impossible to miss. With only two reference systems, a chart can look much more precise than the data behind it.
Collect more runs across different hardware before treating the rankings as reliable. It would also help to show how many results contributed to each comparison, rather than presenting every entry as equally certain.
Reply to this answer
I'd focus on repeatability before chasing a huge database. Keep the benchmark settings consistent and let people run the same test more than once. Show the individual results as well as a summary, so a strange run doesn't quietly become the reference score.
When people submit results, collect the settings alongside them. Hardware names alone won't tell you whether two scores came from comparable tests.
Reply to this answer
Give people enough system context to interpret a score: GPU, CPU, driver version, resolution and the benchmark settings. Linux driver differences can matter, and a comparison without that context might send someone looking for a hardware problem when the test conditions were different.
Reply to this answer
Please show more than one average frame rate if the tool can collect frame times. A run can have a decent average and still feel uneven. A frame time graph or a low percentile result would give people another way to spot that.
lootRoute!Spatula:
Labels for those numbers would help too. Not everyone looking at a benchmark knows what a frame time or percentile means.
Reply to this answer
Keep the benchmark easy to find and explain what it actually tests. A small synthetic scene is handy for comparing systems, but its score shouldn't be presented as a prediction of performance in every game.
Reply to this answer
Add your answer
Answers are reviewed before publication.
lootDraftKiosk:
That makes sense. The current estimates need a clear label, and the chart should show where there's actual data behind a comparison.