Announcement

Collapse
No announcement yet.

UserBenchmark Adjusts Algorthym to Favor Single-Thread Performance, Causing Strange Results

Collapse
X
Collapse
  •  

  • UserBenchmark Adjusts Algorthym to Favor Single-Thread Performance, Causing Strange Results



    UserBenchmark.com has adjusted the algorithm for how the overall performance of a CPU is calculated.




     




    Previously, single-core performance made up 30% of the result, quad-core made up 60%, and the final 10% was reserved for "multicore" performance. This was calculated so that an Intel Core i7-7700K would equal approximately 100% score.




     




    The new scoring places the i9-9900K at the 100% mark, but also changes the makeup of the balancing of the scores. Single-core scoring now accounts for 40% of the result, and quad-core score takes a hit down to providing 57% of the score. Finally, the multicore score (specified to be 64 threads) only makes up 2% of the final score.




     




    UserBenchmark claims that this is to better reflect the importance of single-threaded performance in gaming. However, more workloads have been embracing more threads. The balance change makes the results for effective CPU speed seem off, namely when it comes to AMD CPUs and even Intel CPUs that have more threads, but may lag behind on single-threaded performance.




     




    For example, comparing the Ryzen 9 3900X to the Core i3-9350KF yields a mere margin of 3% in favor of the Ryzen 9. This is comparing a twelve-core 24-thread chip to a four-core chip.








     




    However, comparing the "Real World Speed" tells a different story. The lowest improvement the tests show the Ryzen 9 having is 2%, but the largest gain is a whopping 85%.








     




    And now some before-and-after with the Ryzen 5 3600:




     




    (before)








     




    (after)








     




    The Average Bench drops 15.6%. Granted, some of this drop could be because of the change of reference CPU. But the effective speed, average user bench, and peak overclock bench all take major hits.




     




    Intel has a similar decrease, though not to as much of a degree, on the i7-8700K:




    (before)








     




    (after)








     




     




    And about the i9-9900K being at 100%, I believe it fair to say that it has more than 9% effective speed than an i3-9350KF.








     




    The change in results has caused some uproar, especially among those who are fans of AMD, as this change seems to punish CPUs with higher core counts, which AMD is excelling at.




     




     




     




     




    UPDATE 7/26/2019: UserBenchmark has addressed concerns... by brushing them off at best.




    The UserBenchmark FAQ page on the effective CPU speed has been updated. (Archived version here)




     




    They specifically address the AMD community's displeasure with the change:




    Quote




    AMD community


    Shortly after the Ryzen 3000 release, which we welcomed emphatically, we noticed that our CPU gaming and desktop estimates were unrealistically overestimating CPUs with core counts beyond 8 so we corrected them. The underlying data points for single, quad and multi core performance remain unchanged and are clearly visible together with gaming, desktop and workstation scores on each of our pages. Back in the early days of the AMD FX-8350 our effective speed index was predominantly single core and at that time we were heavily lobbied with cries of "cores are only getting more and more relevant". We are constantly tuning and updating our effective speed indices to match the latest developments and will continue to do so independently.






    So they noticed an issue and "fixed" it?








    This seems to say otherwise.




     




    But it doesn't stop there:




    Quote





    Presently we are aware that we slightly overestimate the latest batch of AMD 5700 graphics cards but, unsurprisingly, nobody is crying fire and continuously spamming us about that.






    Good job, UserBenchmark. You admitted what we have known for a long time: you are only good for estimations. But, more importantly, you are admitting your tests have flaws. Why should we not be upset when your recommendations are saying we can all go get i3s and compare them to Ryzen 9s? People are not going to be concerned with a slight test inconsistency. People are going to be concerned when tests are poorly balanced across all products.




     




     




    And one last bit on their "improved" FAQ page:




    Quote




    Finally ...


    Beware the organized army of shills who pump one brand or another and deal in hot air rather than reason. Only use trusted independent sources to know your chops before parting with any of your hard earned cash. May the force be with you.






    Granted, something similar was on the last version. I think that the concerns the community has are reasonable. People who don't know better may waste money based on this site. And, to top it off, a nice video of how some people put too much fat on sheep before slaughtering them.




     









    More...
      Posting comments is disabled.

    Categories

    Collapse

    Article Tags

    Collapse

    Latest Articles

    Collapse

    Working...
    X