Results 1 to 6 of 6

Thread: New bm suite for gfx at anandtech

  1. #1

    New bm suite for gfx at anandtech

    http://www.anandtech.com/weblog/show...i=675#comments


    "In keeping with our desire to refresh our GPU test suite periodically, we?re going to be redoing our GPU test suite to rotate in some more modern games, along with rotating in some DirectX11 games capable of taking advantage of this generation of GPU?s full capabilities. And while we already have a pretty solid idea of what we?re going to run, we wanted to throw out this question anyhow and see what responses we get."

    What do you think?

    I did anticipate this was going to happen, and we have been discussing it a bit. I did a response at anand:

    "Ryan, total respect for non syntetic bm, and even more respect for involving your customers. Lets have more of that. That means more to me, than the reviews themselves.

    Is the new methology going to be used for the fermi launch?

    If so:
    Dont you find it problematic? - and if, - how?
    You must know a lot speculated that you would alter your bm suite when fermi was comming. Controlling the anticipation like you do here, does not change that. Knowing we anticipated this situation, how do want us to interprete what is happening?

    I think we had the same problems with the intel ssd reviews, fortunately it ended with your own excellent bm suite, you made yourself, instead of the 4k random writes all over. Happy ending here. A great benefit for your customers. I hope your will get there to with your gfx suite.

    But changing it right now is highly problematic, because the comparison to earlier bm is weadk. I think Fermi should stand by itself, like the ssd should.

    "

  2. #2
    Senior Member
    Join Date
    Sep 2009
    Posts
    3,026
    I would imagine that AnandTech will retest all direct competitors to Fermi with the new suite and the newest drivers. That's usually standard on bigger reviews.

    So this means that as of March, 2010, Anandtech does not have a solid benchmark suite? I'm going to go on a daring extrapolation adventure and say that there might not be any GTX's at AnandTech yet. If they had one, would't they be putting it through the paces? Especially since it's really new. They'd want to iron out any bugs quickly to avoid blank lines on their graphs. I think the editorial may imply that they are at least expecting Fermi to show up before the 26th.

    Just for a note, I never look at 3dMark scores. I mainly look at graphs with the 1920x1080 resolution of my monitor and see what I could do with that particular game. I'm glad to see it go. I guess this means that they will not be using the heaven benchmark either. Are they implying someone is optimizing for that in the editorial?

  3. #3
    Senior Member
    Join Date
    Sep 2009
    Posts
    2,055
    Re: Dont you find it problematic? - and if, - how?

    I'm sure Anandtech finds nothing problematic about updating their benchmark suite to include newer games that readers actually care about rather than catering to the small minority who believe that changes to the benchmark suite now must have been bought and paid for by NVIDIA marketing.

    Games like Mass Effect 2, Aliens vs. Predator, CoD:Modern Warfare 2, Battlefield: Bad Company 2, Dirt 2, S.T.A.L.K.E.R. Call of Pripyat, Bioshock 2 and others would be more deserving to be included on the list compared to most of what is there already.

  4. #4
    I'm with distinctively on this one - Anand will likely retest all the HD 58**'s, the HD 5970 and the GTX 275/285/295 to have numbers to include with Fermi. Not a bad idea since this means that they will probably end up using Cat 10.3.
    Athlon II X3 455 (Rana) @ 3.83 GHz, GA-MA78G-DS3H rev 1.0, 4GB DDR2, XFX HD 5850 @ 801/1099 MHz, Win XP Pro-32 SP3, Seagate 7200.11 Barracuda 500GB (OS) - recovered from BSY error on SD1A firmware, Vantec Ion2+ 550W, NZXT Hush

  5. #5
    They'll retest everything, but are they saving face and agreeing to benchmark all of Nvidia's latest and greatest games they benchmark in, but making it look like they chose the new set of games?

    The proof I guess would be in if the new "suite" ends up the same 5 games every other site that manages to get a 480gtx are the same, or if its a distinctively different benchmark "suite" to the one Nvidia will clearly be pushing out as the best way to test their card.

    Even if it was their old test quite I'd imagine they would have used the latest driver from AMD anyway and retested the 5870.

    But it idea does nag at the back of my mind, that this is merely a way to benchmark the card as Nvidia wants them to, but just wanting to appear like its their own idea.

    We'll see in a couple weeks.

  6. #6
    Senior Member
    Join Date
    Sep 2009
    Posts
    3,026
    Quote Originally Posted by Drunkenmaster View Post
    They'll retest everything, but are they saving face and agreeing to benchmark all of Nvidia's latest and greatest games they benchmark in, but making it look like they chose the new set of games?

    The proof I guess would be in if the new "suite" ends up the same 5 games every other site that manages to get a 480gtx are the same, or if its a distinctively different benchmark "suite" to the one Nvidia will clearly be pushing out as the best way to test their card.

    Even if it was their old test quite I'd imagine they would have used the latest driver from AMD anyway and retested the 5870.

    But it idea does nag at the back of my mind, that this is merely a way to benchmark the card as Nvidia wants them to, but just wanting to appear like its their own idea.

    We'll see in a couple weeks.
    I guess its really a matter of trust when it comes to Anand. The best I can say is that AnandTech has earned my trust over the years. There were a couple of times I questioned it, but in the end, I believe I was the one that was wrong. This is entirely my opinion though and I will definitely respect anyone else opinion if they have good reason to believe otherwise.

    One thing though about the benchmarks. AnandTech isn't using synthetic tests so that means no Unigine. That would most likely be one that nVidia pushes. That is unless they really suck in the sections not shown in their graphs.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
WordPress Appliance - Powered by TurnKey Linux