NEW: Rendertest-2010

John_Cline wrote on 8/2/2010, 2:06 AM
I've released RENDERTEST-2010.VEG, it is an updated version of RENDERTEST-HDV.VEG which I originally released on May 8, 2007. This new VEG will run in Vegas v8 and higher.

The fastest render time when I released the original test was 120 seconds on an Intel 2.66Ghz Core2 Quad Extreme. Recently, the render times have been in the 30-40 second range using the latest Intel i7 processors. The new RENDERTEST-2010.VEG should take four times longer to render than the last version so there should be some correlation between these results and the results from the original test.

The render parameters are the same as the original file: render it to HDV using:

Save as Type = "MainConcept MPEG-2"
Template = "HDV 1080-60i"
Select the "Custom" button and make sure "Video Rendering Quality" is set to "BEST."

Here's the exciting part: Steve Mann has created an online database where your results can be posted and compared to other results. Steve's work is a significant improvement over having to make some sense of the nearly 500 posts in the original RENDERTEST-HDV thread on the Sony Vegas forum.

The URL to download the RENDERTEST-2010.ZIP and post your results to Steve's database is:

http://www.mmdv.com/sonyvegas/rendertest

The password is "vegasuser"

Enjoy,
John Cline

Comments

ushere wrote on 8/2/2010, 2:31 AM
just done it - 268secs.....

but how do i find out 'thread' count? thanks earl - 8 threads / cores / whatever

edited to secs and added threads as i can't edit in table....
TheHappyFriar wrote on 8/2/2010, 4:37 AM
Invalid password, even if I copy/paste the password.

What about hosting the veg on vasst or something simular?

EDIT: worked on the 5th try. :)

EDIT2: I'd rather not have my e-mail up for anyone to see if I was going to submit my data.
srode wrote on 8/2/2010, 5:04 AM
Looks like the column for seconds is sorted as a text not a number? 1011 goes to the top above many lower numbers when using that as the sort key.
Grazie wrote on 8/2/2010, 5:08 AM
499 seconds

Thanks John.

Grazie
megabit wrote on 8/2/2010, 6:11 AM
John,

Thanks for this effort - just got it confirmed that my QX6700 is sooo slow compared to the new processors (not that it came as a surprise, but still..)).

Like the others, I'd rather not have my full email being available in yet another public place (even if passworded) - could you please consider hiding that column?

Thanks

Piotr

AMD TR 2990WX CPU | MSI X399 CARBON AC | 64GB RAM@XMP2933  | 2x RTX 2080Ti GPU | 4x 3TB WD Black RAID0 media drive | 3x 1TB NVMe RAID0 cache drive | SSD SATA system drive | AX1600i PSU | Decklink 12G Extreme | Samsung UHD reference monitor (calibrated)

Earl_J wrote on 8/2/2010, 6:23 AM
Hello Stephen,
It is sorted as numbers ...
the only way to solve that problem is to permit leading zeroes in all the entries ...
then, 0976 will get sorted properly to its spot below 1011 ...
sorting three place numbers to four place numbers is like apples and oranges ...
and they sort into very unexpected viewing displays ...

Just saying ...

Until that time... Earl J.
Earl_J wrote on 8/2/2010, 6:35 AM
Hello Leslie,
once the render begins, open the task manager (right-click on any blank portion of the task bar at the bottom of your screen) and open the task manager; once that is open, click on the performance tab; it will open a window with two displays, one is the CPU usage and the other is the memory usage.
The CPU usage should display a number of smaller windows, each window represents a thread ... so if you have 4 windows showing, you have 4 threads ... during the actual rendering, the program breaks all the tasks up into modules of tasks called threads ... the more threads there are, the more pieces the program may use to accomplish its tasks ... in theory, it is very enticing; in practice the improvements are not that quick.
One would expect that an 8 thread machine would work twice as fast as a 4 thread machine ... but then again, the program must use more resources itself to account for twice as many threads and handle each one as it finishes ... and prepare another batch of tasks for that thread... so the speed increases are mostly incremental and not exponential ...

I believe the term core is also interchangeable with thread ... I could be wrong, but there is also a good chance I'm right ... (grin) I'm often not sure that I understand everything I know about these new-fangled thingamajigs we call computers . . . (lol)

Also please excuse any insult to your (or anyone's) intelligence ... I only mean to explain and encourage, not demean or discourage ...

We'll find out shortly, won't we ...

Until that time ... Earl J.
farss wrote on 8/2/2010, 6:44 AM
How is this a valid "render test" ?

1) I was able to almost halve my render time by prerendering to RAM.
2) The main CPU load is in compositing not rendering. As such if and when a new version of Vegas is released with hardware accelerated rendering I would anticipate this test will show little to no improvement.
3) It would seem to largely ignore disk i/o speed.

I'm not trying to be critical, especially given the amount of effort put into this however unless users are aware of exactly what this test is measuring they could be misled.

Bob.
Earl_J wrote on 8/2/2010, 6:51 AM
Hello John,
Steve's done a wonderful job with the site and the collection of data ...
my only criticism ... he may want to put the bottom scroll bar at the bottom of the current display of results ... as the list gets longer, many visitors will not scroll to the complete end of the list just to scroll to the right and left ... and if they do, it will get old very quickly.

I do like the layout and the ease with which the columns can be sorted by the viewer ... most excellent ... so the viewer can sort through the CPUs to see how his/her speed compares with other identical/similar CPUs ... as well as with the overall speed of the render ... except for the four-place versus three-place numbering sort ...

I think this go-round of the render test should progress much smoother and easier for all of us - testing and reporting - hey, it is the 21st Century, after all ... (grin)

Until that time ... Earl J.
JohnnyRoy wrote on 8/2/2010, 7:38 AM
Just posing a question for thought...

Is rendering to HDV still the valid test in 2010?

When I saw that you made the new test 4x harder I thought to myself... we don't need a new render test to take 4x longer... just use the existing render test and switch to rendering AVCHD and it will take 5x longer (or more)! AVCHD puts a much harder load on the CPU and would prove to be a much tougher test with the existing render test. The original render test used HDV because it was the toughest format we could find back then. Now it edits like butter in Vegas. I think AVCHD is the new "tough" format to beat.

Maybe we need two test scores? One for HDV and one for AVCHD? Not trying to complicate things. Just trying to keep it more "real world". ;-)

~jr
ushere wrote on 8/2/2010, 8:11 AM
i think jr has a very valid point in as much as hdv is now a known quantity, whereas avchd is the new kid on the block* and people really don't have many reference points for it yet.

that said, i'm very unlikely to give up hdv, so would like to know what i can expect from future cpu's in rendering terms.

dear old bob, ever the careful watchdog, also introduces a couple of very pertinent point too, disk i/o and the possibility of gpu rendering. i'm not sure that prerendering is important in the overall picture, other than to demonstrate vegas's abilities ;-)

*and goodness knows when the next kid's coming along.....
Grazie wrote on 8/2/2010, 9:27 AM
I agree with jr. And putting it another way, why bother with HDV, just go back and test using DV? Now I bet THAT would be super fast!

Moral of story - test for the outcome that is needed. Sir Peter Medwar wrote a book on a similar subject: "The Art of the Soluble" which used the premise that scientist would take on things that they could prove.

OOPs . . now that's interesting . while I was writing this I had "Render to a New Track" going on, using that script, and I used the Template we are using, it did it 4 minutes and not the 8 minutes . . What can I extract from that info?

Grazie
John_Cline wrote on 8/2/2010, 10:39 AM
"1) I was able to almost halve my render time by prerendering to RAM."

Well, then don't prerender. :)

"2) The main CPU load is in compositing not rendering. As such if and when a new version of Vegas is released with hardware accelerated rendering I would anticipate this test will show little to no improvement."

Yes, this test stresses the CPU with compositing tasks. When Vegas is released with GPU accelerated rendering (which will probably be only AVCHD) then I will release a new test. It will be VERY interesting to see how video card clock speed and number of cores will affect the test results.

"3) It would seem to largely ignore disk i/o speed."

Disk speed is of essentially no consequence in this test and has virtually no impact on the test score.

This test is all about CPU (and RAM) speed and as farss (Bob) pointed out, the CPU is primarily chunking away at compositing, not rendering. In this case, MPEG2 is just as valid as rendering to any other format. The main reason that I stuck with MPEG2 rendering is to maintain some correlation with all the old scores, just divide the new score by four to compare it to any of the old scores.

About the concern of e-mail addresses and the column sorting issue, I'll let Steve respond to that.

John
Chienworks wrote on 8/2/2010, 11:46 AM
"It is sorted as numbers ...

Not to be too picky, but what you're describing is sorting as text. If you really were sorting by numbers then the sort would go by numeric value and the number of digits wouldn't be an issue at all.
Chienworks wrote on 8/2/2010, 11:52 AM
"1) I was able to almost halve my render time by prerendering to RAM."

Or include the prerendering time in the total time if you want to make it a fair comparison. After all, the test is to see how long it takes to get the job done, and prerendering is part of getting the job done.

Kinda like the old DVD rendering debates we had a while back when folks started talking about how much faster it was to render to AVI first, then render that new file to MPEG instead of rendering directly to MPEG to begin with. They'd come up with figures like "it took 1.5 hours to render directly to MPEG, but only 0.6 hours to render the AVI to MPEG." Then i'd ask them how long the initial render to AVI took and after some grumbling and hemming and hawing, "oh, about 1.2 hours.". So, the actual total time of 1.2 + 0.6 of the two renders was longer than doing the single render to begin with.
TheHappyFriar wrote on 8/2/2010, 12:02 PM
the sorting the numbers shouldn't be an issue.... by default languages should have 154 before 1011.

With someone already getting a 154 second test, I'd say this test will last a few months. (relative to the speed at which the previous render test was considered obsolete). something that lasts an hour would be a better test I'd think.
Earl_J wrote on 8/2/2010, 3:04 PM
Hello Kelly,
you're exactly correct ...

... never mind ... (a direct quote from Roseanne Roseanna Dana)

so, I was correct when I admitted that I don't understand everything I know about computer stuff ... (sigh)
or simply can't remember ... (double sigh sigh)

Until that time ... Earl J.
TheHappyFriar wrote on 8/2/2010, 4:23 PM
looks like the sort is doing it based on char strings & not integers/floats. That'd be why that's all messed up. If it must be a char string, reading it, saving a separate string that's backwards for comparison & sorting it with the reverse string would solve the sorting problem.
srode wrote on 8/2/2010, 5:55 PM
I do have to say, the new approach to presenting the data is very very nice!!!!!! Kudo's to the people who worked this up for us!!! Looks like I am not the only one who put their last and first name in the wrong places! :)
Steve Mann wrote on 8/2/2010, 6:37 PM
In Preferences/Video, you can set the number of threads for Vegas to run. I'm not sure the exact definition of a thread according to Sony, but I do know that some people have deliberately set their thread count to less than their CPU core count.

Maybe someone can expand on this for me.

Steve Mann
Steve Mann wrote on 8/2/2010, 6:40 PM
I am using a third-party PHP program for the on-screen display of the data, and all columns are treated as text. Downlowd the CSV file and use Excel to analyze the data.

Steve Mann

Steve Mann wrote on 8/2/2010, 7:03 PM
I am using a third-party PHP program for the display of the data, and all of the data are text. In a future release I will add the leading zeros to the intermediate data so that it will sort correctly.

Steve Mann
Steve Mann wrote on 8/2/2010, 7:06 PM
"I believe the term core is also interchangeable with thread"

Cores and Threads are not interchangeable. You can set (in Preferences) Vegas to one thread if you want. Vegas will be as slow as a Pentium 133, but your processor will have loads of spare time for other tasks.
Steve Mann wrote on 8/2/2010, 7:12 PM
It doesn't really matter what the task that we're performing with Vegas, as long as we all test using the same parameters. Disk speed, well, mechanical disks anyway, would make such a tiny difference from Brand A to Brand B. Solid-state disks (SSD) *would* show much better numbers, as evidenced with the difference when prerendering in RAM. So, to keep all results on the same field, don't prerender.

Steve Mann