Multi-camera (HD) workflow - Sony MXF or MP4?

ReceptorDeceptor wrote on 3/19/2014, 10:02 PM
Hi there,

Just wanted to hear people's opinion on this matter - I'm currently working on a video with Vegas Pro 12 that's been shot using multiple (different) video cameras, all the tracks have been synced and are now color corrected & matched together using PluralEyes 3. The only problem is that - for the sake of the workflow - I try to minimize the playback slugginess and have been developing different kind of approaches on my desktop edit unit throughout the past years.

The problem is that it's hard to decide whether I should render the actual "final" work cuts from each camera as Sony MXF or render them using MainConcept/Sony MP4 codec with steady 50MBps stream rate.

I have a decent GPU card with OpenCL support (AMD FirePro v7900) in my edit station (Intel i7 core-860 + 16GB RAM). Tthe render times are way faster when using GPU-accelerated MainConcept MP4 templates @50MBps constant bitrate than what they are when I'm rendering to Sony MXF files.

Whenever rendering to MXF, my video card isn't utilized _at all_ (checked it out with a program named GPU-Z that's freeware and very handy for monitoring GPU load & usage). With MainConcept MP4 rendouts the rendering goes much more faster and my GPU is actually utilized.

As far as I know, in this case the render time is caused by Sony's plug-ins that I'm using (ie. color correction) that are GPU accelerated when rendering - but only when using certain codecs. MXF it seems is not one of them. There is no GPU usage whatsoever.

However, when MXF rendering, the algorithm actually checks whether or not recompression is required. If not, the render times are lightning fast (about as fast as your HDD/SDD I/O transfer rates).

So - as for me, after rendering the files to as-high-as-possible video formats per each video track, the next step is to start editing the project in multi-track video mode. I'd like to hear your opinion on editing multi-camera HD-video.

Should I render the master duplicate (ie. originals) using Sony/MainConcept (MP4) or stick with MXF? Benefits vs. drawbacks on each one? What about video proxies? Now that the material has been color matched for each camera track, should I make a video proxy rend for each vid track as well in order to avoid sluggish playback?

Thanks a lot for your help, opinions or comments in general regarding this issue.



Comments

musicvid10 wrote on 3/19/2014, 10:34 PM
If any of your footage can be smart-rendered, you should stick with MXF. No question about it.
johnmeyer wrote on 3/19/2014, 11:11 PM
If you are talking about rendering to an intermediate in order to get better (faster) playback on the timeline during editing, then MP4 is the worst possible choice you could make: it compresses the video to within an inch of its life, and it takes a huge amount of CPU power to play it within a video editing application.

I prefer Cineform because I think it holds up a lot better (less quality hit) than MXF, but I haven't done extensive testing, so I could be wrong. If I didn't have Cineform (which is free from GoPro, at least for their somewhat defeatured version), I'd use MXF.

Do not use MP4.

P.S. You mention the CPU usage. That indicates absolutely nothing. If one format (MP4) requires ten times the CPU just to play, then that tells you all you need to know. The fact that MXF [I]doesn't[/I] tax the CPU when you do a simple playback test is actually a good thing.
videoITguy wrote on 3/19/2014, 11:28 PM
as a digital intermediate Sony MXF holds up well in 3 gens, Cineform can do as well thru 5 gens...both are GPU non-dependent which is a real plus+
ReceptorDeceptor wrote on 3/20/2014, 8:35 PM
"P.S. You mention the CPU usage. That indicates absolutely nothing. If one format (MP4) requires ten times the CPU just to play, then that tells you all you need to know. The fact that MXF doesn't tax the CPU when you do a simple playback test is actually a good thing. "

I'm sorry if I was unspecific - I meant rendering times and the CPU+GPU utilization rates upon rendering. Rendering to MXF is slow and doesn't utilize the GPU at all, whereas MainConcept MP4's are way faster and do utilize GPU very well. Yes, I tend to render to intermediate, uniform formats (that are as high-quality as possible) in order to save time in the final editing. This also done to avoid the most common bugs in Vegas (ie. sometimes mixing different types of video source material can cause all kinds of unwanted and unexpected glitches).

Once rendered to go, the playback with both MP4 and MXF files seems to be just about the same in terms of playback rates and so forth.

MP4 rendering with maximum settings @ 50MBps constant bitrate is a way to get good results out really quick, especially with GPU assisted rendering.

MXF (422/best quality) on the other hand, only utilizes the CPU upon rendering (GPU loads stay at 0%) - however it can spot out the parts where recompression is not required and work accordingly, which sort of gives it back some of the inadvantages in rendering times.

videoITguy wrote on 3/20/2014, 8:39 PM
huh, "inadvantages?" uhh, wha
musicvid10 wrote on 3/20/2014, 9:17 PM
Speed, Quality, Size.
Pick two.
Laurence wrote on 3/20/2014, 9:51 PM
Just one clarification here that may or not be relevant to your situation:

Sony XDcam .mp4 is actually the same MPEG2 video compression as is used in the XDcam mxf or HDV .m2t formats (given that the other parameters match). In fact you can smart-render between all three of them if the dimensions and bitrate are the same.

HDV m2t uses data compression on the audio so I try to avoid rendering into that. XDcam mp4 and mxf both use uncompressed audio.

XDcam mp4 lets you do mono audio or no embedded audio and sometimes that makes it a great choice. XDcam mp4 is also read by Handbrake with both stereo audio channels whereas with XDcam mxf into Handbrake you only get one side. I often use it as an intermediate int Handbrake for this reason.

XDcam mxf lets you use higher bitrates and bit depths. You can also do 24 bit audio and surround sound. At the same 35mbps bitrate there is no quality advantage of one format over the other.

A couple of other things:

XDcam mp4 smartrenders way faster but there are occasional glitches at the transitions, especially with cuts.

There is a bug that has made it through the past several versions of Vegas where if you select the embed markers option on an XDcam mp4 render, there will be no embedded markers in the XDcam mp4 render. As far as I know this is the only format where the embed markers function is broken. I fully expect that this bug (along with several others) will still be there in Vegas 13.

You can upload an XDcam mp4 render to Vimeo and the quality is fabulous.
videoITguy wrote on 3/21/2014, 12:25 AM
Mp4 is a container just as MXF is a container - Mpeg2 is a codec.
I doubt that XDCAM.Mp4 is a suitable digital intermediate for the reasons ascribed. Mp4 as a container for other codecs is also less than suitable for digital intermediates than the Sony MXF or Cineform suggestions made.

I can tell that the OP missed the point of dripping sarcasm that several of us leveled at his concern of using GPU to be a "good" thing. I think we are still far off the mark on a communication that is meaningful to the OP's world.
musicvid10 wrote on 3/21/2014, 12:29 AM
Laurence happens to know what he is talking about.
His reasons for favoring XDCAM mp4 over mxf are sound.
The same mpeg-2 encoder is common to both containers in Vegas.
Both use lossy interframe compression, except during smart-render, which allows for infinite lossless generations.

If you were intent on "dripping sarcasm" it was lost on me, sorry.
videoITguy wrote on 3/21/2014, 11:28 AM
disclaimer in that I do not have direct experience in production workflows for XDCAM EX - however

To say that XDCAM EX MP4 and Sony MXF are directly interchangeable is a broad sweep generalization. XDCAM EX is actually a large combination of Mpeg2 and Mpeg4 components. I believe it was intended for camera capture and most production handling has a transcoding step.

There are many variants of Mpeg2 as an encoding scheme.If you ever try to tweak the details of a variant you will find out how quickly it can be messed-up. I would not expect an XDCAM variant to be smart-rendered and intercut with Sony MXF containing its own variant of Mpeg2.

What I was pointing out is that Laurence's own observation that rendering often spoils (read his post above) would suggest that XDCAM is not going to render always well as an intermediate.
Laurence wrote on 3/21/2014, 12:03 PM
While the XDcam mp4 and mxf formats can and are used as containers for a number of codecs across different camera types, the Sony templates both contain the same MPEG2 codecs and you can in fact smart-render between them. I have done this so I am sure. The XDcam mp4 adds higher quality bitrate and audio options, but at 35mbps the MPEG2 codec is exactly the same.

I am the kind of person who uses published knowledge as a starting point but then experiments deeply before settling on a workflow so as to eek out that last bit of quality. There are a number of us here who are like that and I have appreciated our discussions of this very much over the years.

While I wouldn't use most .mp4 formats for working intermediates, XDcam mp4 is an exception. I use it regularly with pristine results. Vegas seems to fly and work at peak efficiency and quality with their own Sony formats and this is another reason I use them so regularly.
Laurence wrote on 3/21/2014, 12:06 PM
As always, don't take anyone's word for any of this. Just use what we say and try it out for yourself. My prediction is that your results will mirror mine.
musicvid10 wrote on 3/21/2014, 12:25 PM
"

Well said.
+1
videoITguy wrote on 3/21/2014, 12:37 PM
Thanks for the comments about play with XDCAM EX. Hmmm, and this is interesting- I am checking the MP4 container in the VegasPro 9.0e dialogue box and it does not give a clue what it will contain for a codec.

Here's the diff that I am seeing XDCAM EX MP4 container offers no customization of the codec except overall bit rate flow. Unlike all the Mpeg2 container/codec combinations where there is advanced access to any and all parameters. Why is this?

Has this changed in later versions of VegasPro?
videoITguy wrote on 3/21/2014, 5:41 PM
Bump
Laurence wrote on 3/21/2014, 5:52 PM
In my experience, the quality of the mpeg2 codec in the XDcam mxf and mp4 containers is quite a bit better than the one you can access from the MPEG2 dialog. I don't know why this is the case.

Also, while the mpeg2 at 1440x1080 25Mbps HDV resolution looks the same when it is smart-rendered across HDV and XDcam formats, in the areas near transitions where it actually rerenders, it is noticeably better looking when rendered to one of the XDcam formats. Again, I don't know why this is the case. Sony seems to have really tweaked their XDcam formats both for quality and rendering/previewing speed. They really look better than mpeg2''s reputation would lead you to believe.
videoITguy wrote on 3/21/2014, 7:51 PM
I went into a deep dive for some third party comments and found the issue of XDCAM EX a bit more complex than I was hoping. Many containers, many variants, and actually some restrictive licensing from SONY.
Specifically I looked into comments of using it on the ordinary timeline, because in fact most workflows of licensed usage refers to camera acquisition and transcoding.
Here is one singular comment from a credible source:
"You'll see some of the most important differences between 35Mb/s EX and ProRes HQ or 100Mb/s NanoFlash on fine patterns such as brickwork or gentle gradients like blue sky where where the 35Mb/s will be a touch blocky compared to the others. The larger the viewing screen the more obvious the difference of course. The differences are not always large, but they are there and they can get worse over multiple generations even if using a high quality intermediate. "
Laurence wrote on 3/21/2014, 8:28 PM
Up until Musicvid's recent thread about using 10bit DNxHD to feed Handbrake, I never thought there was any advantage to using 10 bit formats with Vegas. I thought that because Vegas by design is limited to 8 bit color. What I didn't realize though was that Vegas works with 8bit RGB whereas common 8bit video formats are 8 or 10bit YUV. 8 bit RGB has about the same amount of color information as 9 bit YUV however, and 9 bit vs 8 bits means twice as many colors, which translates into gradients that are quite a bit smoother. Thus I now realize that there is a notable quality difference in Vegas with the 50Mbps 10 bit XDcam mxf over the 35Mbps 8 bit XDcam mp4 or mxf that I have been using until now.

Now if there is just some way we could get Handbrake's developers to recognize both audio channels in an XDcam .mxf intermediate... Musicvid... are you listening... ;-)
musicvid10 wrote on 3/21/2014, 9:41 PM
I am, and from my recent experience with DNxHD 444, you need to get one of the volunteer developers at libav (not ffmpeg) interested in working on it. It's worth noting that nobody at libav or handbrake gets paid a cent for what they do.

I started by submitting a bug report; DNxHD 444 was looked at, dropped for many months, then re-opened, and finally implemented about a year later. In the meantime, I provided lots of test files. It's all a matter of getting someone (probably not a native English speaker) to take it on as a personal project.

Arguing the merits of DNxHD or Cineform over MPEG-2 becomes moot when smart rendering is feasible, as in your workflow and xdcam shooters. BTW, it's a fair assumption that Cineform will never be supported in libav / Handbrake.
NormanPCN wrote on 3/21/2014, 10:40 PM
Thus I now realize that there is a notable quality difference in Vegas with the 50Mbps 10 bit XDcam mxf over the 35Mbps 8 bit XDcam mp4 or mxf that I have been using until now.

50Mbps 422 "Sony MXF" is not 10-bit. It is still 8-bit. According to MediaInfo.

It is 4:2:2 versus 4:2:0 of the 35Mbps XDCAM format and that is why it has a higher bitrate. To compensate for having twice the stored color information.
videoITguy wrote on 3/21/2014, 10:56 PM
422 MXF = 8bit encode - don't know what Laurence was trying to say.

AS far libav development and the whole bag of the open source issues - I really don't have any requirement of it - I'll leave that to those who will.....
Laurence wrote on 3/21/2014, 11:08 PM
Thanks for the correction. My mistake.

How much advantage is there to rendering out footage that originated at 4.2.0 to 4.2.2 after color correction then? I have been using 35mbps XDcam formats because I couldn't see a difference between that and the 50Mbps 4.2.2 formats. I have recently been doubting all my earlier judgments though. Anyone have an opinion as to whether 50Mbps 4.2.2 rendering is a better option given my 4.2.0 camera format?
videoITguy wrote on 3/22/2014, 7:20 AM
Shooting with the Canon XF305 MXF 4.2.2 or Sony PMW 160+ with 4..2.2 - these are $6000 cameras, then I would use the 50Mbps 422 MXF for complex Vegas compositing - otherwise no.