For starters when you pan a camera fast or something moves quickly in front of the camera you get "motion blur" as the camera's shutter is open for a finite period of time. Motion blur is generally a good thing, without it things seem to "judder" rather than appear to move smoothly especially at frame rates below 60fps.

All this blurring can pose a challenge for any codec that uses interframe compression especially at low bit rates. Here the encoder is only looking at the difference between frames and the blur causes a greater difference to occur and hence a challenge for the encoder. If only it were that simple though :)

Almost all cameras today use CMOS sensors which have a problem with skew when things move. The motion blur, especially at nominal shutter speeds, helps mask this.

There's also other issues afoot with fast pans that can cause streaking. Very bight highlights show up issues with sensors that you may not notice unless you move the camera around quickly. These same issues will cause issues for the encoder as explained above so the problem gets worse as the image goes through the pipeline.

"is it a format conversion artifact?"

Probably but without actually seeing it, it is hard to know for sure.
I've been caught out many times when people have complained about certain artifacts that to some extent are natural. Most of us just don't normally "see" things like motion blur so if you are concerned it really pays dividends to provide an example.

Bob.