Yet another sync problem question...

[BDavis]

First off, let me say that I think VideoRedo is one of the nicest video editors I've used. I've played around with a lot of them, and while some of them are very nice feature wise, their interface is usually so clumsy that I can't get anything done quickly. VideoRedo has a nice balance of features and a very fast and easy to use interface...thanks for that!

Flattery aside, I'm still cursed by audio sync problems. I've scanned a number of posts here and elsewhere, and am starting to understand the relationship between PTS and audio syncs, etc. But, let me get down to it already.

When I capture a video (WinTV PVR-250 card) that's about 1:30 hours, I tend to end up with about 20 or so frames of dropped video (I uploaded a log of one of these runs to the FTP site...it has BDavis in the name). It seems that video frames drop about every 20 minutes or so, which alone is irritating.

VideoRedo claims to have found them and fix them (I'm now using build 233, or whichever came out a few days ago), but whenever I import them into Adobe Encore, the audio and video streams have different lengths (up to a second!) which, of course, leads to audio sync problems.

If VideoRedo is truly dropping audio "frames" to match video, why are the streams of different lengths? I tried toggling the Moviefactory switch, and it seemed "closer" to being in sync when it was on, but still no cigar.

Is there something I'm missing here? Shouldn't the stream lengths be close (like within 1 or 2 frames)? When I capture TV channels, they typically import with that much accuracy (and VideoRedo doesn't detect any dropped frames). Or, perhaps, is VideoRedo merely adjusting the PTS and not actually dropping the frames?

Second question, somewhat related: is there an MPEG player or utility that will play my MPEG video in such a way that will highlight these audio sync problems without having to render a DVD file first? Essentially, a "stupid" video player that ignores PTS (or whatever) just like the DVD player does? That way I could test for audio sync failures immediately after capture, rather than taking the time to import the clip into Encore, build a folder, play it in DVD software, and see that my audio is WAY off?

Okay...ramblings over for now....

[DanR]

Thanks for the kind words about VideoReDo.

Regarding the sync issue, VideoReDo is indeed removing frames as the log file indicates. The question is why? Typically audio frames are removed when there are discontinuities in the video stream (i.e. dropped frames). But VideoReDo shows no sign of that. However, if you look at the end of the log file you'll see a total 1/2 second difference between the number of audio and video frames: Video Frames: 169469 (5654.616 sec), Audio Frames: 235590 (5654.160 sec)This 1/2 second difference represents exactly the # of audio frames removed.

The question is, why were those frames removed since there didn't appear to be any video errors. My guess is that this is due to clock drift within the PVR or its drivers. In your 1.5 hr program, the video clocks (PTS) as output by PVR drifts by 31 msec, not an audible difference. However your audio clocks are off by more than 500 msec. If you look at the log file where it say "Audio BUffer: At EOS, No final Encode..." you'll see that the audio clock is subtle shifting by 0.01 msec every 4-5 audio frames, the orig PTS is coming from the actual timestamps in the file. If you do the math it would indicate a total audio error of about 500+ msec, exactly what VideoReDo detected.

My questions are: (1) Does the original program play in sync with VideoReDo, especially near the end?
(2) What about the version output by VideoReDo?
(3) What are the audio sampling freq and bitrates? (Tools>Show Program Info)

You might try turning off re-sync (Tools>Options>Advanced Stream) and seeing what happens.

Any chance you could upload the last 20 MB or so of the source clip using Tools>Trim to our FTP site? I'd like to confirm what's going on.

[BDavis]

I am currently uploading the last 20 MB of a video file, and the debug log that matches it. Unfortunately, due to PC difficulties, the video file that originally generated that log is not available anymore, but this is from a new capture that really exhibits all the same signs as the log file that was sent before. Essentially, I see this every time I capture from this camcorder...but, as I mentioned, when I capture from TV, everything seems fine....

Now, to your questions:


My questions are: (1) Does the original program play in sync with VideoReDo, especially near the end?


Yes, everything plays perfectly in sync on any Windows player, including VideoRedo. It's only after the DVD software (MovieFactory or Encore) get their paws on it that the audio is noticeably out of sync.


(2) What about the version output by VideoReDo?


Since that was an old log file, the version was probably a little older. This most recent clip that's being uploaded was edited with build 231.


(3) What are the audio sampling freq and bitrates? (Tools>Show Program Info)


Video FPS: 29.97 (as expected),
VBV_Buffer: 1792KB
Video Bit rate: 9 MBPS (that's the max I set)
Audio Bit rate: 384 Kbps
Sampling rate: 48 khz
Audio is (if memory serves) MPEG-1 Layer 2 (the only kind PVR-250 can output).


You might try turning off re-sync (Tools>Options>Advanced Stream) and seeing what happens.

Any chance you could upload the last 20 MB or so of the source clip using Tools>Trim to our FTP site? I'd like to confirm what's going on.

I tried to turn of re-syncing, and the streams still differ in length by about 10 frames, which, now that you've explained a little what's going on, seems to be what the logs show (1/3 sec difference....just enough to really trick your brain into realizing the audio is off....grrr). The final clip, in this case, was rendered to only about 1 hr 15 minutes. I'm sure if it were closer to 2 hours, the difference would be more pronounced.

As a side note, perhaps to others reading this post, I actually just upgraded my CPU/Motherboard. I was running a 1 GHz Athlon, now I'm running a 3 GHz P4. The effects are still the same...

Let me know if I can provide anything else for you. I'm anxious to get a solution to this problem, and your response is the closest I've seen to an explanation of whats going on. From what I can tell (based on various forums on the net) many WinTV PVR users are experiencing these problems (and everyone's got their own pet solutions that seem to be equally innefective). I'd be happy to capture more video at different rates (video or audio) if that'd help with the diagnosis.

Thanks for the help and the quick response!
-- Brian

[DanR]

Hello Brian, Thanks for the upload and information. I'll take a look at it over the weekend. Lots of stuff to do outside today, so probably won't get much time to spend on it until late Sunday.

1/3 sec ( 333 msec) difference is dramatic. If differs from person to person, but the average person detects sync differences at ~100 msec. Just for kicks, you can play with the manual audio sync adjustment in VideoReDo to detect your own sensitivity. Find a scene with lots of lip sync (I use newcasts and talk shows).

I look foward to looking at your uploaded file. Hopefully that will shed some insight.

[BDavis]

Thanks for looking at it, Dan. Here's another data point for you, whatever its worth:

After experimenting a little (well, a lot actually), I found that if I:

- Turn on Audio Correction inside of VideoRedo,
- Save the rendered video as elementary streams (which I was doing anyway, because I need to edit the audio to boost the levels),
- Import the VIDEO stream into Encore, which tells me how long IT thinks the stream is (who knows what it's basing it on)
- and stretch the audio stream to match the length that Encore tells me, I'm synced (not at all awkward or tedious to do...)

Let me give you an example: the final video edit of the file I sent turns out to be 1;12;38;24 (hr;min;sec;frame in 29.97 fps drop code), according to Encore. VideoRedo tells me that it's 1;12;38;18. The audio in Encore and Wavelab show 1;12;38;15 using drop-frame code.

First, I tried disabling VideoRedo's audio correction feature. Then, I took audio and stretched it to match the video time in Encore (same as reported above). This didn't work -- for once the audio was a good bit ahead of the video. Since each iteration of this test took about 30 minutes of rendering and button clicking, I jumped non-scientifically to turning on audio correction.

With audio correction on, I stretched the audio file to the length of 1;12;38;18, which is what VideoRedo told me the time length was. That was certainly closer than before once I rendered the disc, but still off enough to be noticed. Then, on a whim, I stretched the audio to 1;12;38;24, rendered the disc, and everything appears perfectly.

Now, as I mentioned, the scientific method was only partially observed, so the data points are sporadic, to say the least, but perhaps this information will clue you, or someone else reading these posts, in to what might be going on.

As a side note, I was using Wavelab's Time Stretch feature to simply re-sample the audio file to match the time code mentioned ( a little easier than doing the math yourself, especially with the drop-frame time code business). I'm not saying that this is the solution that will work every time, but it worked on this particular video file...

Anyway, it's something to mentally chew on.

-- Brian

[DanR]

Brian, Thanks for the update. I'm still trying to figure out why VideoReDo didn't fix your program's sync. FYI, on the 20mb clip uploaded clip the video frame rate is 29.97255 and the audio frame rate is 23.99773. Not far off, but enough to account for 500 msec after 1.5 hours.

I'm still trying to figure out why the original VideoReDo corrections didn't solve your problem. Is there any chance you can send me a DVD with an entire original program as a file? I'd love to get to the bottom of this.

[BDavis]

Well, a different audio frame rate and a different video frame rate would certainly lead to sync issues.

I'd be happy to send you a full video. I actually just captured another one last night...but it's a little too large to fit on a DVD. If I use the Trim option in VideoRedo, will that preserve all the file data for analysis? (Meaning, if I trim it from 4.7 GB down to 4.3 GB to fit on disc, will that at all modify the stream alignment, perhaps making the problem worse?) If Trim will preserve everything fine, just send me the address to send the disc to.

Oh...and I've now successfully stretched out the audio on two projects and it has worked consistently -- supporting the frame rate mismatch theory.

Another side note to all the spectators out there: I'm using a WinTV-PVR 250 card, with the 1.17.390.22029 version of the drivers (1/29/04) and WinTVCap 2.03 as my capturing software. Perhaps someone knows if this combo is evil....?

[DanR]

Thanks very much, I appreciate it. Addresses sent via a private msg.

The Tools>Trim option is perfect. It doesn't alter the file in any way, just copies bytes so all the goodness and problems are preserved. :)

The frame rate problem is probably not the fault of the PVR-250, but rather the camcorder. The PVR-250 will actually gen-lock to the video frames. Your Camcorder is running about 0.1% fast. No one would notice this and is actually pretty good. I've seen this many times in sample videos supplied by VideoReDo clients although usually the difference is much greater and is caused by noisy source material.

I'm going to hook up a VCR to one of my PVR cards and see if we can simulate the behavior here as well.

[DanR]

If nothing else, I'm gradually learning more about this whole video mess. This sync problem forced me to understand a lot more about MPEG than I knew before, but I'm still confused about something you mentioned in your last response to me in the forum (and if you have an answer, you might as well post it out there so all can see it): How is it that my camcorder is causing frame drift? I'm using an S-Video cable, with analog stereo audio...I would think that the capture card is merely sampling audio at 48khz, and "snapping frames" every 29.97 seconds, regaurdless of the source content. So, at least in my mind, if somehow I fed it video that was 24fps, it would still capture at 29.97 for both audio and video. How is it, as you mentioned, that my camcorder is causing a frame-rate shift in audio/video rates? Certainly this theory seems to fit the problem, as when I capture video from TV and edit out commercials for later viewing, I NEVER have sync problems and VideoRedo never corrects any video, which tends to imply that the camcorder may be the culprit... I don't yet know entirely what's happening, hopfefully once I receive your video I'll understand a bit more. First off, the PVR card will attempt to capture at 29.97 FPS, but it must sync itself to the source material otherwise you'll see the sync bars in the middle of your picture. Think about it for a minute. If you source material is at one frequency and the capture device is at another you want to be sure that the top scan line of the source picture is always at the top of the capture. Therefore, the capture device has to adjust itself to the frequency of the source. Consumer video tapes devices aren't perfect and in fact, your device actually seems pretty good, being that its running within 0.1% of the correct speed. However, over 90 minutes this 0.1% difference can cause a noticeable sync issue.

Now, audio doesn't have these issues, so in the past what I've seen is video being recorded at one frequency, for example 29.97 + or - a bit, but the audio was spot on at 24 FPS. Notice that audio is sampled at at 24 msec intervals or 41.66667 audio frames / second. However, unlike the video there isn't any sync in the source signal, its pure analog. So what often happens is that the video sample varies, but the audio stays constant.

When VideoReDo reassembles the stream it is constantly computing and correcting for the offsets between the audio and video. I.e. is measuring the actual difference between the ideal 29.97 FPS and the actual incoming signal. Likewise on the audio. When these get so far apart that the difference is noticeable, it starts dropping frames to bring things back into sync. This seems to work most of the time, so I'm anxious to get your source DVD and dig into your actual program.

[BDavis]

While poking around a little more today with yet another video source clip, I discovered something interesting that may or may not help you in your quest for knowledge.

I trimmed the front and back off of a 1:21 hour clip, and as always VideoRedo dropped some audio frames for me. When done, the log file shows that the Video Output Frame count is 146548 frames. Inside of VideoRedo, it showed an output clip length of 1:21:29.13. Oddly enough, this is precisely the length of the demuxed audio file I have.

Now, I did a little math, and if I did it right I get (at 29.97 fps): 1:21:29.24 (plus .67 of a frame). Interestingly enough, when I import the clip into Encore, it tells me that it is 1:21:29.24 long as well, and if I stretch the audio to the same length, it syncs up just fine.

If I divide the frame count by 29.96, I get 1:21:29.13. Now, perhaps this is something you already know, but it seems that Encore says, "Forget the fact that you told me it was 29.96, I'm playing it back at 29.97 fps!", so the audio doesn't get scaled.

I wonder -- is the Output Length value in VideoRedo calculated based off of the stream FPS rate, or the NTSC rate of 29.97 drop-fps?

Well, like I said, I don't know if this helps or not, but it certainly struck me as an interesting tidbit. Of course, there may be more going on here than I suspect, as I don't know how the PTS info is stored (seconds? msec? frames? based on fps?).

Anyway, something to stew on. Oh....and I appreciated the extra lesson in video capture -- I suppose I hadn't thought about the need for video synchronization, otherwise the "tear" lines would be visible. If memory serves, a video signal contains a regular blanking interval doesn't it, to which I'm presuming the capture card locks onto. Either way, the stretching of audio in software has proved to be an effective workaround to this problem, and I never would've stumbled onto had you not mentioned that I wasn't dropping audio, but rather slipping my synchronization!

-- Brian

[DanR]

The video frame rate is fixed in the MPEG2 headers to a set of values like, 24, 25, 29.97 and 30 FPS. I think there are some others possible values, but I don't remember them off hand. When you are dealing with elem. streams this is all you have, so you can't set the frame rate to 29.96 or another arbitrary value.

However, when working with program streams, there are PTS timestamps within the file to guide the synchronization of audio and video. What happens when there is a difference between the basic frame rate (i.e. 29.97) and the PTS frame rate is not really defined. Each player makes up its own choices, although from what I've seen, most will key off the PTS values otherwise there would be a difference between playback when randomly seeking and sequential playing.

I don't understand something from your msg. Where does VideoReDo say the clip length is 1:21:29.13? Because, I agree with your math, it should be 1:21:29.24.

[BDavis]

The Output time I mentioned was shown in the bottom righthand side of the VideoRedo window -- where Program:, Selection:, Output: , and Joiner: are all listed (where you can switch between Time and File Position information). Granted this was potentially there prior to me saving the video, but I glanced at it prior to closing VideoRedo and noticed the time as being 13 frames, whereas everything else claimed 24 frames...does that clarify my previous post?

-- Brian

[DanR]

That makes sense, its something I'll have to look at when the source material arrives.

[DanR]

Brian, DVD arrived and is readable. Doing some tests on it now. More later this weekend.

[BDavis]

Dan,

I gave build 239 a try and I think it may have solved the problem! I still need to do some additional testing on other captured video, but the first video I tried to use it with worked flawlessly. VideoRedo didn't throw out any audio frames this time, and, to my utter surprise, the stream lengths (after demuxing) were the same.

As I said, I'll try this new version with some other video and let you know the end results!

-- Brian

[DanR]

Brian, What happened with your testing? Is everything working?

[BDavis]

Dan,

Sorry it took me so long to get back to you...too many projects...too little time.

I recaptured more video and tried to edit and demux them using VideoRedo. The sync problem seems to be gone, even after burning a DVD. I'll continue to play around with it, but even elementary streams have the correct length when imported into Adobe Encore.

Thank you so much for your help! This has made a world of difference! Let me know if you need me to be some other sort of guinea pig for you!

-- Brian