Incorrect durations on TS->DVR-MS conversions

[idbirch]

I know there is an 'incorrect duration' related thread already but I thought it different enough from my circumstances to warrant a new thread.

I am in the process of converting a batch of BBC-HD H264 programs into HD MPEG2 .ts streams. VideoRedo performs my last 3 steps being:

1) Correct audio sync
2) Re-save as .ts instead of .mpg (for archiving purposes)
3) Re-save as DVR-MS (for 360 streaming)

It is the DVR-MS saving which screws up the duration (at least the duration as perceived my MCE and the 360). The .TS file which VRD has created itself has the correct duration of somewhere between 58 and 59 minutes and even when it is loaded into VRD, the correct duartion is shown in the bottom right. Once the DVR-MS is made, it plays absolutely fine and the inaccurate duration doesn't seem to affect playback at all but all 3 of the episodes I have done so far show as somewhere between 1h13 and 1h17, so about 15 mins longer than they really are. What's going on there?

[DanR]

We will take a look at this. Perhaps you can do us a favor. Take one of the TS files of 58+ min and save 30 minutes of it in DVR-MS format, and let us know what the 360 says the duration is.

Also, have you tried opening any of the dvr-ms files in WMP to see what it thinks the length is?

[idbirch]

OK Dan, thanks for the help. I saved just the first 30 mins of one of these TS files to DVR-MS and MCE and my 360 and Windows Media Player 11 all see it as 38 mins. Hmm.

I'm not sure if you know this already but I discovered this in co-operation with Crypto over at DVBPortal - if you take a file that doesn't natively have any metadata attached, such as one of my TS conversions and save it as a DVR-MS, the playback progress indicator will be screwed up in MCE. By screwed up I mean the duration shows as about 3x what it really is and the last 2/3 is greyed out. For example, with a 1 hour show, MCE will show a slider that represents a 3 hour show that only recorded for the first hour.

For this reason, in all my converts so far, once DVR-MS output is complete I have to right-click the DVR-MS file, go to Properties -> Summary and enter something in the "Title" field. Once this metadata is stored, the MCE slider suddenly displays correctly. Bizarre.

Interestingly, the duration inaccuracy also show in the Properties -> Summary -> Advanced window. I have a sneaking suspicion that this is all metadata's fault - damn that pesky metadata!

[jpconard]

Good catch, yes if you add a title to the summary it will fix the slider.

I'm not sure it is the fault of the duration or Original Run time fields in the metadata. Both show the seconds times 10,000,000 or something like that and appear to be correct.

There is a metadata editor on the web.

Now, are you able to edit DVR-MS back to DVR-MS? Mine gets in a loop or something at the end and will not finish. I thought I could do this before, but maybe I always went to MPEG then converted back to DVR-MS.

My WMP10 shows the correct duration even before putting an entry into the title field.

[phd]

Now, are you able to edit DVR-MS back to DVR-MS? Mine gets in a loop or something at the end and will not finish.
Where are you seeing this behavior? In VideoReDo or in MCE after saving the edited file?

[jpconard]

In VideoRedo. Maybe the files that I am starting with have too many glitches?

They save to MPEG just fine but did indicate several errors. Then I could take them from MPEG to DVR-MS just fine, except the slider would be off.

Then with the suggestion above of right clicking the file and entering a value in the Title the MCE slider is then corrected.

I can live with the two step process of going to MPEG then to DVR-MS. It has been a while since using VideoRedo and I thought I did it before but maybe my recollection is wrong.

Maybe I should run the Quickstream Fix on the original DVR-MS file then start my edits on the resulting file and then try saving to DVR-MS? I will try this tonight.

By the way I posted in the support area, but then found this post with problems similar to mine.

Also this is a great product! The recent updates are nice, I particularly like the Favorites feature so that you can easily get to different folders.

[phd]

Your files may need a QuickStream Fix first.

[idbirch]

Any ideas on the original problem I reported?

[idbirch]

Still no ideas on this problem? It is now worse as I have found with alot of HD dvr-ms files, adding a title to the metadata doesn't fix the problem either (I am now on Vista). There is no problem with the slider, it is completely filled but as an example, I have converted the Kingdom of Heaven BluRay MPEG2 stream to a DVR-MS and the reported duration is about 4h50m. The true duration should only be 3h10m.

This wouldn't be that much of a problem if the duration field were actually editable on a DVR-MS file because then I could correct it myself, but it isn't. Seeing as the original MPEG2 stream shows the correct duration in all players but the dvr-ms does not, I'm pretty sure VRD is to blame for this one. Any thoughts?

I have uploaded a 100MB, 28 second sample to a folder called idbirch on the FTP. Please run it through the latest build of VRD on a Vista machine and then check the duration reported by explorer and Media Center.

[idbirch]

OK, I have yet more information for you on this problem. It would appear that the DVR-MS outputted by VRD holds 2 values for the duration, one is correct and other is not. I noticed that if I hit (i) while watching a movie I see the correct duration appear:

http://img463.imageshack.us/img463/4926/correcttimepc2.jpg (http://imageshack.us)

Under the title of the movie, we see the correct duration of 3 hours 10 minutes. The problem is that if I FF/RW or pause, the slider thinks the film is much longer:

http://img463.imageshack.us/img463/1426/wrongtimeho0.jpg (http://imageshack.us)

4 hours 52 minutes. I figured this must be coming from metadata so I downloaded a free DVR-MS metadata editor and found the following:

http://img467.imageshack.us/img467/8019/metadatavz6.jpg (http://imageshack.us)

See the value of 175021050000? I don't know what the last digits represent but the first 5 digits are the duration is seconds:

17502 seconds / 60 / 60 = 4.862 hours

So the question is, where does this value of 17502 come from? I have tried changing it using the above editor but get an error saying that read-only attributes cannot be changed. And what about the correct duration, where does that come from? It seems to be decided by the percieved start and finish times of the recording as if I look in "Video Details" it says the recording took place on 11/06/07 from 04:08 to 07:17. This isn't the case of course as this MPEG2 stream is sourced from a BluRay, not recorded from TV, so how do these times get set? Does this info help at all?

[DanR]

What version of VideoReDo are you using?

If you use VideoReDo to cut the file in 1/2, what happens to metadata values in the new file?

Using your sample, I get correct file durations, but incorrect end-record time. Very strange. Also, I don't see the little HD symbol.

[idbirch]

I'm using 2.5.5.516.

I just ran another of the 100mb chunks through VRD. The true duration of the whole file is 31 seconds, which VRD correctly identifies when first opening it. After outputting to dvr-ms, explorer and Media Center now think it is 49 seconds. If I cut it in half, choosing to convert only 15 seconds, explorer and Media Center see 25 seconds.

When you say you get the correct file durations do you mean that in Vista explorer, in the status bar at the bottom, it shows the correct duration? And what do you mean by end-record time? The start and finish times will be incorrect as this is not a recording so that's OK.

The HD symbol is only there becuase I set the HD attribute to true in that DVR-MS metadata editor I showed above.

edit: I thought maybe this has something to do with video codecs on my machine but I have just unregistered the nvidia, cyberlink, nero and sonic decoders and run the test again and I still get the same result.

[DanR]

idbirch (or anyone else): Can you upload an NTSC HD sample recorded either off the air or cable? Duration can be anyway from 30 secs to a min, and please make sure that its not copy protected.

There seems to be something wierd with the output of NTSC HD material in DVR-MS format. We are able to duplicate your problem with other HD material but not SD material. I'll need to get some samples to our DVR-MS developer.

[idbirch]

Unfortunately I can't provide any actual recordings as I live in the UK and only receive DVB-T or 'Freeview' which is SD only. The only HD content I can get my hands on is from HD-DVD or BluRay discs. I think this will require someone in the states to help out.

If anyone reading this has Vista, uses Media Center and can record HD content direct to DVR-MS, would you be so kind as to upoad a sample to the FTP?

edit: I think I may have found at least one sample that someone else uploaded for the mplayer devs ages ago, I'm trying to download it now but it's on Megaupload.com and its sloooow. I've made a new folder on the FTP called idbirch and have uploaded 2 samples. I have found more to download but they're in excess of 200MB and downloading at about 20k/s - go megaupload! These are from American TV and may be quite old but hopefully, they're of use.

[DanR]

Thanks for the samples, but they all work fine including the one HD one.

Will have to think about this some more.

[idbirch]

Ah, sorry, I thought they were all HD samples. OK, I have quite a large collection of HD movies and some of them get the correct durations when run through VRD and some don't. I will find a few examples of each and upload them to the FTP later today.

Just for your information though, the Kingdom of Heaven movie shown is directly off a BluRay disc (was already in MPEG2 on the disc) and the Pirates one is re-encoded from a H264 BluRay disc. The main thing in common is ridiculously high bitrates, could that be anything to do with it?

The Pirates movies I re-encoded with HCEnc with an average bitrate of 22mbps and max 30mbps. The Kingdom move is probably about the same or a bit higher.

[DanR]

I don't have a Kingdom of Heaven clip. Can you try re-encoding Pirates without using repeat flags? I'm thinking it may be related to the use of repeat flags.

[idbirch]

OK, I think we're getting our wires crossed now. Repeat flags are an attribute of DVR-MS files yes? I'm not encoding directly to dvr-ms, I don't think you can even do that. I'm encoding to a standard MPEG2 .ts file to begin with, obviosuly these have no metadata associated with them at all. I convert .ts files to dvr-ms for easy streaming to my xbox 360. So just to clarify, the content I am converting to dvr-ms is not recorded off the TV, it is either from a BluRay disc or from an MPEG2 .ts file.

Any flags being set in metadata are being set by VRD as it is VRD that takes the MPEG2 .ts file and converts it to dvr-ms. Unless Vista adds some of it's own metadata upon file creation? Obviously I have no control over this either. If I have misunderstood your use of the term 'repeat flags', please explain.

I've started uploading a 100MB segment of Kingdom of Heaven for you which has the same problem. This is not encoded by me but it staight from an MPEG2 steam (.m2ts) off a BluRay. The duration is 33 seconds but after running through VRD and converting to dvr-ms, the duration of the dvr-ms is 50 seconds.

This is the process I use to get the MPEG2 .m2ts file on the BluRay disc into the .ts file:

1) Demuxed the video and audio using xport
2) Remuxed with Manzanita Mp2tsms 2.13.
3) The resulting .mpg file is run through VRD to remove null packets and save as a .ts.
4) The new .ts file (which shows the correct duration when opened in VRD) is run through VRD a final time to convert to dvr-ms.

The resulting dvr-ms file shows to be almost 5 hours when it should be 3h10m.

This seems like a tricky problem to troubleshoot. If necessary, I can supply my MSN Messenger ID so that we can discuss this via IM?

New folder on FTP is called idbirch again, file will be done in about 30 mins.

[idbirch]

I have made a discovery. I was experimenting with different 100MB chunks of Pirates 2 and found that with some clips, the duration, although still out, was only slightly out. And it seems these are the clips where the bitrate is lower.

In the very first 100MB of the film, the duration is 54 seconds which means the bitrate must be, on average, about half that of the previous example. The resulting dvr-ms comes out at 57 seconds. It seems the duration is being based on some sort of equation involving the bitrate and the number of frames. That's just a guess but it must have something to do with it?

I'm upoading this first chunk to the FTP now, the KoH clip is already done.

[idbirch]

Well, more progress. The KoH clip I uploaded earlier will come out as about a 1 minute dvr-ms (should be about 30 seconds). I took that exact same clip, ran it through HCEnc and re-encoded to a much lower bitrate (about 5mbps). Everything else was kept the same, same audio, still 1080p video etc. Remuxed with Manzanita, ran through VRD to save as a .ts.

This .ts was then run through VRD again and saved as dvr-ms - the duration was correct! Seeing as the audio content is identical, we can rule that out. The video was re-encoded but everything was kept the same apart from bitrate. Encoder setting were essentially the same as used for the Pirates 2 re-encode, except bitrate of course.

I may be wrong but it seems to me that once the average video bitrate gets above about 18mbps, VRD starts to screw up the dvr-ms output but I have no idea why. It does seem though that the higher the bitrate, the further out the duration is. Does this help?

edit: 2nd pirates sample and low bitrate Koh sample both done.

[idbirch]

Is there any progress on determining the cause of this yet? I have come to realise that this problem also causes seeking problems i.e. the image displayed while seeking is not actually where you are in the file.

[idbirch]

"No" or "Not yet" are valid answers you know, just some sort of aknowledgement would be nice. Is this even being looked at?

[DanR]

Not yet. Its still being looked at. Was waiting to hear back from our DVR-MS developer/consultant. Today he said he would try to get to it within the next few weeks as he has some other assignments that keeping him otherwise occupied.

[idbirch]

OK, thanks for the update.

[idbirch]

I was excited to see the 532 update posted earlier so I've just tried it on an MPEG2 bluray stream but unfortunately the problem remains. I enabled the prompt for metadata and made sure all fields were populated. In this case, the content was Phone Booth on BluRay. The video stream is already mpeg2 on the disc so I just remuxed it with the AC3 audio track and then ran through VRD.

The Manzanita .ts file showed the correct duration of 1h 21m when opened in VRD but the resulting dvr-ms file reports 2h 06m.

I can confirm that this issue causes big problems with seeking because when watching back one of these dvr-ms files, the PC/Extender does accurately show how far you are into the movie, even though the total duration is incorrect, please bear with me while I explain:

Let's say I'm watching Phone Booth. The film is 1h21m but the dvr-ms file thinks it's 2h06m. I have been watching it for exactly 30 minutes. I hit pause and the seeking bar correctly says I'm 30 minutes into the movie but shows the total duration as 2h 06m. The problem comes when I try and seek or skip forward/backwards.

Lets say I press "Skip fwd" - the extender/PC immediately seeks to the 30m30s mark, but because the total duration of the file is much longer than it really is, the 30m30s timestamp is further back in the file hence when pressing skip forward, you actually end up going quite a long way backwards! Very annoying. Hope I explained that clearly....

[DanR]

Did the metadata update work?

[idbirch]

If you mean did the new metadata feature populate the metadata properly, yes but it does not resolve the problem of incorrect durations.

[DanR]

idbirch,

I'm having problem replicating with 534 the problems with incorrect durations now. Are you still having the problem?

[idbirch]

Yeah, problem is still exactly the same with 534. Have you tested the Kingdom of Heaven clip on a Vista box?

[idbirch]

Any progress on figuring this one out yet?

[idbirch]

You fixed it! Thanks very much for getting round to this one, I've just tested a couple of HD re-encodes I did and the durations are coming out correctly. I can see from the log file that the duration patching is being done at the end of each conversion.

All I have to do now is convert about 100 movies back to .ts and then back to dvr-ms again - is there any way an option to patch existing dvr-ms file durations can be added to save me this massive job? e.g. open a dvr-ms with VRD and then go Tools -> Patch dvr-ms duration -> Enter new duration -> Go. Thanks.

[DanR]

All I have to do now is convert about 100 movies back to .ts and then back to dvr-ms again


When you open the "bad duration" DVR-MS files in TVSuite does it show the correct durations? If so, simply run QSF on the whole lot, DVR-MS to DVR-MS no need to go through .ts conversion. You can set this up in the batch manager quite easily so it will run unattended.

[idbirch]

Yeah, I already assumed that VRD relied on the duration of the source file being correct but as mentioned earlier in this thread, VRD also sees the incorrect duraion whne opening the dvr-ms so QSF or re-saving to dvr-ms doesn't work. For this reason, conversion to .ts is required. How about the Tools -> Patch Duration option? Do-able?

[navjo]

Yeah, I already assumed that VRD relied on the duration of the source file being correct but as mentioned earlier in this thread, VRD also sees the incorrect duraion whne opening the dvr-ms so QSF or re-saving to dvr-ms doesn't work. For this reason, conversion to .ts is required. How about the Tools -> Patch Duration option? Do-able?

How was it fixed? What do I have to do to get it fixed? Download a new version?

Regards,

John