Topfield .rec output in beta 459

[dedics]

Hi Dan,
had a brief go with this last night and it seems to be ok now.

well done!

I'll do a bit more testing today.

and get back to you.

dedics

[mark0326]

I have tried this on a couple of mpg files and get no sound and a black picture, I tried the test file from the ftp site and that worked.
Are there any special settings I should be making in the conversion?

[DanR]

Which test file did you try?
Are you saving as a .rec file?
Which topfield do you have?
Are you opening and saving a .rec or converting a non topfield to a .rec?

Thanks.

[dedics]

OK I've done a bit more testing

I'm on the topfield 5800 in the uk

I recorded a 2 minute file on my toppy test3

then I copied this to the pc

I then used vrd459

to give test3459 this was a straight read the rec file in and save it out again as a .rec file: this played ok on the toppy

then I saved the original file as an mpg file test3459.mpg

this mpg I then reloaded into vrd and saved out as a rec file test3459mpg

this also played ok

however when I qsf'd the mpg file and saved it out as a test3459qsf (as a single operation)

I got the blank screen situation on the toppy.

So there's something amiss with the way qsf works with the topfield header generation.

It may just not be quite consistent yet throughout the whole vrd program.. but then that's what betas are for isn't it.


I'll upload these files to the ftp site for you to look at.

I also tried another test
test1459 which I read into vrd and saved out as a .rec file (worked ok)

theres also another invalid test file which I couldn't get to play at all which was test1459 saved as an mpg then reloaded and saved out as a .rec file

this is test2459.rec this won't play at all, even if I run it through qsf.


I'll put them in a new folder tf5800vrd459test

Hope these files are useful.

dedics

[mark0326]

Which test file did you try?
Are you saving as a .rec file?
Which topfield do you have?
Are you opening and saving a .rec or converting a non topfield to a .rec?

The file I downloaded was 'VrdTest5800_b.rec' from the ftp folder 'beta456 tf5800 tests t2210' which worked properly.
I also tried 2 other files from my HD which were not recorded on my TF5800 (UK), one was a TmpgEnc created .mpg file, the other a .mpa file both of which were converted to .rec in VRD.
Shouldn't we be specifying what the output file should be for, 5000 or 5800?
Hope this answers your questions.

[dedics]

HI Dan I'm still uploading files for you,

I've looked at it a bit further and it seems that if you save your toppy.rec file in any other format out of videoredo, like .mpg or .vob
then re-read it in and re mux out to a .rec format the file is invalid, presumably it loses some header info during this process.

The files usually come up with content but 0 minutes and won't play or play a blank screen.

I've tried qsf from both the original files saved to .mpg and reading the file without loading it into vrd first, just qsf the file and saving as a .rec or an mpg but it doesn't recover the file.

It doesn't seem to matter if the original file is a straight recorded file from the toppy with no cuts or edits or if you have edited the file in vrd before saving.

The good news is that you can read in a .rec file from the toppy and edit the adverts out and save as a .rec file which you can then re-load onto the toppy.

So we're nearly there, it's just what vrd does if you save out to mpg then reload that,that we need to get right, I can see people wanting to convert their camcorder footage or other captured stuff to .rec files to show on the toppy.

I'm just going to try qsf on a short.rec file and save it out as a .rec file to see if that works.

I'll get back to you.

dedics

[dedics]

the qsf file saving to rec file from an original rec file works fine,

qsf of a .mpg file to .rec does not.

hope this helps

I'm still uploading files for you.

dedics

[HyperReality]

Are there any 5000 owners out there who have done any testing? Anyone?

To be honest, I'm not too keen on going out and buying a Toppie just to test this stuff.... ;)

[TF5000UserDE]

I purchased VRD in Jan. 2005 and at once became great fan of VRD. I use it mainly for cutting out commercials from .rec files of my TF5000PVRt (European Vers.), which is similar to the UK 5800. Since beta 459 I can save back the cutted files as valid .rec files to HDD or rather to PVR and they will be played back on TV normally. Well done! as dedics said.
By the way @ Dan: is it right that 459 doesn't include the 458 Setup files for German and French versions ? I don't need it under any circumstances but it's nice to have ;) On occasion I will send some feedback for the translations to German. A german Version is a good presupposition to be noticed by german PC-Magazines !

[DanR]

We will get the German and French 459 posted in the next day or two.

Can you load an MPEG file (i.e. note a .rec ) and save it as a .rec that works properly?

[geoffgmf]

Hello Dan,

Unfortunately I've had no success with your beta 459 changes on an Australian issue TF5000pvrt. I took a vob file from a dvd and converted it to .rec, but it appears as an invalid file on the Topfield.

I don't know if you're already aware of this, but there is a program to convert ts files to .rec for the 5000. If I turn the vob file into a TS file in VRD, the run it through the converter program so it outputs as .rec, the 5000pvrt will recognise the file as valid and play it with no problems.

I have no idea how it works, but maybe you can see the source code for it, which might give you some clues re. the missing piece in the jigsaw. Here is the link:http://home.swiftdsl.com.au/~peteru/toppy/index.html

Regards,

Geoff F (Australia)

[DanR]

Thanks Geoff,

This should be such a simple solution, but we were dealing with a near lack of accurate information.

First we had the Topfield .rec format document. It had mistakes in it :(

Then we found some accurate information and coded it up. However the first user to test it had a 5800 not a 5000. We realize that there was a difference in devices or .rec formats for a bit :(

So, now we have a 5800 almost working and the 5000 version is broken :(

A program called TFHead.exe, which displays the Topfield header, is unclear about which .rec version it supports.

Had Toppy sold these units in the USA we would have purchased a couple and had the issues sorted out in no time. Anyway, we will put the 5000 option back into the next beta (or maybe the one after that).

Will check out the program you posted. Maybe it can help us.

Thanks

[TF5000UserDE]

Can you load an MPEG file (i.e. note a .rec ) and save it as a .rec that works properly?

Hello Dan,

I'm sorry, but same result on my TF5000PVRt: mpg ---> .rec doesn't work.
Invalid file with zero minutes length in the display window.

[HyperReality]

A wealth of information about the TF5000PVRt version sold in Australia is linked from here (http://www.dtvforum.info/index.php?showtopic=3684), including links to various other bits of peteru's and other peoples work.

[dedics]

Hi Dan,
I took a small section of a toppy 5800 .rec file saved it as a test5.ts and then used peteru's convert to fix it (test5.rec). then I also read the ts file into vrd459 and saved it out as test5ts2rec.rec file, the straight save of the .ts file from vrd did not play and was half the size in MB of the original and peteru's corrected one.

Shall I upload these files to the ftp site?

I have found peteru's correction program has worked for me on the toppy 5800 in the past.

dedics

[stubbsy]

I'm another Australian Toppy 5000 user and as already mentioned a non toppy mpg (I used a dolby digital vob trailer) saved as rec doesn't work. If I put the file through ProjectX to add a 5000 series header it does work.

One thing I can do if it's useful is to upload the first x bytes of the VRD generated .rec along with the first x bytes of the same source file, but with ProjectX adding the 5000 series header so you can compare headers. If so just tell me where to put the files.

[phd]

Thanks, we believe we have a handle on it now.

[DanR]

FYI, There's a new build to try, 460 that should handle both 5000 and 5800. Please see the beta release notes for more details.

Thanks.

[stubbsy]

Sorry Dan. Just installed and tried the new beta with my test vob file (a dolby digital trailer vob) and it still won't work on my TF5000. Looking at the file it still seems the header is different to what projectX adds. On my Toppy it shows as "Invalid File" and with a length of 0 minutes

[dedics]

hi Dan, dedics here again, on topfield 5800

I ran some tests on you 460 beta but It's still not working.

This is what I did:

selected tf5800 output in the tools/options/window

took a topfield.rec file
went about 1minute in and took 2minute section from that.

I saved this as a .rec file (test460orig.rec)
then I saved the same segment as an mpg (test460mpg.mpg)
then I read this mpg file back in and saved it out as a .rec file again (test460mpgrec.rec)

neither of the rec files played on my topfield just blank screens and invalid files....

I then tried selecting the tf5000 output in the tools/options window

Following the same procedure as above I created 3 more files

test4605000orig.rec this one played fine
test4605000mpg.mpg
and
test4605000mpgrec.rec this one did not.

have you got your pointers for the options screen mixed up?

I'll upload these files to a new folder
tf5800460tests OK ?

dedics

[dedics]

I've tried a few other things now,
if you resave a testfile thats been saved with a tf5000 header and then change the header option to 5800, the file plays ok.

Correct playback doesn't depend on whether you start from the very beginning of a file or you trim off a bit at the beginning and then use a segment.

if you set the header to vrd tf5000 it works to save a rec file from a rec file.
the vrd tf5800 header file does not.

but if you try to save from an mpg file neither file header works.

hope this helps

dedics

[DanR]

There's a file on our FTP site: VRDTopfield.zip. Please download and overwrite the Videoredo.exe in your Build 460 folder with its contents. There was indeed an error in determining whether to write a 5000 header or 5800 header.

Thanks.

[dedics]

I've tried that, dan
and run the tests again using the new vrd file for the tf5800 option,

both files are not playing correctly and show up as invalid files.

Also when they transfer to the toppy they cause the Altair program to lose sight of them on the toppy until you close and restart the altair ftp program again.

do you want me to upload these files?

I don't think you've fixed the problem, neither the 5000 or the 5800 .rec files work now!

dedics

[DanR]

No, I'll try again. I'm sorry for the all the back and forth. This is why we like to have the actual hardware in our lab so we can shake these things out before you guys see it.

[DanR]

If you open a 5800 .rec file, edit and save it, is it bad?

[dedics]

fraid so,

none of the .rec saves seems to work now, I tried both...5800 and 5000

dedics

[DanR]

OK, thanks.

[dedics]

I'm happy to help with running tests,

It can't be easy if you don't have accurate documentation and the hardware to play with...

It's also a shame about the time zone difference, I'm halfway through the evening here....

It's tantalisingly close now.. I'm sorry I can't shed anymore light on the header for you, I'm just a user of the topfield and your program.

(but I did train as an electronics engineer so I appreciate the design process!)

There are other topfield users out there waiting for this too though.
good luck

dedics

[DanR]

Does the 5800 read the 5000 .rec headers or does it need its own custom header?

[dedics]

I'm not sure,
when I corrected one of the earlier test files with projectX
I tried correcting with both the t5x00 and the t5000 header added, and both of these played successfully on the 5800, with the most uptodate firmware.

but if there are two slightly different headers in projectX for the two machines, it would imply that there are some differences:...

I'll try and run a couple more tests, and upload the results.

dedics

[dedics]

right:
both the added topfield headers for tf5000 and tf5x00 produce working files on my 5800, from an original saved tf5800 file made by vrd460(your latest.zip file)

The names are self explanatory, I'll upload them and the projectX logfile from when they were created.

I'll put them in dedicstest460 folder.

dedics

[DanR]

Please give VRDTopfield2.zip a try. Its in the root directory of the FTP site.

Thanks.

[dedics]

ok I've tried it, and it works for both the 5000.rec and the 5800.rec, with the 5800.rec file, the mystuff TAP(= additional user loaded software) archive screen only shows this file up after it has been played from the topfield basic archive screen.

with the 5000.rec file, the mystuff archive shows it correctly, so there's something which the 5800 header is doing differently to the 5000 one in this case.

I'll ask the tap writer why this happens maybe he can shed some light on the subject.

Neither of the reconstructed from mpg .rec files was valid, suggesting that the rewriting of the metadata is incorrect still.

Anyway this is better than the previous situation.

ONe of the problems with topfields, is that the uk and australian machines run on different firmwares, and can need to be backwards compatible with previous firmwares for those who don't behave like excessive computer geeks,
maybe this is where the discrepancy in the header files is creeping in.

At present on my topfield with my firmware( the latest OTA download which i what most people are using, but not all) the 5000 header works fine and the 5800 header works too but with some slight differences in readability for TAPs

I'll get back to you on the mystuff header reading thing.

dedics

[dedics]

ok dan here is the reply I got from the writer of the Mystuff tap

"Hi dedics.

Odd that I dont display it, but there are a couple of checks I do to kick out any .recs which I think are duff.

1) File data must start with "TFrc" (no quotes)
2) The file size as stored in the header must not be 0. I use (hopefully correctly!) the 9th and 10th byte of the file, so one of these must not be 0

Why playing it would make a difference I dont know, unless it changes the header (actually, it will with the new f/w, as it will save the last played position to the file, and so may also update the header)

HTH

Cheers"

Having gone out and turned everything off again before doing so, when I re-booted everything, neither of the two playable files showed up in the mystuff archive page until after they had been played successfully from the native topfield archive screen.

After this is appeared correctly in both lists.

dedics

[dedics]

I've compared the file headers in a hex viewer and can see no difference between the hex code for the original and the played file, so the showing up in different archive screens is still a mystery.

However if you look at the header vrd creates from a .rec file, and what projectX puts when it corrects the header form the same rec file converted from an mpg they are very different.

I'll upload the text files with the headers in to the ftp site, in the latest folder.

dedics

[DanR]

Thanks, can you ask the MyStuff guy is there is an accurate description of the 5800 or 5800 .rec header?

[DanR]

Please give VRDTopfield3 a try. Especially the conversion from non-.rec to .rec.

Thanks.

[dedics]

yep, will do.

I'm corresponding with the guy who gave you the info about the topfield header (it's on the ftp site, in the first set of test files folder I did for you I think.(beta456 tf5800 tests t2210))

I'll see if there's anything else he knows.

dedics

[dedics]

HI Dan,
seems like topfield3.zip is better.

I ran the same tests again, saving a portion of a .rec file to .mpg and then taking that .mpg file and saving it back to .rec and all was well !


this was using the 5800 header option, I also tried the 5000 option and that worked too.

I also tried saving to vob and back and that worked too.

I used a slightly longer test file this time, and the issue with the mystuff tap not seeing the file was down to the actual length of the file being very short previously.

I'll save out and back to all the other formats to check there's no other funnies.

and check qsf.


dedics

[dedics]

All good so far...

I've saved out to mpg,vob,tivo,dvr-ms and ts and all files re-load and can be re-saved in.rec 5800 format and all play fine.

I've also tried qsf ing an .mpg to .rec and a qsf of .rec to mpg which when re-saved as a rec plays ok too. There didn't seem to be any buffer underflow problems either, which I was getting with the first vrdtopfield.zip file.

I think you've cracked it.

The message showing what time the file was edited is showing up correctly in the info box when you play the file too.

now all you need is a tf5000 owner to run these tests for you.

I haven't really got any mpg files created by other means (non topfield)to try in vrd unless I demux something and re-mux it in project x.


Any other tests you need? let me know, I haven't tried editing the reconverted files but I would be surprised if that's the problem.

dedics

[wellsie]

I'll do a test to convert a non Topfield recording to .REC output and report back :o)

[wellsie]

OK, I had a transport stream recorded using a Hauppauge Nova-T PCI card. This was passed through QSF to create an mpeg which was then edited to create a final mpeg. This was all done using the exe file from VRDTopfield.zip. Then I created a .REC output from the final mpeg file using the exe from VRDTopfield3.zip. There were buffer underruns during this (see attached log). The .REC file was then copied to the TF5800 which played it back fine.

However, the latest firmware release for the TF5800 last December was updated to add a last played position into the Topfield header so that you could continue playback from a previous point. This resume playback feature does not work with the non-native .REC file - it always plays back from the beginning so something is still not quite right in the header.

A native .REC file pulled from the TF5800, edited with VRD and put back onto the TF5800 does continue to resume playback correctly.

If there are any other tests I can help with please let me know.

[geoffgmf]

I'm a TF5000 Australian issue user who's happy to test VRDTopfield3.zip, if someone can send me the ftp site address?

Thanks

Geoff F

[HyperReality]

ftp://upload:upload@videoredo.net/

The zip file is in the top level directory

[geoffgmf]

Thanks Hyper.

Houston, - we have lift off. It worked beautifully for me when I transformed a vob file from dvd to TF5000 .rec file and played back on Australian TF5000. Well done to Dan R and all other testers/contributors.

[stubbsy]

The new VRDTopfield3 worked perfectly on my Australian Toppy too (5000 series) when I converted my test dolby digital tariler VOB file. Looks like you've nailed it.

[bohemian]

I am a an Australian 5000PVRt user, and just tried the beta VRDTopfield3 on a couple of files.
It worked fine on a rec file i edited then transfered back to the toppy, and also worked on a transport stream file i recorded on my PC.
both files played back fine. good stuff:)

[DanR]

That's wonderful news on the Australian TF5000. Am I correct in understanding that the only problem outstanding problem is the "playback resume" position on the TF5800 files when VRD creates the headers.

One thing I'm thinking of adding is the ability to customize the metadata in the .rec header similar to what we now do for DVRMS files. In this scheme, you specify a metadata template for short and long event text. The template has 3 predefined substitution tokens. date, time and orig. Orig means original text from the source file (if such text exists).

Example:
Original meta data: "Cold Case"
Template: {ORIG}, edited on: {DATE} at {TIME}
Meta data in output file: "Cold Case, edited on: 31-03-2006 at 08:06 AM"

Note: Date and time use the machines default date and time format.

Would this be useful or are we just wasting our time?

[wellsie]

Yes, Dan, the only problem outstanding for the TF5800 files is the "playback resume" position.

As for the customised metadata, I doubt I would personally use it but others may well do and it seems like a nice feature to have.

[stubbsy]

One thing I'm thinking of adding is the ability to customize the metadata in the .rec header similar to what we now do for DVRMS files. In this scheme, you specify a metadata template for short and long event text. The template has 3 predefined substitution tokens. date, time and orig. Orig means original text from the source file (if such text exists).

Example:
Original meta data: "Cold Case"
Template: {ORIG}, edited on: {DATE} at {TIME}
Meta data in output file: "Cold Case, edited on: 31-03-2006 at 08:06 AM"

Note: Date and time use the machines default date and time format.

Would this be useful or are we just wasting our time?

Dan - that would be absolutely brilliant if it's not too much work for you. This would also let us add similar text for material that didn't originate on the Toppy. And thanks for getting this all working so far.

[geoffgmf]

I would be an occasional user of the header feature you've just described Dan.

One enhancement I would be a very enthusiastic user of is this: the ability to create .rec files from non-Toppy video files for the Australian TF5000 with a Dolby Digital soundtrack in addition to (or instead of) the normal mpeg audio. I know the TF5000 can output a 5.1 audio track via its optical s/pdif output because once in a blue moon a broadcaster here will transmit a show with a 5.1 soundtrack. I have a short recording of such an event if you'd like me to upload it to your ftp site.

In a broader sense, it would be great if VRD had a DD audio option for all video file creations. To give you an example, using VRD I recently transferred some Topfield shows onto DVD to give to a friend who had missed the broadcasts and doesn't have a Topfield. VRD of course used the Topfield's mpeg audio, but I found by saving the Topfield files as demuxed transport streams I was then able to use DVDAuthorGUI to add a DD 2 channel audio stream as well.

Why would I bother? For this reason: if a user has their dvd player set to output a bitstream and only has a digital cable connection to their home theatre receiver (i.e. no 2 channel analog cables connected), they won't get any audio from the mpeg soundtrack unless they change their dvd player to output PCM (a bit technical for the average Joe).