Deleting files.

[KryptoNyte]

The directory where all my saved shows are stored is also the target Monitor Folder for VAP. Sometimes when I just watch a program, it will save the file to this folder. Sometimes I'll schedule a program, it's saved to this directory, and then I may decide to delete the file at a later time without further processing.

By the time I've reached the point where I want to delete the file, usually the QSF and com-detection has already occurred automatically, and temp files have been created.

Is there a way to delete (1) line item in the VAP monitored files list with one click of a button, and have it also get rid of all the associated temp files without having to go in a delete them manually? Or perhaps there is a more efficient method to achieve the same results that I'm just not thinking of?

[dlflannery]

Generally you should only deposit files in the VAP monitor folder if you intend for VAP to process them.

The File Ignore Match Strings feature (tab on advanced config page) allows you to block processing on input files containing specified strings, including extensions, but I don't think that helps you.

You can specify an arbitrarily long delay time between VAP detection and starting processing on the Other tab of advanced config. Don't know if that would work for you or not.

I'll think about some automatic clean up routines for partially processed files, which would have to occur on at least two occasions:
1. When the file is removed from the monitor folder.
2. When you Reset a file that has been partially processed.

[KryptoNyte]

So, you're basically giving thumbs down for a 'delete' button right below the 'reset' button then?

There's a little spot right there for it ... ;)

[dlflannery]

So that's what you want. (It wasn't obvious. ;) )

A Delete button that works as follows would not be too difficult:
1. Delete the input file.
2. Delete files associated with that input file in the Temp(QSF) and output folder (which may be meta-renamed). Note that if the file has completed processing to the point where the final output file has been created, that too will be deleted. Further note that if you have configured to delete or move input files after processing, that input file will no longer even be listed so you can't use the delete button on it -- and whatever output file it created will have to be deleted manually if you don't want to keep it.

The difficult case is if you click the button while the file is being processed. This would require aborting VRD and a clean up that would be very complicated, especially since VRD itself creates temporary files with random names that VAP can't know about. Thus a compromise behavior to avoid all this complication for this case is as follows:

VAP remembers that the delete button was clicked for the file in process but doesn't stop processing that file.
When processing for that file completes, or reaches the AWAITING CUTS status (or processing is stopped by the operator), VAP then deletes the file and the temporary files as described above.

How does this sound? I also invite comments from other users on this.

[KryptoNyte]

... and probably stalled files.

Actually, it would work fine (in my situations at least) if the "delete" button was only available when processing was stopped.

[dbjorck]

Hi!

As Dan said, you should have a separate VAP folder. You're running the risk of watching an uncompleted file. And having it separate makes it very easy to see which is which. If you want to watch it before it processes, you can just double click on it in the VAP folder.

It sounds like you have a similar situation as I do. I have to reboot often, and am left with temporary and corrupted files all over the place. .tmp, .wtv.tmp, wtv.tmp_1 etc. Files with a 0 size. Recordings only a couple of minutes long. And sometimes the system gets "shocked", and a file gets stuck in a folder and never picked up.

So I've just developed a program for housekeeping VAP/VRD/DVRMSToolbox. It deletes all temporary and corrupt copies all over the disk. It retriggers stuck files. If the result is corrupt (say a zero size .wtv in the output folder, which has happened to me), it rolls processing back and reprocesses it from the beginning (this requires saving a backup copy of your input file). If there are two duplicate output files, one .tmp and one .wtv, it either gets rid of the smaller one and makes the larger one available for viewing, or rolls back processing and reprocesses it (I don't know how it happened, but I did have a recording that had one wtv and one tmp, where the tmp was larger than the wtv). When you've watched the show, it deletes all other copies of it on disk (including the backup if you have that). It also recycles the VRD log, as it will otherwise grow indefinitely. It is working great for me, and has cleared quite some disk space.

Brgds

Danny

[dlflannery]

Hi!

As Dan said, you should have a separate VAP folder. You're running the risk of watching an uncompleted file. And having it separate makes it very easy to see which is which. If you want to watch it before it processes, you can just double click on it in the VAP folder.

It sounds like you have a similar situation as I do. I have to reboot often, and am left with temporary and corrupted files all over the place. .tmp, .wtv.tmp, wtv.tmp_1 etc. Files with a 0 size. Recordings only a couple of minutes long. And sometimes the system gets "shocked", and a file gets stuck in a folder and never picked up.

So I've just developed a program for housekeeping VAP/VRD/DVRMSToolbox. It deletes all temporary and corrupt copies all over the disk. It retriggers stuck files. If the result is corrupt (say a zero size .wtv in the output folder, which has happened to me), it rolls processing back and reprocesses it from the beginning (this requires saving a backup copy of your input file). If there are two duplicate output files, one .tmp and one .wtv, it either gets rid of the smaller one and makes the larger one available for viewing, or rolls back processing and reprocesses it (I don't know how it happened, but I did have a recording that had one wtv and one tmp, where the tmp was larger than the wtv). When you've watched the show, it deletes all other copies of it on disk (including the backup if you have that). It also recycles the VRD log, as it will otherwise grow indefinitely. It is working great for me, and has cleared quite some disk space.

Brgds
Danny
What "Dan" do you refer to?

Check out the latest release of VAP -- no more infinite log file build up!

I would just note that VAP itself creates no files with a .tmp extension. These are created by VRD apparently, and it should be cleaning them up except for abnormal terminations.

What language is your program written in?

[dbjorck]

Hi!


What "Dan" do you refer to?
My profuse apologies... I meant dlflannery of course... (It's Dan at the VRD forum so I got confused... ;))



Check out the latest release of VAP -- no more infinite log file build up!
I know, and notice that I didn't mention the VAP log - only the VRD log...



I would just note that VAP itself creates no files with a .tmp extension. These are created by VRD apparently, and it should be cleaning them up except for abnormal terminations.

I've had a lot of abnormal terminations, which is what sent me down this route of writing a Housekeeping program. VRD creates .tmp files, and actually so do I. I have DVRMSToolbox set up to move the recordings between the various folders, and in order to be absolutely sure of stopping various triggering programs (DVRMSToolbox, VAP) from picking them up too soon, I rename them to .tmp during copying. I have a very busy external USB drive, and the copying can take a long time and seem interrupted (which I'm afraid could be interpreted as finished) if there's a lot going on.



What language is your program written in?
VB.net. The actual processing code is done, and is so far stable and performing well. Right now I'm working on the UI, which is going to be a SysTray thing like yours, with a popup form for configuring and seeing status. It's the first time I try VB.net though and it's all "learning by doing"...

PS. I have just achieved VRD Nirvana - not a single stalled file anywhere. This is partly because of the Housekeeping; it retriggers and removes corrupt copies (which can't be processed).

Brgds

Danny

[KryptoNyte]

As Dan said, you should have a separate VAP folder. You're running the risk of watching an uncompleted file. And having it separate makes it very easy to see which is which. If you want to watch it before it processes, you can just double click on it in the VAP folder.


If I had a seperate folder for VAP, that would somewhat undermine the automation of the system as a whole. The neat thing right now is that SageTV records the shows I like, and VAP takes it from there. I don't actually watch the files until processing is complete and they've landed in their target directory(s).



It sounds like you have a similar situation as I do. I have to reboot often, and am left with temporary and corrupted files all over the place. .tmp, .wtv.tmp, wtv.tmp_1 etc. Files with a 0 size. Recordings only a couple of minutes long. And sometimes the system gets "shocked", and a file gets stuck in a folder and never picked up.


So far, I don't believe I've *ever* had a stalled file that was the fault of VAP or VRD. I have had some issues with my HD-PVR + STB + IR Blaster, that results in a bad file from that hardware combination, and that can potentially become a stalled file, but every single recording from my HDHomeRun device (mpg files) has processed successfully through VAP.

So far, I've generally found that when there is some problem in processing, that's it's a problem with my other hardware, or a problem in my understanding of how to configure VAP to process the files.

I was basically just looking for a way to do a quick delete of a file on which I made a late decision to terminate processing. That said, it's kind of interesting to hear the plethora of different ways that folks are using VAP.

[KryptoNyte]

I should mention that I'm still not opposed to finding the "right" other solution for my work flow, rather than trying to change the way VAP operates. Anyone else have some input?

[dlflannery]

I should mention that I'm still not opposed to finding the "right" other solution for my work flow, rather than trying to change the way VAP operates. Anyone else have some input?
Just curious why a time delay between file detection and starting processing ,as I mentioned in post #2, wouldn't work for you?

[dbjorck]

Hi!


If I had a seperate folder for VAP, that would somewhat undermine the automation of the system as a whole.
It's still automated for me. I use DVRMSToolbox to automatically move the file from the recording folder to the VAP folder as soon as it has finished writing. And it takes a backup, should processing fail; OR if the AdScan removes chunks of the show instead of the ads... Then I can reopen the backup and view it, or tweak the AdScan settings in VRD and reprocess it. Another reason for failing is if the output disk runs out of space, which has happened to me twice. (The backup disk is not the same as the output disk BTW...)

Brgds

Danny

[KryptoNyte]

Well, the time delay might work, or maybe not.

I believe things like VAP and computers are here to make life easier, so let me give you a possible "real life scenario."

1) I have SageTV setup to save my favorite shows, let's say 10 shows a week in the busy season, heck, I may even decide on a whim that there are 3 more shows (non-recurring) that I *might* want to watch later. Pretty much all of my recording occurs on shows in mpeg-PS format, it's the same packet data that I receive from my cable provider. VAP is left in processing mode.
2) The VAP input directory is the same as my SageTV save-to directory.
3) I leave home on business for 8 days, or perhaps I just leave my media computer unattended for 8 days.
4) When I return, VAP has already QSF'd and commercial detected all 10 programs.
5) I decide at this point if there are any of my recorded programs that I would prefer not to watch/keep for later, and delete all the files, including all the temp files and vprj files that VAP has created (manually).
6) Using VAP to hop to VRD per file, I manually edit the commercial cuts on all remaining files.
7) I allow VAP to continue processing, which saves the resulting mpg files to the VAP output directory (shared across my LAN) with all commercials cut.
8) I watch the programs on any video device in the house via shared directory with WDTV device and then delete the mpg files at some point when they're no longer needed.

How might the delay processing scenario apply here? Is there a better way?

Keep in mind, VAP's entire process functions for me, with almost 100% success, at least on the mpg files. As far as I can tell, I don't need any secondary programs to move files here and there ... it just works as it is.

The entire reason for the feature suggestion of the "delete" button is Step 5 above.

[jcthorne]

I too could use a DELETE button on the VAP page. Would work fine if it only worked when processing was stopped. For the same reasons as KryptoNyte noted above.

I usually let kttmg and VAP run automated until that point and then decide if the file will be kept or not when looking at the commercial cuts and have it open anyway.

Thanks,

[dlflannery]

Try Ver. 0.84, just released.

[jcthorne]

Try Ver. 0.84, just released.

Great! Thanks!

[KryptoNyte]

That little delete button works a treat.

Thanks, DL.

[msmart]

Haven't tried the delete button yet, but thanks. There are times when TTG downloads a show a second time for what ever reason or a wrong show got recorded because the channel didn't change properly so there are times when I have to delete shows that have been partially processed. The delete button will clean them up nicely.

[KryptoNyte]

Hey, DL, next time you hit VAP with a [more worthwhile] revision, could you include the ability to delete a file that has been recognized, but not yet had any processing? This would mainly be due to my forgetfulness in turning processing back on, rolled in with some personal laziness. Occasionally, I also manage to catch a sporting event that SageTV is recording live and block processing on it.

Obviously not a completely necessary VAP feature, but might just be handy once in a while if the change doesn't require too much time on your end.

[dlflannery]

Try Ver. 0.86, just released. In addition to deleting files in the NEW status, it will delete files that are partially processed, and clean up.

[KryptoNyte]

Well ... now I'm just starting to feel spoiled.

Thank you.