vapPostProcess files to trim output video

[dlflannery]

Two files to support post process trimming of output files for VAP6 are contained in attached postProcessTrim.zip
The files are:
VapPostProcessTrim.cmd
trimEnds.vbs

These scripts implement trimming of the start and end of the output video file using the COM function toolsTrimAndCopy

Instructions:
1. Place the two files in the folder where VAP6 will place output video files you want to trim (which could be a meta-renamed folder),,
2. Rename the .cmd file to VapPostProcess.cmd
3. Edit trimEnds.vbs and adjust the three user-specified values to set the (approximate) bit rate of the output video and the desired start and end trim times.
4. Edit vapPostProcess.cmd noting the choices to be made regarding the pause command and whether to delete the untrimmed output file.
5. Trimmed output video file name will be the untrimmed video file name with "trimmed_" prepended. You can edit the .cmd file to change this as desired.

Notes:
1. The trimAndCopy function used (either in the TVS6 UI or the COM versioin used in these files) does not produce viable output files for .mp4 files but does do so for .mpg files. This is a known behavior and has to do with whether the file has a header. I have not tested .ts files or .mkv files or any other file types in this regard.
2. You can configure VAP6 to do nothing but trim an input file by unchecking both do QSF and do Adscan (and checking do Postprocess). The input file will be copied to the output folder and the trim postprocess scripts will trim it.
3. As always postProcessing can be customized to file categories using meta-renamed output folders and profile match strings.
4. Several messages are logged as a result of these scripts running. They don't appear in the log until after the postProcess completes however.

EDIT 4 Dec 2021:
A vapPostProcess.cmd /VBScript file set that will trim any type video file, including .mp4 is now available:

https://www.videoredo.net/msgBoard/index.php?attachments/trim5-zip.3342/

The file is loaded into TVS6 then automatically edited to define a selection excluding cuts at the two ends then saved with an output profile automatically determined based on the input file extension. It's set to trim 5 secs off each end but you can see in trim5.vbs where simple editing can adjust those amounts. For a 43 minute PlayOn file it took 34 seconds to do the trim on my PC, which is not the swiftest. It actually trimmed a total of 12 secs, probably due to slight discrepancies in TVS6 determining total duration. The postProcess .cmd file deletes the original file then renames the trimmed file to the original name (all in the VAP output folder). This obviously can be changed as desired.

 

[jtcreate]

Sorry, i've been a little delayed. I am not sure but I think I just found a bug in VAP6. When running multiple configurations, Vap6 doesn't seem to save directory changes and setting changes (at least on the main Gui). If I sent up config:2, it will remember the original settings I enter in, but if I change the directories later on, VAP6 does not retain the settings. Mine tried to revert back to the first settings when I restart my entire system. I deleted VAP6 profile from my profile and started over to implemented the changes.

Also, I can't remember in the original VAP if we ever corrected the .net framework error that pops up when VAP has to monitor a folder also being watched/ read from/ written too by Onedrive. I was experimenting with VAP6 and onrdrive, and the error came up. I believe you once said its because VAP monitoring is struggling against Onedrive with the directory being briefly locked up.

Anyway, I'll give the post file a test shortly.

 

[dlflannery]

Sorry, i've been a little delayed. I am not sure but I think I just found a bug in VAP6. When running multiple configurations, Vap6 doesn't seem to save directory changes and setting changes (at least on the main Gui). If I sent up config:2, it will remember the original settings I enter in, but if I change the directories later on, VAP6 does not retain the settings. Mine tried to revert back to the first settings when I restart my entire system. I deleted VAP6 profile from my profile and started over to implemented the changes.

Also, I can't remember in the original VAP if we ever corrected the .net framework error that pops up when VAP has to monitor a folder also being watched/ read from/ written too by Onedrive. I was experimenting with VAP6 and onrdrive, and the error came up. I believe you once said its because VAP monitoring is struggling against Onedrive with the directory being briefly locked up.

Anyway, I'll give the post file a test shortly.

I just reviewed the code to be sure configs of distinct numbers (or with no number) are loaded and saved as needed and can't see what would cause a problem, except: Some changes, e.g., folder settings are set in program memory but are not written to the XML configuration file until the program shuts down. I can think of two ways this could lose settings: (1) rebooting without closing VAP or (2) VAP crashing. Do you think either of those could be the explanation? I'll put a mod to force immediate updates of the XML file upon changes on my todo list.

If there is some interaction of monitoring with One Drive, I wouldn't know what to do to fix it. Might have to download from One Drive to a separate folder then move the file into the VAP monitor folder.

 

[jtcreate]

I just reviewed the code to be sure configs of distinct numbers (or with no number) are loaded and saved as needed and can't see what would cause a problem, except: Some changes, e.g., folder settings are set in program memory but are not written to the XML configuration file until the program shuts down. I can think of two ways this could lose settings: (1) rebooting without closing VAP or (2) VAP crashing. Do you think either of those could be the explanation? I'll put a mod to force immediate updates of the XML file upon changes on my todo list.

If there is some interaction of monitoring with One Drive, I wouldn't know what to do to fix it. Might have to download from One Drive to a separate folder then move the file into the VAP monitor folder.

It happens when Windows restarts or shutdown. I don't have or issue a command that close VAP first. I never did with the original VAP either and never had this issue. So one theory you have is Windows is not closing VAP correctly. Actually, even when I stop VAP from running manually and then close the app, it still happened. So, immediate updating from XML files would be the best thing to do.

Ok, so I have to work out a different approach with onedrive. ok. I use another watchfolder program called, "Data Mystic Watch Folders," which doesn't error when watching a Onedrive folder. Its not as sophisticated as VAP. I can send you a link to an older version of the program if you like to experiment with.

 

[dlflannery]

I don’t normally run multiple configs. Problems like you’re having are likely to depend on the sequence of doing things, so I could set up a second config and not observe the problem because I don’t repeat whatever sequence triggers it. You could help my debugging a lot if you could give me the simplest possible sequence of events that illustrates the problem.

One warning: If you change the folders in VAP it should be done when VAP.isn’t doing anything and no actions are pending. I’m curious why you are changing folders beyond initial configuration?

 

[jtcreate]

I don’t normally run multiple configs. Problems like you’re having are likely to depend on the sequence of doing things, so I could set up a second config and not observe the problem because I don’t repeat whatever sequence triggers it. You could help my debugging a lot if you could give me the simplest possible sequence of events that causes the problem.

One warning: If you change the folders in VAP it should be done when VAP.isn’t doing anything and no actions are pending. I’m curious why you are changing folders beyond initial configuration?

I didn't like the configuration. Overdone. I thought of a simplier approach. I always have VAP shut off when making directory changes. Since the original VAP worked so well with multiple configurations, I don't mix VAP6 with onedrive now.

 

[dlflannery]

It happens when Windows restarts or shutdown. I don't have or issue a command that close VAP first. I never did with the original VAP either and never had this issue. So one theory you have is Windows is not closing VAP correctly. Actually, even when I stop VAP from running manually and then close the app, it still happened. So, immediate updating from XML files would be the best thing to do.
..........................

I ran two instances of VAP6, changed the output folder in one of them and restarted Windows without closing VAP6. I did this twice; once with processing not running and once with processing running. In both cases the proper folder setting was retained after restart. VAP6 has code to detect if it's being shut down due to Windows being shut down and updates the XML file in that case.

So I'm puzzled by your problem, assuming you're saying it's not triggered by a VAP crash rather than a Windows shut down. I suppose there could be some difference in our PC's or OS version. that keeps the VAP shut down handling from working correctly. i"m still considering a mod to force immediate saving of folder changes to the XML file, provided I can assure myself there will be no unintentional side effects.

 

[jtcreate]

I ran two instances of VAP6, changed the output folder in one of them and restarted Windows without closing VAP6. I did this twice; once with processing not running and once with processing running. In both cases the proper folder setting was retained after restart. VAP6 has code to detect if it's being shut down due to Windows being shut down and updates the XML file in that case.

So I'm puzzled by your problem, assuming you're saying it's not triggered by a VAP crash rather than a Windows shut down. I suppose there could be some difference in our PC's or OS version. that keeps the VAP shut down handling from working correctly. i"m still considering a mod to force immediate saving of folder changes to the XML file, provided I can assure myself there will be no unintentional side effects.

Well, I haven't been able to duplicate the issue. It might be due to a Windows update that I rolled back last night. It hasn't happened since then. However, that annoying .net framework crash is still happening. I'll attach the log report. I really don't want to, but I have to run three config of VAP6. Its not the first time I've done it with success. One to handle NPVR recordings (config:1). One to handle PLayon(config:2). They both send to config:3 for adscan and -vprjonly. config: 3 is getting the error. I am using VAPpostprocess.cmd as a middle-man to move the files to config:3, to minimize conflicts and to write additional logs. Here is the crash report.

 

[dlflannery]

Thanks for the crash report. Just FYI when you copy/paste those reports, all the stuff starting with "************** Loaded Assemblies **************" doesn't need to be included.
This tells me the code location of the exception, which is helpful. For some reason it is looking for a file name in internal storage that isn't there. I hope you can furnish some additional data to help me track it down:

The following are located in your VAP6 app data folder: C:\Users\<account name>\AppData\Roaming\VAP6

1. VideoFileManager3.xml
2. VRDAutoProcessLog3.txt

The number 3 is to select your config 3 data, assuming you are using CL option -config:3 for that VAP6 instance.
The only part of the log file I'm interested in is the portion just before the crash and it would be preferable (but not essential) if the XML file was as it existed just before the crash.

Can you tell me the file name and type being transferred into the config3 monitor folder that triggers the crash?
Also, what command do you use to move or copy the files from the vapPostProcess scripts used by config1 and config2? Have you tried just configuring the output folders of those VAP6 instances to the monitored folder of the config3 VAP6?

 

[jtcreate]

I got the files. I'll attached them below. The lock up happened when processing "Wheel of Time".
The first crash was when VAP6 config:3 adscan some trailers from the show (4:26:48am. I found the lock up, restarted VAP which restarted the process around 5:30am). Second lockup is at 5:31am. That's a playon capture coming in from VAP6 config:2.

The previous encodes at 8:52am came from VAP6 config:1, which comes from NPVR recorder. It seem to work fine. You can tell the difference in the filename convention. Question? In the log, it mentioned VAP looking for Wheel of Time as a .TS file in the QSFtemp directory. All incoming files from VAP 1 and VAP 2 are .mp4. Is that normal? Could that be the issue?

Yes, I use the CL -config:3 option to start all VAP processes.

 

[dlflannery]

Before we go much further, I notice you are using old versions of both TVS6 and VAP6:

TVS6 your version is 6.61.1.819 - Jun 27 2020 ----- My version is 6.62.5.831 - Jun 5 2021
(more important) VAP6: You're running 1.01 while 1.02 is the latest version.

I've gone to great pains to duplicate your process flow.
-vprjOnly
Do Qsf
Do Adscan
Output Profile = MP4
Output Folder put on different drive from Monitored and TempQSF

I took a small .mp4 file I had and renamed it to exactly the file name involved in your 4:26am crash.
Started VAP6,
Started processing.
Then copied the test input file into the monitored folder.

Worked as expected with no crash.

I will not want to pursue this any further unless you can upgrade your program versions, especially VAP6. If your problems continue after upgrading I will probably want to get the postProcess files you use with configs 1 and 2 and a copy of a playOn recording that causes a crash. My PlayOn is working so I can actually duplicate your process flow for those inputs.

EDIT:
I did some searching to refresh my mind about the previous (2018) crashes you were having with "old" VAP (the TVS5 version). The crash reports for that problem were different than the current case (attachment to post #8 here). See this post:
Net Framework Unhandled Exception has returned | VideoRedo

 

[jtcreate]

Oops, I thought I was updated. I missed that as I use Videoredo on more than one machine. I'll let you know.

 

[jtcreate]

I updated everything and did as you suggested. The crash issue went away. So thank you for your help on that. PLayon processing works really smooth now, with the exception that I can't auto trim the beginning and end, because, Videoredo doesn't generate working .mp4 files when using that feature. I sent them a request to resolve this issue. Hopefully they do.

 

[dlflannery]

I’ll be surprised if you get the response you desire from VRD on this, because it’s much more complex to do it on .mp4 files. However, I think it would be possible to do a VBScript using the COM functions to accomplish the trims. This would be called from a vapPostProcess script. The VBScript would essentially silently mimic what you would do manually using the TVS6 UI. (Load the file, define a selection that excludes the end points and save the selection.). It would run at the same speed a file save runs in the UI.

So let me know if you would like to pursue this idea.

Note that TVS6 doesn’t preserve metadata when it processes .mp4 files. I can’t remember if this is important to you but there may be ways to insert metadata in the final trimmed output file as an additional step in the vapPostProcess script.

 

[jtcreate]

I’ll be surprised if you get the response you desire from VRD on this, because it’s much more complex to do it on .mp4 files. However, I think it would be possible to do a VBScript using the COM functions to accomplish the trims. This would be called from a vapPostProcess script. The VBScript would essentially silently mimic what you would do manually using the TVS6 UI. (Load the file, define a selection that excludes the end points and save the selection.). It would run at the same speed a file save runs in the UI.

So let me know if you would like to pursue this idea.

Note that TVS6 doesn’t preserve metadata when it processes .mp4 files. I can’t remember if this is important to you but there may be ways to insert metadata in the final trimmed output file as an additional step in the vapPostProcess script.

Sure, I would like to give it a shot. Its only a 5 second trim at the beginning and a 5 second trim at the end. So this would be part of VAP config:1 and would result in completing an .mp4 playon file with 5 seconds removes off the beginning and end? Sounds workable. As far as MetaData, its not crucial, but nice to have. While we are on the subject of VAPpostprocess, I am going to start another topic about the different strings VAP puts out for batch processing using the post process script.