VAP auto cutting based on a preset time.

jtcreate

Member
Hi, it’s been a while. Can VAP be made to autocut a set amount of time from the beginning and tail of any mp4 it detects? Let’s say I want the first 10 minutes to be automatically removed from the beginning of a video and the last 5-10 minutes to be automatically removed from the end? I figured this might involve tools from the commercial cut features of VideoReDo/vap. Hope all it well.
 

dlflannery

Moderator
There is a TVSuite COM function that kind of does this, according to its description here:


If this can be used to do what you want then it could be called from a VapPostprocess script. VAP6 can be configured to do nothing but pass the input file through to the output folder and then do the postProcess script. I haven’t actually used this particular COM function. Note that you must select the portion to be output by specifying a start byte count and a total output file byte count. Depending on how fussy you are about exact times for the trims, you should be able to get usable byte counts by doing simple math using the total file size and video run time. Even simpler would be if you could specify trim amounts either as % of total file size or absolute byte count, e.g., start at a certain byte count and set total bytes output to 90% of total file size.

I am willing to work on this approach. I don’t know if you would want, or be able, to work on the .vbs file to call the COM function, which would be called from the postProcess script?
 

jtcreate

Member
Ok, sorry for the long response, but before we dive further into this, I felt I should give you the entire picture.

The reason I inquired into this feature is that I want to use VAP with PLayon. I am not sure if you are familiar with PLayon. Its a Software/service used to captured streaming shows and it can be set to grab new episodes released each week from sites like Hulu and CBS all access (I am currently using the Home software, not cloud). The good thing with Playon is that it captures all shows with the same standard file output ( 720P @ 29.97 mp4 at a bitrate around 3000 kbps - 2 channel 44.100kHz at 190kbps). So re-encoding is actually not necessary, only editing and remux as an .mp4 file. The negative side of playon is that it creates folders for each title and it can't be disabled. It also creates subdirectories based on the season number, if its a episodic show (although VAP does not directly deal with subdirectories, isn't there a workaround for this?). The other negative thing Playon does is generate an annoying title card and the beginning and end of each captured video with your account name and IP address on it. The title card lasts 5 seconds. This is what I want to trim off.


I took a look in VideoRedo and I see it in the "Tools" section. I haven't experimented with the "trim and copy source file" yet, but apparently, it does eiter the beginning or the end, not both at the same time. As you say, its based on file size set in MByte, not KByte. I ran a few playon generated mp4 files through video redo and extracted the annoying generated title cards. At 5 second each, they come out at 588 kbytes in size. 10 seconds comes out at around 1.99mbs. I am not fussy about the exact trim time. So rounding up to 1 or 2 MBytes is not an issue, as long as the size can still be established by setting. However, the trick is to get VAP to do both the start and tail in one process.

Also, some streaming services (like in Peacock) do force commercials without offering a commercial free option. So ad-scan would be needed creating a .vprj file. I don't want to run two VAP applications (config 1 and config 2) to accomplish everything. I can look into doing some VBS but I wasn't sure all this info might change your initial approach. Thanks.
 

dlflannery

Moderator
I’m familiar with PlayOn, actually I’m a lifetime subscriber although I haven’t used it for several years now. Sounds like the mp4 parameters haven’t changed from when I last used it. I think the toolsTrimAndCopy COM function called in a vapPostProcess script can do what you need. We should be able to calculate the byte parameters just using the file size, and this is made simpler by knowing all the recordings have a fixed bit rate. The same postProcess script will work whether you have done AdScan or not. If you are reviewing cuts, you would not trim the ends during the review but just let the postprocess script do that later.

I’ll go ahead and work on this. I should be able to generate my own PlayOn recordings for test although if there is a convenient way to provide a small sample for download, that would save me the trouble of getting PlayOn going again.

Also, I created a little utility program called FileCollector that will move any video file in a subfolder of a designated folder (e.g., the VAP monitor folder) into that folder. See here:
 
Last edited:

jtcreate

Member
I’m familiar with PlayOn, actually I’m a lifetime subscriber although I haven’t used it for several years now. Sounds like the mp4 parameters haven’t changed from when I last used it. I think the toolsTrimAndCopy COM function called in a vapPostProcess script can do what you need. We should be able to calculate the byte parameters just using the file size, and this is made simpler by knowing all the recordings have a fixed bit rate. The same postProcess script will work whether you have done AdScan or not. If you are reviewing cuts, you would not trim the ends during the review but just let the postprocess script do that later.

I’ll go ahead and work on this. I should be able to generate my own PlayOn recordings for test although if there is a convenient way to provide a small sample for download, that would save me the trouble of getting PlayOn going again.

Also, I created a little utility program called FileCollector that will move any video file in a subfolder of a designated folder (e.g., the VAP monitor folder) into that folder. See here:

True, cutting the tail won't affect the accuracy of the adscan VRPJ file but I am concerned how trimming the beginning will knock the scan out of sync. So the beginning and end will have to be completed before the adscan takes place. Also, traditionally when I use VAP, VPRJ files are just an added option. I don't always use them as my VAP is configured to complete the file encoding anyway (and I won't be encoding here). Not all the stuff I am grabbing will have commercials. So I can just ignore the VPRJ file for those. Also, in case bitrates does eventually change, will I be able to set the amount of bits to be trimmed?

I haven't used filecollector but does it auto delete the directories that are created as well as extract the files? Where do I obtain filecollector?
 
Last edited:

dlflannery

Moderator
If you load a PlayOn recording that has commercials into TVSuite and run adscan, do the end screens screw up commercial detection? My concept would do that and then the final output file (already having commercials removed) would be trimmed by the postProcess script.

There would be constants in the postProcess script that could be adjusted to adapt trimming to different bit rates.

Are you not using VAP now? I’m confused how you configure it to automatically sometimes cut commercials but other times not do that? Are you changing the VAP configuration manually between running batches of files of different types? Perhaps I need a detailed description of the process flows you’’re using.

Just go to the link in my previous post to find the download link for FileCollector. No it does not delete the subfolders. That would have to be done later manually. FC monitors all subfolders, i.e., is designed to run continuously, moving files as they appear.
 
Last edited:

jtcreate

Member
I am running VAP for broadcast recordings, but many channels and streaming services like ABC and FX group their commercials too close together, there are no slugs. So videoredo makes a lot of mistakes which is why I don't use the auto-cut feature.. I have VAP provide the VPRJ file instead. But I don't like VAP waiting for me to make changes to the VPRJ file before creating the final .mp4 file. So, a couple years back, you created an alternative "ad scan" option, where if I put in a -Vprjonly CL line, VAP will do the autoscan and then finalize the process by providing me the encoded .mp4 file and the .vprj file (all ad scan options in the VAP Gui have ot remained unchecked for this). This was back when you were working on VAP ver. 142. I provided a link to the original conversation. However, with VAP6 1.02, it no longer seems to be working.


 

dlflannery

Moderator
The -vprjOnly option is working on VAP6 1.02 for me. You have to have Do Adscan checked for the file. If both Do QSF and Do Adscan are unchecked for a file, VAP just passes a copy of the input file to the configured output folder (and no VPRJ file is created).

However my idea for the postProcess trimming will not work with that process flow. Because the trimming has to occur AFTER the final file is created (using the vprj file). But if you are creating the .vprj file, presumably so you can bring it up in TVSuite and review/adjust cuts, then can't you just edit out the start and end frames at that time?

My idea WILL work if either (1) you don't want to adscan the file at all or (2) you adscan but let VAP wait until you review cuts.

I tried to get my PlayOn working but when I try to record a clip from Peacock it says I need the "Home" edition of PlayOn. My version is the "Desk Top" edition (and it used to support making recordings).
 
Last edited:

jtcreate

Member
The -vprjOnly option is working on VAP6 1.02 for me. You have to have Do Adscan checked for the file. If both Do QSF and Do Adscan are unchecked for a file, VAP just passes a copy of the input file to the configured output folder (and no VPRJ file is created).

However my idea for the postProcess trimming will not work with that process flow. Because the trimming has to occur AFTER the final file is created (using the vprj file). But if you are creating the .vprj file, presumably so you can bring it up in TVSuite and review/adjust cuts, then can't you just edit out the start and end frames at that time?

My idea WILL work if either (1) you don't want to adscan the file at all or (2) you adscan but let VAP wait until you review cuts.

I tried to get my PlayOn working but when I try to record a clip from Peacock it says I need the "Home" edition of PlayOn. My version is the "Desk Top" edition (and it used to support making recordings).

Ok, when I use -vprjOnly along with adscan engaged and QSF, VAP provides only the .vprj file in the VAPoutput directory. Its not processing the original .ts source file to .mp4. Instead, a copy of the .ts source file is stored in the QSFtemp directory.

As far as PLayon, I'll get you some video samples shortly. It looks like I will have to run two different VAP configs to handle both the autocuto and the adscan. Will Vap be able to rename the playon files and do the online metadata searches? THanks. I don't want to post an open link here concerning the Playon files. So I sent you a private message containing the link. It expires at the end of this week. There should be two files in the link. They are uploading as I speak, so it might take 15-20 minutes for them to appear.
 
Last edited:

dlflannery

Moderator
I processed the 9-1-1 file. Using an input file name metadata parsing template as follows:
Code:
{title} - s{seasonnum}e{epnum} - {eptitle}
(with extension = .mp4)
theTVDB metadata search succeeded.
Ok, when I use -vprjOnly along with adscan engaged and QSF, VAP provides only the .vprj file in the VAPoutput directory. Its not processing the original .ts source file to .mp4. Instead, a copy of the .ts source file is stored in the QSFtemp directory.
Yes. The vprj file, if you load it into TVS6 will load the file in the tempQSF folder, which has been QSF'ed and is H.264 with the same video and audio specs as the input file. AdScan cuts will be there and can be reviewed/adjusted. I noted the adscan performance for this file is terrible and would require many adjustments during review. To get the encoded .mp4 file with cuts you would have to do a file save to that profile.
............. It looks like I will have to run two different VAP configs to handle both the autocuto and the adscan. Will Vap be able to rename the playon files and do the online metadata searches? THanks. I don't want to post an open link here concerning the Playon files. So I sent you a private message containing the link. It expires at the end of this week. There should be two files in the link. They are uploading as I speak, so it might take 15-20 minutes for them to appear.
Yes, meta-based renaming and folders will function with regard to the .vprj file.
Metadata will be saved to a <filnename>.txt file beside the .vprj file if you use CL option -metaPyTivo. It will also be saved to a BTV style XML file if you use CL option -metaBtv

If you were not doing the -vprjOnly thing, VAP6 would be able to output the final .mp4 with (optionally) adjusted cuts and embed the metadata in it. Then a postprocess script could be used to trim off the ends.
Not sure what you have in mind in re: using two instances of VAP.
 

jtcreate

Member
It looks like I will have to have VAP config 1 do the file naming, TS to mp4 conversion and start/tail trimming of the playon files. Vap Config 2 will do the adscan and vprj completion.
 

dlflannery

Moderator
It looks like I will have to have VAP config 1 do the file naming, TS to mp4 conversion and start/tail trimming of the playon files. Vap Config 2 will do the adscan and vprj completion.
Not sure I follow that. For example I don’t know why you need a TS-to-mp4 conversion, since the PlayOn files are already mp4. Here’s my approach based on my guess of what you’re trying to do (assuming a PlayOn mp4 input file):

VAP Config 1:
(Does not use -vprjOnly option)
Configure QSF only with H.264 mp4 output profile selected, and do postProcess
Use input file name metadata parsing template(s) as per post #10 above so theTVDB search can get metadata.
Configure for meta-based output file and folder naming.
Configure to embed metadata in output file, and/or output .txt or XML metadata files and use them somehow later.
postProcess file trims ends of output file and moves trimmed file to monitored folder of VAP Config 2
Caveat: I doubt the toolsTrimAndCopy COM function will preserve any embedded metadata.

VAP Config 2:
Does use -vprjOnly option
Configured for AdScan only and no postProcess.
Caveat: Even if input file has embedded metadata, I’m almost positive TVSuite will not pass it thru to the final output file after you load the .vprj file and do a save-as. You would have to figure out how to use the .txt or XML files created by VAP config 1.

But I don’t have to understand the details. All I need to know is what (if anything) I can do to help. Would you like me to work on a postProcess script to do the trimming? If so please specify the **exact** configuration of VAP6 that the script will be used with, including the input file format (PlayOn .mp4 ??) and configuration settings.

EDIT: Additional info learned:
The PlayOn m;4 recordings do have embedded metadata but the format is different from what VAP6 currently can read. (Will have to use input file name parsing and TVDB lookup for now, pending VAP6 mods.)
TVSuite 6 does NOT preserve embedded metadata in output files. (VAP6 inserts metadata that it knows into output files using AtomicParsley unless that is disabled in configuration. But not for -vprjOnly case.)
HERES THE BIG ONE: The trimAndCopy function (either in TVS6 program or using the COM function) produces unplayable and unloadable (into TVS6) videos when applied to .mp4 videos! This is a known limitation
of TVS6. It does work for some file formats, e.g., .mpg. It's because some formats (e.g., .mp4) have a header at the beginning, which can't be trimmed and retain a viable video file.

Thus my whole concept of using toolsTrimAndCopy for your PlayOn recordings is kaput! Sorry but I had never used trimAndCopy before this so didn't know the limitations.
 
Last edited:

jtcreate

Member
Not sure I follow that. For example I don’t know why you need a TS-to-mp4 conversion, since the PlayOn files are already mp4. Here’s my approach based on my guess of what you’re trying to do (assuming a PlayOn mp4 input file):

VAP Config 1:
(Does not use -vprjOnly option)
Configure QSF only with H.264 mp4 output profile selected, and do postProcess
Use input file name metadata parsing template(s) as per post #10 above so theTVDB search can get metadata.
Configure for meta-based output file and folder naming.
Configure to embed metadata in output file, and/or output .txt or XML metadata files and use them somehow later.
postProcess file trims ends of output file and moves trimmed file to monitored folder of VAP Config 2
Caveat: I doubt the toolsTrimAndCopy COM function will preserve any embedded metadata.

VAP Config 2:
Does use -vprjOnly option
Configured for AdScan only and no postProcess.
Caveat: Even if input file has embedded metadata, I’m almost positive TVSuite will not pass it thru to the final output file after you load the .vprj file and do a save-as. You would have to figure out how to use the .txt or XML files created by VAP config 1.

But I don’t have to understand the details. All I need to know is what (if anything) I can do to help. Would you like me to work on a postProcess script to do the trimming? If so please specify the **exact** configuration of VAP6 that the script will be used with, including the input file format (PlayOn .mp4 ??) and configuration settings.

EDIT: Additional info learned:
The PlayOn m;4 recordings do have embedded metadata but the format is different from what VAP6 currently can read. (Will have to use input file name parsing and TVDB lookup for now, pending VAP6 mods.)
TVSuite 6 does NOT preserve embedded metadata in output files. (VAP6 inserts metadata that it knows into output files using AtomicParsley unless that is disabled in configuration. But not for -vprjOnly case.)
HERES THE BIG ONE: The trimAndCopy function (either in TVS6 program or using the COM function) produces unplayable and unloadable (into TVS6) videos when applied to .mp4 videos! This is a known limitation
of TVS6. It does work for some file formats, e.g., .mpg. It's because some formats (e.g., .mp4) have a header at the beginning, which can't be trimmed and retain a viable video file.

Thus my whole concept of using toolsTrimAndCopy for your PlayOn recordings is kaput! Sorry but I had never used trimAndCopy before this so didn't know the limitations.

After I read up on that limitation, I ran some test on the .mp4 - mpg - mp4 variations. I didn't like the results every time and decided that TrimAndCopy tool isn't going to work unless Videoredo fixes the mp4 issue.

Sorry, I didn't give a proper picture of how my system works, but I am currently running 3 configurations of VAP. I am still recording certain shows with NPVR which is how I get the h264 .ts files. Config:1 processes those .ts files to mp4 and sends them to the adscan input folder. Although PLayon is already an mp4 file, I still run it through VAP which is config:2. Config:2 doesn't convert the playon .mp4 but it does rename it (config:2 was done created in anticipation of triming the file). Config:2 outputs to the same adscan input folder as Config:1. Finally we have Config:3 with monitors the adscan Input folder and creates the .vprj file.
 

dlflannery

Moderator
We seem to have two threads going on mingled topics. Let's confine our discussion of your requirements to this thread from here on if possible.

Before reading your previous post I just completed a vapPostProcess/VBScript software set that will trim any type video file, including .mp4. See attached zip. It's set to trim 5 secs off each end but you can see in trim5.vbs where simple editing can adjust those amounts. For a 43 minute PlayOn file it took 34 seconds to do the trim on my PC, which is not the swiftest. It actually trimmed a total of 12 secs, probably due to slight discrepancies in TVS6 determining total duration. The postProcess .cmd file deletes the original file then renames the trimmed file to the original name (all in the VAP output folder). This obviously can be changed as desired.

As previously noted metadata is lost in this trimming process because TVS6 doesn't pass metadata through during the save op that does the trim. However there are several options for getting metadata into the trimmed file. Before getting into those details, I still need to get a better understanding of your desired process flow. For each type input file (NPVR and PlayOn) Can you give step by step the ops you want to occur and whether you want metadata in the final file(s) involved? Please don't worry about HOW to do it, just the operations and resulting files you want to exist.
 

Attachments

dlflannery

Moderator
FWIW, ver. 1.03 of VAP6, just released, will read metadata directly from PlayOn .mp4 recordings. Thus getting metadata via TMDb or theTVDB searching is optional for those files.
 
Top Bottom