Feature Requests and Response Status

[TPeterson]

I tested with the 1.40T4 file and got the same result. I inspected the files on that PC and the character encoding is the same as you observed "á" is Unicode E1 and it gets converted by Comskip (the same version as in VAP 1.39's download) to something else that neither VRD nor notepad display as "á". I'm guessing that this is some sort of interaction between Comskip and the Windows 7 OS on that PC. I tried changing the default keyboard to "US international" with no apparent effect. I give up for now and will just manually fix these few "stalled files" as they occur.

(Sorry about my aside above. I thought that you were the Admin that I had been conversing with about lack of email notices)

 

[TPeterson]

I may have stumbled upon the source of the trouble with extended characters on my older-VRD system. Last night's Late Show again had an extended character in it's title so it failed to complete VAP processing. However, when I opened the resulting vprj file in notepad and selected "save as" I noticed that the default encoding offered was "UTF-8". I tried saving it after selecting "Unicode" and my VRD didn't like that at all. So I tried saving it as "ANSI" and VRD happily finished processing it.

Evidently my old VRD doesn't like UTF-8 extended characters either and that's what Comskip seems to be using. I looked for a character-encoding option in Comskip's ini and didn't see one. Do you happen to know if there's a way to force it to use ANSI encoding?

So, i finally went to Google and "discovered" that my problem is old news at Comskip. This 2015 thread points out the same issue. At least I now have the simple workaround of loading the vprj file into notepad and saving it in ANSI when there's a stuck file.

 

[dlflannery]

I looked for such an option in Comskip also. VAP converts the vprj from comskip to Unicode page cp1252 (Windows Western European) after comskip creates it -- if VAP is running comskip.

 

[TPeterson]

...and my old version of VRD does not support Unicode extended characters. It does support ANSI though. Maybe VAP could offer that as an option?

 

[dlflannery]

What version of VRD are you running? The last 2 log files you attached showed ver 5.3.x, which I don't think is an "old" version. By "old" I thought you would mean ver 3.x or 4.x.

VAP knows what version of VRD you are running and it could automatically convert the vprj to ANSI if the version is "old" -- provided I know what VRD version numbers are "old".

 

[TPeterson]

On the problem system, VRD appears to be version 4.21.10.681 from 2013 (see attached). 5+ years old. Non-problem system (i.e., where latest VAP has solved the extended-character problem) is running version 5.x.

 

[dlflannery]

I'm stumped here. When I take a .vprj file as produced by Comskip (with no character encoding conversion by VAP), load it into notepad and save in ANSI format, I get exactly the same file as when VAP applies the Windows cp1252 conversion (which is just one of many ANSI encodings). IOW this says my VAP conversion is already creating the same result you get using your manual notepad procedure. This is for a file name with á in it.

 

[TPeterson]

Thanks, I'll explore further when this occurs again (tonight's show should do it). I'll take care to preserve both .vprj files. BTW, I just noticed that you've referred to cp1252 in two posts above as both "ANSI" and "Unicode". I think that it is the former, whereas I think that Comskip outputs the latter (actually UTF-8).

 

[dlflannery]

You are correct. cp1252 isn't unicode. It's the default Windows ANSI codepage (in the USA at least). The .net code that my programs use automatically reads/writes text using that encoding to/from its default internal unicode which is fixed 16 bits per character, unless you force the IO operations to use a different encoding.

UTF-8 is not always 16 bits per character, so I tend not to think of it as Unicode, but you are again correct in that it is part of the Unicode standard. It allows encoding 16-bit unicode characters without always using 16 bits.

I inspect text files such as .vprj files with a hex (binary) file editor so I can see the actual bits (bytes) used to encode each character. That is my basis for comparing different encodings.

 

[TPeterson]

Last night's TLS had two acute-accented "e" characters in its name. I attach a zip here with the original vprj file created in a Comskip-VAP(1.40T) session this morning along with the results of saving that file from notepad (Windows 7) in "UTF-8" and "ANSI". I note that notepad evidently correctly identified the original encoding as UTF-8 because that's what it suggested to use in the Save-As dialog. The saved UTF-8 file, however, has a couple of prefixed bytes that are not from the original. All three of these files display the extended characters correctly in notepad, but only the ANSI one works with VRD 4.21. I hope that this helps to sort out what's going amiss in your fix.

 

[dlflannery]

Verify the VAP version. 1.40T is not a valid version number (unless I made a mistake).

 

[dlflannery]

I renamed a .ts file of mine to "181120-29.1-The Late Show With Stephen Colbert-Michael Douglas; Ben Sasse; José Andrés.ts" And processed with VAP 1.40. I captured the .vprj file raw out of ComSkip and the version after converting to cp1252 ANSI by VAP.

The é character byte encoding was:
In raw vprj out of ComSkip: C3 A9 (UTF-8 , matching your UTF-8 and your version from VAP version ??)
In VAP-converted vprj: E9 (ANSI cp1252, matching your ANSI save from notepad)

So, why is your version of VAP producing a different encoding than mine? Need to really tie down what version you actually used.

 

[TPeterson]

I just reviewed the VAP program files folder and refreshed my memory: I'm running an installation of version 1.3.9 with the exe replaced by the "1.40T4" version that you attached above in this thread. The exe reports itself as "1.4.0.0" modified on 10/29/18. Would the full installation of version 1.4 make a difference? Maybe I should go ahead and find out....

 

[dlflannery]

No the full installation won't make a difference. To get the actual version number you need to look in the top border of the VAP UI. The test versions of 1.40 (e.g., 1.40T4) will still report as just 1.4.0.0 in the file info.

 

[dlflannery]

I just checked my code repository. Versions 1.38, 1.39, 1.40T4 and 1.40 all have the identical code for reading in the Comskip vprj file as UTF8 and rewriting it encoded as ANSI cp1252.

 

[dlflannery]

Well, mystery solved. Reviewing the code I discovered the UTF8 to ANSI conversion only occurred if VRD version 5 was being run. This is corrected in the VAP executable Ver. 1.41T1 (not an installer) contained in this zip:
https://vap.videoredo.net/VAPexe141T1.zip

 

[TPeterson]

Thank you, thank you, thank you. I'll stick in on that machine tonight.

Edit: As expected, it worked perfectly.

 

[TPeterson]

I've discovered another vagary with (version 1.40) VAP's filename handling. It refuses to "see" any file in the monitored folder with "Conflict" somewhere in the name. For example: 190416-2100-29.1-FBI-Conflict of Interest.ts is invisible, but when I take out "Conflict" it shows up. (But then, of course, the TVDB search fails).

EDIT: I just confirmed that the behavior persists in version 1.41.

 

[dlflannery]

I can't duplicate this. I put a file named exactly as yours in my Monitor Folder. It was detected and the TVDB search succeeded.

Check the "Input File Filters" tab of the Advanced Configuration form, in the lower section titled "FILE IGNORE MATCH STRINGS". Is "conflict" or any substring of it present?

 

[TPeterson]

The only entries in that box are "NBA Basketball" and "NFL". The symptom for files with "Conflict" in their names is the same as the ignored files so that's a good guess! Is there some other place where such a configuration could be hiding out??