Improvement of seeking.

[jirim100]

Introduction:

My daily job is to cut out ads from recorded videos. First, I move the mouse cursor to the area where the ad is. Then I use the arrow keys to find exactly where the ad starts or ends. I keep the arrow keys + Ctrl pressed and I watch the fast-changing scene of the video with my eyes. As soon as the eyes register that I have crossed the place where the advertisement begins or ends, I release the pressed keys. And now the problem: when editing the H264 and H265 video, the VRD6 is not able to handle many keystroke messages in short time so quickly, and even though I've already released the keystrokes, it still takes a few seconds (even on a fast PC with i9-10900 CPU, 1GB NVMe drive, 32GB RAM and even with enabled Cache in VRD6) than VRD6 handles keystrokes that are in the queue and this is very annoying (I have to wait on the end of it before I can continue and when it ends - video scene is on different position than I release the keys - this is next problem). I need to shred (discard) all other presses in the message queue when I no longer hold down the keys. Because I process a lot of videos every day, I have to work this way, I can't wait and I can't work by this way: make one press of the arrows and wait and then one more press and wait again and then again and again...- I wouldn't do anything.

I have a simple solution to this problem:
You should be add a new configuration parameter, and if the time difference between the keystroke and the time when VRD6 processes it is greater than the value of this parameter, VRD6 would do nothing. If the value of this parameter will be 0, everything would work in old manner.






In the source code of the application, it could be written like this:

if (Discard_timeout > 0 and (Current_time - Time_of_key_press) > Discard_timeout) then do_nothing_(discard_this_key_press)
else do_action_for_this_key_press()

 

[Dan203]

We can't actually do this. Windows uses a message queue, which is basically just a big list of events including key presses, mouse movements, etc... Programs process these messages one by one, oldest to newest. You process a message, remove it from the list, process the next, remove it from the list, etc... Which means that when you release the key a keyup message is added to the bottom of the list but VideoReDo doesn't get this message until after it's already processed all the keydown messages that came before it.

 

[jirim100]

We can't actually do this. Windows uses a message queue, which is basically just a big list of events including key presses, mouse movements, etc... Programs process these messages one by one, oldest to newest. You process a message, remove it from the list, process the next, remove it from the list, etc... Which means that when you release the key a keyup message is added to the bottom of the list but VideoReDo doesn't get this message until after it's already processed all the keydown messages that came before it.

Ok. I understand. Operating system to messages WM_KEYDOWN, WM_KEYUP not attaching time when they are added to system list.

But exist solution for this problem:
VRD6 create internal message list with times, and processing of messages will be in other thread:

Pseudo code:

class VRD6_message {
  int message;
  int wParam;
  int lParam;
  DATE_TIME date_time;
  ...
}

// gui thread
WindowProc(message, wParam, lParam) {
...
case WM_KEYDOWN: MessageList.Add_to_end_and_signalize_new_msg_arrived(new VRD6_Message(WM_KEYDOWN, wParam, lParam, getActualTime()))   // assign to message actual time
...
}

//different working thread:
thread_for_processing_messages() {
  ...
  while (Wait_until_some_message_in_list()) {  // WaitForSingeObject or similar
    VRD6_message message = MessageList.Get_first_message_and_remove_it();
    if (Discard_timeout > 0 and (Current_time() - message.date_time) > Discard_timeout) then do_nothing_(discard_this_key_press)
    else Do_action_for_this_key_press(message)
  }
  ...
}

Function Add_to_end_and_signalize_new_msg_arrived and Get_first_message_and_remove_it will be very fast. These never add significant delay and the whole logic will be work as intended. The time packed with message will be always very close to real time of the key press and slow processing will run in different thread.

Please, for me is this behavior, this "detail", very important.

 

[Dan203]

VideoReDo uses MFC, we don't process the message queue directly. MFC has its own internal message queue processing and simply calls event functions, like OnKeyUp and OnKeyDown, when it hits one of these messages. What you're describing would require a massive underlying change that would simply add too much effort and complexity for very little benefit.

I realize this issue is frustrating for you. Have you tried increasing the buffer sizes? That can help speed up seeking in the area directly around a big jump. Try increasing both the buffer sizes in the Stream Parameters section and the cache size in the H264/H265 section. That might help.

 

[jirim100]

What you're describing would require a massive underlying change that would simply add too much effort and complexity for very little benefit.

I can't agree. I am too programmer (20 years). "Very little?" - for me is it daily very important thing. Please, think about it.

Try increasing both the buffer sizes in the Stream Parameters section and the cache size in the H264/H265 section. That might help.

I already tried both of these (Buffer size: Min and Max is set to 2048, Cahe size: Max frames is 128 (I tried higher values too), none compression).

 

[Dan203]

I can't agree. I am too programmer (20 years). "Very little?" - for me is it daily very important thing. Please, think about it.

Sorry but it's just not possible. As I said we currently use MFC, which hides the queue from us and is not easy to override. Also we're currently in the process of porting the code to Qt, which obscures the message queue even more since it's cross platform, so it's actually impossible to override.

I get that it's important to you, but it's just not something we can fix, especially in the new code.

 

[Otter]

Cutting adverts is also what I primarily use VRD for and I use much the same approach, but mainly move by mouse, not arrow keystrokes.

VRD on my 2k monitor shows 21 frames in the preview bar, and the entire video on the timeline. I place my mouse on the timeline, hold down the left-mouse and drag along the timeline until I see ad content, I then backup by rolling the mouse wheel and finish fine-tuning cut location by clicking on the preview images to move small amounts.

There isn't any problem with VRD keeping up, even if I drag from one end of a 2 hour video to another in 1-2 secs (of course my eyes can't register much at that rate). VRD always keeps up and stops when I do - no lag and no carryover of movement.

 

[Dan203]

Seeking is highly dependent on the structure of the file. Mainly how many frames make up a GOP. i.e. how many B/P frames are between each I/IDR frame. We can only start decoding from an I/IDR frame, so the bigger the GOP the more we have to decode every time you try to move. With MPEG-2 it's easy as the max allowed is 32 frames. With H.264/H.265 there is no limit and I've actually seen files with 10 minute GOPs. (18,000 frames) Most H.264/H.265 have them in the 20-30 second range, which is more manageable, but even then it takes much better hardware to keep up. (faster drive and more memory for the buffers)

 

[jirim100]

... even if I drag from one end of a 2 hour video to another in 1-2 secs

With this is not problem!
Problem is when is queued too many key presses.

With MPEG-2 it's easy as the max allowed is 32 frames. With H.264/H.265 there is no limit and I've actually seen files with 10 minute GOPs. (18,000 frames) Most H.264/H.265 have them in the 20-30 second range

Thanks for explanation. But it's still for me very paintfull (with H264/H265 files on i9 intel with NVMe drive and 32 GB ram) because I have to work fastly.

Also we're currently in the process of porting the code to Qt,

Maybe time for make deeper interventions into the code ?
In Qt have to exist similar thread synchronizations as in MFC.

which obscures the message queue even more since it's cross platform

Maybe obscure but still exist concrete input point where the application processes key messages.

 

[Dan203]

I realize this is something ypu feel strongly about, but the effort required to make it happen is not worth the benefit. For most users it would be indistinguishable.