A filter that can read pixel data?

[Garvey]

Hi, I'm trying to create a filter that can read per frame pixel data. I am trying to follow this thread:

https://obsproject.com/forum/threads/grabbing-frame-from-source-video-capture-device.48967/

In there, it was recommended to use async-delay-filter. However, I am not finding that available in the drop down menu for filters in the GUI.

I noticed in the code, under \plugins\obs-filters\async-delay-filter.c, near the end there is this:

struct obs_source_info async_delay_filter = {
.id = "async_delay_filter",
.type = OBS_SOURCE_TYPE_FILTER,
.output_flags = OBS_SOURCE_VIDEO | OBS_SOURCE_ASYNC,
.get_name = async_delay_filter_name,
.create = async_delay_filter_create,
.destroy = async_delay_filter_destroy,
.update = async_delay_filter_update,
.get_properties = async_delay_filter_properties,
.filter_video = async_delay_filter_video,
#ifdef DELAY_AUDIO
.filter_audio = async_delay_filter_audio,
#endif
.filter_remove = async_delay_filter_remove,
};

It uses the .filter_video function, which seems to be different from the other filters. For example, in \plugins\obs-filters\chroma-key-filter.c, we have

struct obs_source_info chroma_key_filter = {
.id = "chroma_key_filter",
.type = OBS_SOURCE_TYPE_FILTER,
.output_flags = OBS_SOURCE_VIDEO,
.get_name = chroma_key_name,
.create = chroma_key_create,
.destroy = chroma_key_destroy,
.video_render = chroma_key_render,
.update = chroma_key_update,
.get_properties = chroma_key_properties,
.get_defaults = chroma_key_defaults,
};

It doesn't seem to use the .filter_video function, which I need. I tried adding one, but it didn't seem to get called. I next tried changing the .output_flags to include OBS_SOURCE_ASYNC, but after I did that the chroma filter also vanished from the GUI.

So my changes to to \plugins\obs-filters\chroma-key-filter.c look like this:

static struct obs_source_frame *MY_FILTER_FUNC(void *data, struct obs_source_frame * frame)
{
// This function is not getting called //
return frame;
}

struct obs_source_info chroma_key_filter = {
.id = "chroma_key_filter",
.type = OBS_SOURCE_TYPE_FILTER,
.output_flags = OBS_SOURCE_VIDEO | OBS_SOURCE_ASYNC, // adding OBS_SOURCE_ASYNC causes the filter to disappear from dropdown menu
.get_name = chroma_key_name,
.create = chroma_key_create,
.destroy = chroma_key_destroy,
.video_render = chroma_key_render,
.update = chroma_key_update,
.get_properties = chroma_key_properties,
.get_defaults = chroma_key_defaults,
.filter_video = MY_FILTER_FUNC, // My filter video function I'm trying to add
};

Any help appreciated, I would just like to be able to edit some filter so that I can read pixel data from the captured video frames.

Thanks

Edit: I was told that async filters are not available from a "Display Capture" source, which is probably why I wasn't seeing them. I guess that means I cannot use an async filter to solve this problem, as indicated by the thread I linked? In that case I am pretty stumped as to where to go from here.

 

[mashitianxia]

Hi there,
I read through your thread on grabbing frame data from a source filter, and I’m running into exactly the same issue you described.
I’m trying to create a custom OBS filter that allows me to access pixel data for each video frame in real-time. I followed your observations and attempted to use .filter_video() with an OBS_SOURCE_ASYNC flag, but like you, I noticed that once I add OBS_SOURCE_ASYNC to the output flags, the filter disappears from the GUI (presumably because Display Capture doesn’t support async filters).
I also tried modifying a synchronous filter like chroma-key-filter.c by adding a .filter_video function and changing the flags, but .filter_video() still doesn’t get called — just like in your case.
I saw your note about async filters not working with Display Capture. Given that constraint, did you find any workaround that allows reading per-frame pixel data from a filter — especially for synchronous sources like Display Capture?
Any suggestions or insights would be hugely appreciated!
Thanks in advance.

 

[ahmad34]

You're right—async filters won’t show for sources like Display Capture since they're not async sources. Try using a video source like a capture card or webcam instead, or hook into the video_render path for sync sources.