C++ | Using NDIlib_video_frame_v2_t with other programs || Using NDI as webcam

yashinski.dev

New member
Hi
I am trying to integrate NDI with openCV in a custom C++ program
It's goal is to connect an NDI source to OpenCV and run a face detection on it with the option to change a source during runtime.

Currently,
The program can connect to a specified NDI source.
The OpenCV code is already written and uses a webcam as a video source.
The NDI frames are not used in any way, just received.

The missing stage taking these NDI frames and telling the computer "Hey! this is a webcam, you can use it for anything!"
After it's registered as a webcam the OpenCV is able to capture from it easily.

How can I acheive this?
I read a little about ffmpeg and directshow but couldn't rally understand how I'm supposed to use them in this case.
Also, can this actually be acheived in a single program? or would I need to write a seperate program to route the incoming video for the OpenCV one?

Hope I managed to explain my problem,
Thanks in advance,
Nimrod.
 

yashinski.dev

New member
Update:

This is more of an OpenCV issue I think, but also might be solved with the right conversion (which I have little to no knowledge on how to do)

So I've managed to get the actual pixel data like this:
Code:
NDIlib_source_t source1;
	source1.p_ndi_name = "BIRDDOG-XXXXX (SDI)";
	source1.p_url_address = NULL;

NDIlib_recv_connect(pNDI_recv, &source1);

NDIlib_video_frame_v2_t video_frame;
NDIlib_audio_frame_v2_t audio_frame;
NDIlib_metadata_frame_t metadata_frame;
while (true) {
	switch (NDIlib_recv_capture_v2(pNDI_recv, &video_frame, &audio_frame, &metadata_frame, 1000))
	{
			case NDIlib_frame_type_none:
				break;
			case NDIlib_frame_type_video:
				uint8_t* data = video_frame.p_data;
                                printf((char8)data);
         }
}


But when entered into open cv, like this:
Code:
cv::Mat mat1(yres, xres, CV_8UC3, data);
cv::imshow("Display window", mat1);

I receive this error:
Code:
Exception thrown at 0x000007FEB5819AB3 (opencv_world342.dll) in 
opencvtest.exe: 0xC0000005: Access violation reading location 0x0000000003F65000.


I'd really appreciate any help since I don't know where else to look anymore,
Thanks in advance again,
Nimrod.
 

labi

New member
hi Nimrod!

Do you have any working solution for this feature (OpenCV + NDI frame)?

Thanks,
Lorant
 

Jarno

Post-LW Engineer
It should work, as long as the format of the video frame and the format set of the cv matrix match. To get going with this you'll probably want to create the NDI receiver using NDIlib_recv_color_format_BGRX_BGRA or NDIlib_recv_color_format_RGBX_RGBA, with the cv mat using CV_8UC4 (to match the 4 elements per pixel).
Then when that is working correctly, the more advanced next step would be to have the entire pipeline use UYVY instead of RGB to avoid color conversions.

My guess is that Nimrod's crash is due to the NDI video frame being in UYVY format, which is effectively 16 bits per pixel, while CV_8UC3 expects 24 bits per pixel (in RGB or possibly BGR). So there isn't enough data to fill the matrix and it crashes running off the end of the data.

---JvdL---
 

labi

New member
It should work, as long as the format of the video frame and the format set of the cv matrix match. To get going with this you'll probably want to create the NDI receiver using NDIlib_recv_color_format_BGRX_BGRA or NDIlib_recv_color_format_RGBX_RGBA, with the cv mat using CV_8UC4 (to match the 4 elements per pixel).
Then when that is working correctly, the more advanced next step would be to have the entire pipeline use UYVY instead of RGB to avoid color conversions.

My guess is that Nimrod's crash is due to the NDI video frame being in UYVY format, which is effectively 16 bits per pixel, while CV_8UC3 expects 24 bits per pixel (in RGB or possibly BGR). So there isn't enough data to fill the matrix and it crashes running off the end of the data.

---JvdL---
Thanks so much your response indeed! I'll try to implement!
 
Top Bottom