Handling frame sync in multi-source environments

[pixelplow]

My company is building a switcher app that affords NDI inputs and outputs, among other types. We'd like to lean more heavily on NDI, but I have a dilemma with syncing multiple input sources. Our test input sources, thus far, are Magewell HDMI-to-NDI conversion boxes, the Sienna iPhone apps, and Scan Converter instances. I've noticed occasional differences in processing latencies between those sources, usually of only a frame or two at 30p. I know NDI has the notion of timestamping and timecode, but how does this realistically work in a multi-source environment?

It would seem that timestamping isn't at all useful as a cross-device sync mechanism unless every output source somehow derived its clocking from the same source. It's unclear to me how each of my test devices would come up with a time with which to stamp on output frames. Is it just their system clock, which is used to derive the Unix epoch time at the point of frame creation? If so, that seems fragile at best. Does NDI have a mechanism for centralized time signal distribution to overcome this problem? You know, a more traditional timecode-kind-of-thing? I'd hoped that was part of the Discovery server, but that doesn't appear to be the case.

[pixelplow]

Nobody knows how NDI handles this? Interesting. Seems like it would be kind of fundamental in terms of being a requirement, especially for larger scale NDI deployments. Maybe I found the Achilles heel of NDI?

[kanep]

In the TriCaster products there are video and audio sync options available on a per input basis so that you can time sources together that have dissimilar latency. This isn't just for NDI, but any kind of input, it will let you adjust SDI sources as well.

[livepad]

Nobody knows how NDI handles this? Interesting. Seems like it would be kind of fundamental in terms of being a requirement, especially for larger scale NDI deployments. Maybe I found the Achilles heel of NDI?

If you are really that concerned you can use NTP or PTPv2 to synchronise all the clocks of the devices (if they support adjustment of their system clock - clearly some devices may not).
Then all your NDI Timestamps would be based on a common origin.

[pixelplow]

If you are really that concerned you can use NTP or PTPv2 to synchronise all the clocks of the devices (if they support adjustment of their system clock - clearly some devices may not).
Then all your NDI Timestamps would be based on a common origin.

Yeah, I found a reference late yesterday that said "devices" can use NTP to sync their internal clocks, which is used by NDI to generate timestamps. That was on a bit of marketing literature from 2015. Seems like this kind of information should be more obvious in documentation somewhere. Certainly would be good for Newtek to document what the protocol expects, or at least how their implementations of it work.

I'm pursuing Magewell to get them to add NTP sync to their firmware. At present, there isn't any ability to set date or time to anything on the Pro Convert's.

[pixelplow]

In the TriCaster products there are video and audio sync options available on a per input basis so that you can time sources together that have dissimilar latency. This isn't just for NDI, but any kind of input, it will let you adjust SDI sources as well.

Yeah, that's what we are building into our video switcher product. That intelligence needs to be driven by information, though, from each source. Many of these sources have variable latency on each launch of a listener receiving an NDI stream from them. Manually-defined, fixed latencies clearly aren't an option in that environment. Hence, while our switcher can also delay individual incoming streams, we'd have to know how much to delay each one every time the stream was instantiated. You can only do that when the timestamping that is happening on every device is based on a fixed, common reference point.