Bug 1216393

Summary: Electron/Chromium can't render h264 in webrtc
Product: [openSUSE] openSUSE Tumbleweed Reporter: Björn Bidar <bjorn.bidar>
Component: OtherAssignee: Callum Farmer <gmbr3>
Status: NEW --- QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: Andreas.Stieger, bjorn.bidar, brunopitrus, gmbr3, m.szczepaniak.000
Version: Current   
Target Milestone: ---   
Hardware: All   
OS: openSUSE Tumbleweed   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Björn Bidar 2023-10-18 21:35:23 UTC
*  Description:

When trying to stream WebRTC with h264 in Electron/Chromium the screen stays black even thou OpenH264 is correctly installed.

* How to reproduce:
1. Open Chromium
2. Open  https://mozilla.github.io/webrtc-landing/pc_test_no_h264.html
3. Click test

** Another option is to use:
https://mozilla.github.io/webrtc-landing/pc_test.html
1. Tick require h264
2. Start the video, wait
3. The message now should be "No H264 found in the offer!!!"  


* Background and reason for the bug:

openSUSE builds both Chromium and Electron in a way  that h264 isn't compiled into both packages
but loaded using the Openh264 provided by openSUSE but blessed by Cisco.

However Chromium has to use some of the openh264 to build even thou it's not used during runtime.

The gyp option `rtc_use_h264` has be set to true and third_party/openh264 has to 
be included in the tarball (already the case in the upstream third_party/openh264 tarball).

This is due webrtc using openh264 headers in third_party/webrtc/modules/video_coding/codecs/h264/h264_encoder_impl.cc
Comment 1 Bruno Pitrus 2023-10-19 14:36:40 UTC
We cannot build chromium against openH264 as legal does not allow to ship that library in Factory.

What we can do is to patch it to detect whether the library is available and dlopen it. There is a patch from Fedora that purpots to do this:
https://src.fedoraproject.org/rpms/qt6-qtwebengine/blob/rawhide/f/qtwebengine-webrtc-dlopen-h264.patch

I'm not sure what's the use case for H264 RTC support in Electron. Does Element need it? No other apps use that functionality (Signal has its own RTC implementation).
Comment 2 Callum Farmer 2023-10-19 16:54:18 UTC
(In reply to dziobian from comment #1)
> We cannot build chromium against openH264 as legal does not allow to ship
> that library in Factory.

It is now permitted through Cisco distribution of openSUSE rpms

> 
> What we can do is to patch it to detect whether the library is available and
> dlopen it. There is a patch from Fedora that purpots to do this:
> https://src.fedoraproject.org/rpms/qt6-qtwebengine/blob/rawhide/f/
> qtwebengine-webrtc-dlopen-h264.patch

I believe there's an openh264-dlopen-headers.tar.xz somewhere but I haven't tested Chromium integration

> 
> I'm not sure what's the use case for H264 RTC support in Electron. Does
> Element need it? No other apps use that functionality (Signal has its own
> RTC implementation).

I'll have a greater look when I'm available next week

Callum F
Comment 3 Bruno Pitrus 2023-10-19 16:56:45 UTC
I know of the Cisco repo. You can't build Factory packages against it or depend on it from Factory so the point stands.
Comment 4 Callum Farmer 2023-10-19 17:05:19 UTC
(In reply to dziobian from comment #3)
> I know of the Cisco repo. You can't build Factory packages against it or
> depend on it from Factory so the point stands.

It'll work if it can be changed to dlopen() instead of linking and by using the dlopen-headers tarball I mentioned (I think from openSUSE ffmpeg).  It may be possible or it may not be.
Comment 5 Björn Bidar 2023-10-19 17:26:07 UTC
(In reply to dziobian from comment #1)
> We cannot build chromium against openH264 as legal does not allow to ship
> that library in Factory.

Is that also true when only the headers are used and openh264 is used later as dlopen?

From what I saw that's what I assumed, it does sound strange that ffmpeg h264 isn't enough.

> What we can do is to patch it to detect whether the library is available and
> dlopen it. There is a patch from Fedora that purpots to do this:
> https://src.fedoraproject.org/rpms/qt6-qtwebengine/blob/rawhide/f/qtwebengine-webrtc-dlopen-h264.patch
> 
> I'm not sure what's the use case for H264 RTC support in Electron. Does
> Element need it? No other apps use that functionality (Signal has its own
> RTC implementation).

There's a Teams app for Electron that requires H264 since that's the only codec supported in Teams.
I think there are others that use the builtin webrtc support.

I assume the Signal RTC support is also based on the same Chromium library.
Taking a quick look at their repo it does look like it:
https://github.com/signalapp/webrtc/tree/main
Comment 6 Bruno Pitrus 2023-10-19 19:14:36 UTC
The last time i was forced to use it, Teams used lots of binary modules (at least on Windows). They will likely not work with another version of Electron, and we cannot rebuild them since we don't have the source.

As for Signal, I know that since i'm a maintainer. They use their own heavily-patched webrtc copy and put it in a rust wrapper which is then loaded as a node module: https://build.opensuse.org/package/show/network:im:signal/nodejs-signal-ringrtc

Signal does not seem to use the web APIs for RTC. It is not possible to use Electron's copy of webrtc in any other way, since it does not expose the C/C++ bindings.
Comment 7 Bruno Pitrus 2023-10-19 19:18:59 UTC
> From what I saw that's what I assumed, it does sound strange that ffmpeg
> h264 isn't enough.

In fact, ffmpeg's h264 is enough for *decoding* video which works out of the box in Chromium, Electron and QtWebEngine.

The webcodecs API and WebRTC don't use ffmpeg.
Comment 8 Björn Bidar 2023-10-19 19:23:24 UTC
> The last time i was forced to use it, Teams used lots of binary modules (at least on Windows). They will likely not work with another version of Electron, and we cannot rebuild them since we don't have the source.

There's a foss versiono of the Electron client. The old Teams client is deprecated, new client is based on Edges version of Electron.
https://github.com/IsmaelMartinez/teams-for-linux
Comment 9 Björn Bidar 2023-10-19 19:25:29 UTC
> As for Signal, I know that since i'm a maintainer. They use their own heavily-patched webrtc copy and put it in a rust wrapper which is then loaded as a node module: https://build.opensuse.org/package/show/network:im:signal/nodejs-signal-ringrtc

Even thou it's a fork the logic of loading openh264 runtime should still apply I think.
Comment 10 Bruno Pitrus 2023-10-19 19:39:44 UTC
(In reply to Björn Bidar from comment #8)
> There's a foss versiono of the Electron client. The old Teams client is
> deprecated, new client is based on Edges version of Electron.
> https://github.com/IsmaelMartinez/teams-for-linux
The client you're promoting seems to have a “feature” allowing whatever website it loads to execute arbitrary code on your computer:
https://github.com/IsmaelMartinez/teams-for-linux/blob/84aaf108e9ee7142d3d0fd8276530f8e87d49683/app/mainAppWindow/index.js#L460
As for what you mean by “Edges version of Electron”, i have no idea.


(In reply to Björn Bidar from comment #9)
> Even thou it's a fork the logic of loading openh264 runtime should still
> apply I think.

Signal does not use h264, only vpx (which is not encumbered by unjust laws) for RTC.
Comment 11 Björn Bidar 2023-10-19 20:36:27 UTC
(In reply to dziobian from comment #10)
> (In reply to Björn Bidar from comment #8)
> > There's a foss versiono of the Electron client. The old Teams client is
> > deprecated, new client is based on Edges version of Electron.
> > https://github.com/IsmaelMartinez/teams-for-linux
> The client you're promoting seems to have a “feature” allowing whatever
> website it loads to execute arbitrary code on your computer:
> https://github.com/IsmaelMartinez/teams-for-linux/blob/
> 84aaf108e9ee7142d3d0fd8276530f8e87d49683/app/mainAppWindow/index.js#L460

Calling it a feature with the intention to allow arbitrary code to be executed  in this way sounds like slanderous term. I don't know the indent, anyway I will report/ask.

> As for what you mean by “Edges version of Electron”, i have no idea.
https://techcommunity.microsoft.com/t5/microsoft-teams/teams-2-0-moves-away-from-electron-to-embrace-edge-webview2/td-p/2484565
> 
> (In reply to Björn Bidar from comment #9)
> > Even thou it's a fork the logic of loading openh264 runtime should still
> > apply I think.
> 
> Signal does not use h264, only vpx (which is not encumbered by unjust laws)
> for RTC.
OK good to know.