Sync with livekit's m125 (#42)

* Update to m125. (#119)

Use M125 as the latest version and migrate historical patches to m125

Patches Group:

## 1. Update README.md
webrtc-sdk/webrtc@b6c65fc
* Add Apache-2.0 license and some note to README.md. (#9)
* Updated readme detailing changes from original (#42)
* Adding membrane framework (#51)
* Updated readme (#83)

## 2. Audio Device Optimization
webrtc-sdk/webrtc@7454824
* allow listen-only mode in AudioUnit, adjust when category changes
(webrtc-sdk/webrtc#2)
* release mic when category changes
(webrtc-sdk/webrtc#5)
* Change defaults to iOS defaults
(webrtc-sdk/webrtc#7)
* Sync audio session config
(webrtc-sdk/webrtc#8)
* feat: support bypass voice processing for iOS.
(webrtc-sdk/webrtc#15)
* Remove MacBookPro audio pan right code
(webrtc-sdk/webrtc#22)
* fix: Fix can't open mic alone when built-in AEC is enabled.
(webrtc-sdk/webrtc#29)
* feat: add audio device changes detect for windows.
(webrtc-sdk/webrtc#41)
* fix Linux compile (webrtc-sdk/webrtc#47)
* AudioUnit: Don't rely on category switch for mic indicator to turn off
(webrtc-sdk/webrtc#52)
* Stop recording on mute (turn off mic indicator)
(webrtc-sdk/webrtc#55)
* Cherry pick audio selection from m97 release
(webrtc-sdk/webrtc#35)
* [Mac] Allow audio device selection
(webrtc-sdk/webrtc#21)
* RTCAudioDeviceModule.outputDevice / inputDevice getter and setter
(webrtc-sdk/webrtc#80)
* Allow custom audio processing by exposing AudioProcessingModule
(webrtc-sdk/webrtc#85)
* Expose audio sample buffers for Android
(webrtc-sdk/webrtc#89)
* feat: add external audio processor for android.
(webrtc-sdk/webrtc#103)
* android: make audio output attributes modifiable
(webrtc-sdk/webrtc#118)
* Fix external audio processor sample rate calculation
(webrtc-sdk/webrtc#108)
* Expose remote audio sample buffers on RTCAudioTrack
(webrtc-sdk/webrtc#84)
* Fix memory leak when creating audio CMSampleBuffer
webrtc-sdk/webrtc#86

## 3. Simulcast/SVC support for iOS/Android.
webrtc-sdk/webrtc@b0b9fe9

- Simulcast support for iOS SDK (#4)
- Support for simulcast in Android SDK (#3)
- include simulcast headers for mac also (#10)
- Fix simulcast using hardware encoder on Android (#48)
- Add scalabilityMode support for AV1/VP9. (#90)

## 4. Android improvements.
webrtc-sdk/webrtc@9aaaab5
- Start/Stop receiving stream method for VideoTrack (#25)
- Properly remove observer upon deconstruction (#26)
- feat: Expose setCodecPreferences/getCapabilities for android. (#61)
- fix: add WrappedVideoDecoderFactory.java. (#74)

## 5. Darwin improvements
webrtc-sdk/webrtc@a13ea17
- [Mac/iOS] feat: Add RTCYUVHelper for darwin. (#28)
- Cross-platform `RTCMTLVideoView` for both iOS / macOS (#40)
- rotationOverride should not be assign (#44)
- [ObjC] Expose properties / methods required for AV1 codec support
(#60)
- Workaround: Render PixelBuffer in RTCMTLVideoView (#58)
- Improve iOS/macOS H264 encoder (#70)
- fix: fix video encoder not resuming correctly upon foregrounding
(#75).
- add PrivacyInfo.xcprivacy to darwin frameworks. (#112)
- Add NSPrivacyCollectedDataTypes key to xcprivacy file (#114)
- Thread-safe `RTCInitFieldTrialDictionary` (#116)
- Set RTCCameraVideoCapturer initial zoom factor (#121)
- Unlock configuration before starting capture session (#122)

## 6. Desktop Capture for macOS.
webrtc-sdk/webrtc@841d78f
- [Mac] feat: Support screen capture for macOS. (#24) (#36)
- fix: Get thumbnails asynchronously. (#37)
- fix: Use CVPixelBuffer to build DesktopCapture Frame, fix the crash
caused by non-CVPixelBuffer frame in RTCVideoEncoderH264 that cannot be
cropped. (#63)
- Fix the crash when setting the fps of the virtual camera. (#62)

## 7. Frame Cryptor Support.
webrtc-sdk/webrtc@fc08745
- feat: Frame Cryptor (aes gcm/cbc). (#54)
- feat: key ratchet/derive. (#66)
- fix: skip invalid key when decryption failed. (#81)
- Improve e2ee, add setSharedKey to KeyProvider. (#88)
- add failure tolerance for framecryptor. (#91)
- fix h264 freeze. (#93)
- Fix/send frame cryptor events from signaling thread (#95)
- more improvements for E2EE. (#96)
- remove too verbose logs (#107)
- Add key ring size to keyProviderOptions. (#109)

## 8. Other improvements.
webrtc-sdk/webrtc@eed6c8a
- Added yuv_helper (#57)
- ABGRToI420, ARGBToI420 & ARGBToRGB24 (#65)
- more yuv wrappers (#87)
- Fix naming for yuv helper (#113)
- Fix missing `RTC_OBJC_TYPE` macros (#100)

---------

Co-authored-by: Hiroshi Horie <[email protected]>
Co-authored-by: David Zhao <[email protected]>
Co-authored-by: davidliu <[email protected]>
Co-authored-by: Angelika Serwa <[email protected]>
Co-authored-by: Théo Monnom <[email protected]>
# Conflicts:
#	README.md
#	media/engine/webrtc_video_engine.cc
#	media/engine/webrtc_video_engine.h
#	modules/audio_device/audio_device_impl.cc
#	sdk/BUILD.gn
#	sdk/android/BUILD.gn
#	sdk/android/api/org/webrtc/RtpParameters.java
#	sdk/android/api/org/webrtc/SimulcastVideoEncoder.java
#	sdk/android/api/org/webrtc/SimulcastVideoEncoderFactory.java
#	sdk/android/api/org/webrtc/VideoCodecInfo.java
#	sdk/android/src/jni/pc/rtp_parameters.cc
#	sdk/android/src/jni/simulcast_video_encoder.cc
#	sdk/android/src/jni/simulcast_video_encoder.h
#	sdk/android/src/jni/video_codec_info.cc
#	sdk/objc/api/peerconnection/RTCAudioDeviceModule+Private.h
#	sdk/objc/api/peerconnection/RTCAudioDeviceModule.h
#	sdk/objc/api/peerconnection/RTCAudioDeviceModule.mm
#	sdk/objc/api/peerconnection/RTCAudioTrack.mm
#	sdk/objc/api/peerconnection/RTCIODevice+Private.h
#	sdk/objc/api/peerconnection/RTCIODevice.mm
#	sdk/objc/api/peerconnection/RTCPeerConnectionFactory.h
#	sdk/objc/api/peerconnection/RTCPeerConnectionFactory.mm
#	sdk/objc/api/video_codec/RTCVideoEncoderSimulcast.h
#	sdk/objc/api/video_codec/RTCVideoEncoderSimulcast.mm
#	sdk/objc/base/RTCAudioRenderer.h
#	sdk/objc/components/video_codec/RTCVideoEncoderFactorySimulcast.h
#	sdk/objc/components/video_codec/RTCVideoEncoderFactorySimulcast.mm

* fix: duplicate simulcast entries

* remove duplicate declaration

* remove duplicate audioDeviceModule

* fix: removed livekit's external audio processor

* fix: add back simulcast factories

* Fix missing RTC_OBJC_TYPE macros

* Fix missing headers and Metal linking

# Conflicts:
#	sdk/BUILD.gn

* Fix Mac Catalyst `RTCCameraVideoCapturer` rotation (#126)

* Fix set frame transformer (#125)

* Fix webrtc_voice_engine not notifying mute change (#128)

Looks like this line was missed during the m125 update.

webrtc-sdk/webrtc@272127d#diff-56f5e0c459b287281ef3b0431d3f4129e8e4be4c6955d845bcb22210f08b7ba5R2289

Adding it back in so that mic is properly released when muted.
# Conflicts:
#	media/engine/webrtc_voice_engine.cc

* android: Allow for skipping checking the audio playstate if needed (#129)

Pausing/stopping the audio track can lead to a race condition against
the AudioTrackThread due to this assert. Normally this is fine since
directly pausing/stopping isn't possible, but user is using reflection
to workaround another audio issue (muted participants still have a
sending audio stream which keeps the audio alive, affecting global sound
if in the background).

Not a full fix, as would like to manually control the audio track
directly (needs a bigger fix to handle proper synchronization before
allowing public access), but this will work through reflection (user
takes responsibility for usage).

* Allow to pass in capture session to RTCCameraVideoCapturer (#132)

Expose initializers to pass in capture session to RTCCameraVideoCapturer
so we can use AVCaptureMultiCamSession etc to capture front and back
simultaneously for iOS.

* Fix NetworkMonitor race condition when dispatching native observers (#135)

There is a race condition in NetworkMonitor where native observers may
be removed concurrently with a notification being dispatched, leading to
a dangling pointer dereference (trying to dispatch an observer that was
already removed and destroyed), and from there a crash with access
violation.

By ensuring dispatching to native observers is done within the
synchronization lock that guards additions/removals of native observers
protects against this race condition. Since native observers callbacks
are posted to the networking thread in the C++ side anyway, there should
be no risk of deadlock/starvation due to long-running observers.

Bug: webrtc:15837
Change-Id: Id2b788f102dbd25de76ceed434c4cd68aa9a569e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/338643
Reviewed-by: Taylor Brandstetter <[email protected]>
Commit-Queue: Harald Alvestrand <[email protected]>
Reviewed-by: Harald Alvestrand <[email protected]>
Cr-Commit-Position: refs/heads/main@{#42256}

Co-authored-by: Guy Hershenbaum <[email protected]>

* Support for Vision Pro (#131)

TODO:
- [x]  fix compile for RTCCameraVideoCapturer
- [ ]  fix RTCMTLRenderer ?

---------

Co-authored-by: Hiroshi Horie <[email protected]>

* Multicam support (#137)

TODO:
- [x] Return `.systemPreferredCamera` for devices (visionOS only).
- [x] Use `AVCaptureMultiCamSession` only if `isMultiCamSupported` is
true.
- [x] Silence statusBarOrientation warning.

---------

Co-authored-by: [email protected] <[email protected]>

* tvOS support (#139)

17.0+ only atm

---------

Co-authored-by: cloudwebrtc <[email protected]>

* Add isDisposed to MediaStreamTrack (#140)

* chore: handle invalid cipher from key size. (#142)

* Allow software AEC for Simulator (#143)

~Allow to use "googEchoCancellation" constraint for software AEC.
For devices "googEchoCancellation" should be false to use
VoiceProcessingIO.~

* Fix AudioRenderer crash & expose AVAudioPCMBuffer (#144)

* fix: Fix bug for bypass voice processing. (#147)

* chore: remove aes cbc for framecryptor. (#145)

* Change audio renderer output format (#149)

Instead of converting to Float, output original Int data without
conversion.
Output the raw format and convert when required.

* Fixed issue with missing network interfaces on iOS (#151)

Related issue: webrtc-sdk/webrtc#148
Cherry-pick :
https://webrtc.googlesource.com/src/+/fea60ef8e72fb17b4f8a5363aff7e63ab8027b4f

Fixed issue with network interfaces due to a missing return value in the
"nw_path_enumerate_interfaces(...)" block. Exposed in iOS 18,
RTCNetworkMonitor::initWithObserver will only enumerate the first
interface, instead of all device interfaces

Bug: webrtc:359245764
Change-Id: Ifb9f28c33306c0096476a4afb0cdb4d734e87b2c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/359541
Auto-Submit: Corby <[email protected]>
Commit-Queue: Jonas Oreland <[email protected]>
Reviewed-by: Kári Helgason <[email protected]>
Reviewed-by: Jonas Oreland <[email protected]>
Cr-Commit-Position: refs/heads/main@{#42818}

Co-authored-by: Corby Hoback <[email protected]>

* Custom audio input for Android (#154)

# Conflicts:
#	sdk/android/api/org/webrtc/audio/JavaAudioDeviceModule.java
#	sdk/android/src/java/org/webrtc/audio/WebRtcAudioRecord.java

---------

Co-authored-by: CloudWebRTC <[email protected]>
Co-authored-by: Hiroshi Horie <[email protected]>
Co-authored-by: davidliu <[email protected]>
Co-authored-by: Guy Hershenbaum <[email protected]>
Co-authored-by: Corby Hoback <[email protected]>

Author: 6 people

sdk/BUILD.gn

@@ -119,6 +119,7 @@ if (is_ios || is_mac) {
       "objc/base/RTCVideoFrame.mm",
       "objc/base/RTCVideoFrameBuffer.h",
       "objc/base/RTCVideoRenderer.h",
+      "objc/base/RTCAudioRenderer.h",
       "objc/base/RTCYUVPlanarBuffer.h",
     ]
 

sdk/android/BUILD.gn

@@ -924,23 +924,6 @@ if (current_os == "linux" || is_android) {
     ]
   }
 
-  rtc_library("simulcast_jni") {
-    visibility = [ "*" ]
-    allow_poison = [ "software_video_codecs" ]
-    sources = [
-      "src/jni/simulcast_video_encoder.cc",
-      "src/jni/simulcast_video_encoder.h",
-      "src/jni/simulcast_video_encoder_factory.cc",
-      "src/jni/simulcast_video_encoder_factory.h"
-    ]
-    deps = [
-      ":base_jni",
-      ":video_jni",
-      ":native_api_codecs",
-      "../../media:rtc_simulcast_encoder_adapter"
-    ]
-  }
-
   rtc_library("libaom_av1_encoder_jni") {
     visibility = [ "*" ]
     allow_poison = [ "software_video_codecs" ]
@@ -971,7 +954,9 @@ if (current_os == "linux" || is_android) {
     allow_poison = [ "software_video_codecs" ]
     sources = [
       "src/jni/simulcast_video_encoder.cc",
-      "src/jni/simulcast_video_encoder.h"
+      "src/jni/simulcast_video_encoder.h",
+      "src/jni/simulcast_video_encoder_factory.cc",
+      "src/jni/simulcast_video_encoder_factory.h"
     ]
     deps = [
       ":base_jni",
@@ -993,7 +978,6 @@ if (current_os == "linux" || is_android) {
       ":generated_swcodecs_jni",
       ":libvpx_vp8_jni",
       ":libvpx_vp9_jni",
-      ":simulcast_jni",
       ":native_api_jni",
       ":simulcast_jni",
       ":video_jni",

sdk/objc/api/peerconnection/RTCAudioTrack+Private.h

@@ -29,8 +29,6 @@ NS_ASSUME_NONNULL_BEGIN
                          source:(RTC_OBJC_TYPE(RTCAudioSource) *)source
                         trackId:(NSString *)trackId;
 
-- (void)didCaptureSampleBuffer:(CMSampleBufferRef)sampleBuffer;
-
 @end
 
 NS_ASSUME_NONNULL_END

sdk/objc/api/peerconnection/RTCRtpEncodingParameters.mm

@@ -102,6 +102,9 @@ - (instancetype)initWithNativeParameters:
   if (_scalabilityMode != nil) {
     parameters.scalability_mode = std::optional<std::string>(std::string([_scalabilityMode UTF8String]));
   }
+  if (_scalabilityMode != nil) {
+    parameters.scalability_mode = absl::optional<std::string>(std::string([_scalabilityMode UTF8String]));
+  }
   parameters.bitrate_priority = _bitratePriority;
   parameters.network_priority = [RTC_OBJC_TYPE(RTCRtpEncodingParameters)
       nativePriorityFromPriority:_networkPriority];

sdk/objc/api/peerconnection/RTCRtpTransceiver.h

@@ -16,6 +16,8 @@
 
 @class RTC_OBJC_TYPE(RTCRtpCodecCapability);
 
+@class RTC_OBJC_TYPE(RTCRtpCodecCapability);
+
 NS_ASSUME_NONNULL_BEGIN
 
 extern NSString *const RTC_CONSTANT_TYPE(RTCRtpTransceiverErrorDomain);
@@ -125,6 +127,8 @@ RTC_OBJC_EXPORT
     NSArray<RTC_OBJC_TYPE(RTCRtpHeaderExtensionCapability) *>
         *negotiatedHeaderExtensions;
 
+@property(nonatomic, copy) NSArray<RTC_OBJC_TYPE(RTCRtpCodecCapability) *> *codecPreferences;
+
 /** The currentDirection attribute indicates the current direction negotiated
  *  for this transceiver. If this transceiver has never been represented in an
  *  offer/answer exchange, or if the transceiver is stopped, the value is not

sdk/objc/base/RTCVideoCodecInfo.h

@@ -32,6 +32,11 @@ RTC_OBJC_EXPORT
              scalabilityModes:(nullable NSArray<NSString *> *)scalabilityModes
     NS_DESIGNATED_INITIALIZER;
 
+- (instancetype)initWithName:(NSString *)name
+                  parameters:(nullable NSDictionary<NSString *, NSString *> *)parameters
+             scalabilityModes:(nullable NSArray<NSString *> *)scalabilityModes
+    NS_DESIGNATED_INITIALIZER;
+
 - (BOOL)isEqualToCodecInfo:(RTC_OBJC_TYPE(RTCVideoCodecInfo) *)info;
 
 @property(nonatomic, readonly) NSString *name;

sdk/objc/components/audio/RTCAudioSession+Configuration.mm

@@ -97,6 +97,33 @@ - (BOOL)setConfiguration:
     }
   }
 
+  if (self.mode != configuration.mode) {
+    NSError *modeError = nil;
+    if (![self setMode:configuration.mode error:&modeError]) {
+      RTCLogError(@"Failed to set mode to %@: %@",
+                  self.mode,
+                  modeError.localizedDescription);
+      error = modeError;
+    } else {
+      RTCLog(@"Set mode to: %@", configuration.mode);
+    }
+  }
+
+  // Sometimes category options don't stick after setting mode.
+  if (self.categoryOptions != configuration.categoryOptions) {
+    NSError *categoryError = nil;
+    if (![self setCategory:configuration.category
+               withOptions:configuration.categoryOptions
+                     error:&categoryError]) {
+      RTCLogError(@"Failed to set category options: %@",
+                  categoryError.localizedDescription);
+      error = categoryError;
+    } else {
+      RTCLog(@"Set category options to: %ld",
+             (long)configuration.categoryOptions);
+    }
+  }
+
   if (self.preferredSampleRate != configuration.sampleRate) {
     NSError *sampleRateError = nil;
     if (![self setPreferredSampleRate:configuration.sampleRate

sdk/objc/components/capturer/RTCCameraVideoCapturer.h

@@ -36,6 +36,17 @@ RTC_OBJC_EXPORT
 + (CGFloat)defaultZoomFactorForDeviceType:(AVCaptureDeviceType)deviceType;
 #endif
 
+- (instancetype)initWithDelegate:
+    (nullable __weak id<RTC_OBJC_TYPE(RTCVideoCapturerDelegate)>)delegate;
+
+- (instancetype)initWithDelegate:
+                    (nullable __weak id<RTC_OBJC_TYPE(RTCVideoCapturerDelegate)>)delegate
+                  captureSession:(AVCaptureSession *)captureSession;
+
+#if !TARGET_OS_VISION
++ (CGFloat)defaultZoomFactorForDeviceType:(AVCaptureDeviceType)deviceType;
+#endif
+
 - (instancetype)initWithDelegate:
     (nullable __weak id<RTC_OBJC_TYPE(RTCVideoCapturerDelegate)>)delegate;