webrtcHacks

WebRTC actually feels exceptionally tinkerable to me, with human(-ish) readable SDP offers, public RFCs, various implementations, and importantly it being accessible from JavaScript playgrounds right within the browser! I've learned a lot about it over the years by following https://webrtchacks.com/, which does just that, i.e. taking apart various open and closed VoIP implementations (WebRTC or otherwise) and seeing how they work.

> The main progress has been WebRTC Yes, it is a kind of combination of the best approaches to NAT traversal from the SIP world, supporting both STUN & TURN and ICE with the Trickle addon. Some very good references on the topic I found recently: https://webrtchacks.com/ and https://www.html5rocks.com/en/tutorials/webrtc/infrastructure/.

> Cloudflare has been doing some amazing stuff with live video. webrtchacks posits Cloudflare uses pion for ICE, an amazing software you built! https://webrtchacks.com/how-cloudflare-glares-at-webrtc-with-whip-and-… / https://archive.is/irgee > I am so glad they did WHIP/WHEP support. Here's their recent blogpost: https://blog.cloudflare.com/cloudflare-calls-anycast-webrtc / https://archive.is/qdyJS

mediasoup (server side, so the Node + C++ component you mean) does not implement "RTCPeerConnection". That's just needed for browsers. In mediasoup we don't use SDP but specific RTP parameters as defined here: https://mediasoup.org/documentation/v3/mediasoup/rtp-parameters-and-ca… If you want to know why we don't use SDP (as communication means between client and server) here a good reading: https://webrtchacks.com/webrtc-sdp-inaki-baz-castillo/

There was a bunch of abuse that already happened with host candidates, was fixed recently though! [0] Tor also did a bunch of really great work figuring out all the ways you can be fingerprinted.[1] When I shared [2] a lot of people gave me flack for enabling malware. I don't come from that background, so hard to think 'how can people abuse this technology' It is too late now to roll back all the WebRTC stuff in the browser now though :) Definitely would be mind blowing how much data is flowing because of it (and how much money is being made because of it). [0] https://webrtchacks.com/dear-ny-times/ [1] https://trac.torproject.org/projects/tor/wiki/doc/Snowflake/Fingerprin… [2] https://github.com/pion/offline-browser-communication

Any sources on that? I'm pretty sure it's a heavy re-architecting at the very least, but if it's the same team and lead, have they gone entirely from scratch or repurposed some of Hipchat's code? (just questioning) Not curious enough to do the kind of blackbox exploration folks at https://webrtchacks.com or other sites do :)

Hangouts uses a bunch of non-standard Hangouts-specific APIs in Chrome. Maybe it's be best to call it "Google-RTC". https://webrtchacks.com/hangout-analysis-philipp-hancke/ In summary, they use weaker encryption than allowed by the spec, nonstandard ICE (firewall hole punching), nonstandard data channels, and an abandoned way to do multistream.

We started our journey with Janus about three months ago and I can just fully recommend it. It is an amazing well-written piece of software which is just as flexible and integrative as developers wish. E.g. Slack used Janus, at least in 2016 [0]. It is important to understand: Janus offers the ingredients for building great WebRTC applications (examples [1]), whereas Jitsi is more like a ready-to-go solution and got much more attentation as Janus did. Lorenzo and his colleagues are doing a really great job. In the space of SFU/MFU, one really needs to decide beforehand what kind of solution is suitable for which requirement. I have chosen Janus because we could integrate it by 100% in our software. For example, I was also looking into Jitsi. But compared to Janus it feeled so much more complicated and not suited for that specific job. However, it is important to point out, that this is no a ready-to-go solution. There is a long list of things you will have to dig into: - ICE (a way to connect if you switch between WIFI and LAN or to punch a hole into your fw) [2] - Cross-browser compatibility (Thank you iOS [4]) - TURN/STUN (Which matrix of udp/tcp and ports is needed for Hole Punching?), I recommend coturn. - Scalability: How many clients are planned? In my experience, CPU and bandwith are bottlenecks, we went with horizontal scaling - How do you gonna test your WebRTC application? So far great results with https://testrtc.com, but you probably also could accomplish…

Btw, found this which explains a bit when you need a TURN server: https://webrtchacks.com/symmetric-nat/

Ideally I would not have to trust the signalling server to ensure I'm transferring files to who I think I am[0]. Not as much of a knock against your project as a knock against webRTC in general. [0]: https://webrtchacks.com/webrtc-and-man-in-the-middle-attacks/

Here is a good write up on how the WebRTC implementation works for their new browser based calls. https://webrtchacks.com/facetime-finally-faces-webrtc-implementation-d…

Yes! Once it's available in WebRTC, we'll implement end-to-end encryption for group calls. There is progress being made in this direction: https://webrtchacks.com/true-end-to-end-encryption-with-webrtc-inserta…

I think Google Meet is WebRTC based, too. Presumably most browser based tools are? Though Zoom apparently isn't completely WebRTC: https://webrtchacks.com/zoom-avoids-using-webrtc/ Fwiw, WebRTC doesn't imply p2p/serverless.

Chad Hart just posted his observations on using the getDisplayMedia API itself. https://webrtchacks.com/jitsi-recording-getdisplaymedia-audio/

I think the biggest difference is that Zoom's implementation will be opaque, and Jitsi (and mediasoup's, etc.) will not be. https://webrtchacks.com/true-end-to-end-encryption-with-webrtc-inserta… https://jitsi.org/blog/e2ee/ https://github.com/versatica/mediasoup/issues/383

How does zoom's implementation compare with the insertable stream approach that I think Jitsi were also talking about? https://webrtchacks.com/true-end-to-end-encryption-with-webrtc-inserta…

Does anyone know if Zoom is built on any open source video streaming tech? I’m have a hard time believing their product isn’t going to meet massive amounts of competition very soon. Edit: After some Googling - https://webrtchacks.com/zoom-avoids-using-webrtc/

Actually, if you are using 4+ rooms, we will in many cases be better than some other webrtc services because we for these bigger rooms use a server in the middle to distribute all the streams. That will result in lower resource use. You only encode and send once, and it is distributed by the server. But if other webrtc kills CPU on 4 or less, we probably won't be much better. This "more than 4 room" is not p2p and end-to-end encrypted, since the server (SFU) needs to read and change some headers. So it is a tradeoff. There's now "insertable streams" which allow you to do encryption on the media client side, but it's a test for now: https://webrtchacks.com/true-end-to-end-encryption-with-webrtc-inserta…

https://www.html5rocks.com/en/tutorials/webrtc/infrastructure/ For STUN/TURN https://docs.bigbluebutton.org/support/faq.html For implementation details like average network usage and storage and other things http://www.aosabook.org/en/jitsi.html Outdated but seemed the most relevant so had to include https://webrtchacks.com/sfu-cascading/ By a jitsi dev apparently I'm no expert but have gotten into this whole thing trying to get one set up for my university. One problem I've come to find is that no one seems to discuss about is the total network usage if you are hosting this on cloud. The egress on this seem bankruptable on any of the major cloud providers. How are people doing self hosting if they don't have an idle Intel xeon on prem? Where else can I get affordable egress pricing on ~35 TB/ month / 250 users?

>What I don't understand is why the server needs to decrypt the traffic at all In short, to make it scale for many (tens to hundreds) participants. Clever techniques like Simulcast[0] and SVC[1] allow that, but routing server must support it to meet different requirements of individual participants. 0. https://webrtchacks.com/sfu-simulcast/ 1. https://webrtchacks.com/chrome-vp9-svc/

Huh, I happen to be knee-deep in this stuff right now. This article noted that Slack seemed to be running an old TURN server (pre-coturn): https://webrtchacks.com/slack-webrtc-slacking/ Given that the latest coturn has this vulnerability mitigated by default, perhaps all this boils down to is "Slack runs outdated software, we exploited it."?

The point is not the crypto, but what WebRTC supports. As of now it does not. See https://webrtchacks.com/you-dont-have-end-to-end-encryption-e2ee/ for reference

You are wrong. This is the current state of webrtc. Without considerable hacks (patent-encumbered unaccelerated wasm ffmpeg encoding of pixel data, rolling your own SRTP-like encrypted stream over datachannels and full mesh distribution of keys) this is not currently possible for anything other than a full mesh. Whenever an SFU (effectively a MITM) is involved (needed for more than a small number of participants) e2e encryption is lost. If/when Insertable Streams are commonplace, this will be possible without so many hacks. Edit: I see I was downvoted for correcting you, but there was an article on this very subject yesterday: https://webrtchacks.com/you-dont-have-end-to-end-encryption-e2ee/

This is more of a low level API. Not some ready to use thing like Jitsi. That is already a big difference with alternatives. Performance seems good: https://webrtchacks.com/sfu-load-testing/ And it is very actively developed, which is arguably the most important metric in the current webRTC environment. Since the spec and browser implementations are a fast moving target right now and for the years to come.

Not sure if this answers your specific questions - but I found an article written by a dev from Jitsi, describing the architecture of Jitsi Meet: Improving Scale and Media Quality with Cascading SFUs https://webrtchacks.com/sfu-cascading/ (November 2018) Edit: Based on another Jitsi dev's comment in this thread: SFU stands for "Selective Forwarding Unit", which was extracted to a standalone project, Jisti Videobridge.

It's described at: https://webrtchacks.com/zoom-avoids-using-webrtc/ They use WebRTC data channels, but nothing else. I'd speculate that this is the reason why they try to push people into using the desktop client.

> This is in part because they were obviously too good for WebRTC native audio and instead gutted ffmpeg and compiled it to WebAssembly (I wish I was kidding but I'm not: https://webrtchacks.com/zoom-avoids-using-webrtc/). Not a Zoom apologist I—I am also deeply creeped out by the fetish for covert data exfiltration in a platform that is so widely used in these quarantine days—but, as far as the tech goes, the story you linked seems to say that they do use WebRTC as of September 2019.

I know Zoom does their codec in wasm [1] and so on, but I don't know much about how other different webrtc impls work. Is there any reason jitsi or any other project would be faster / have better perf than peerjs.com? [1] https://webrtchacks.com/zoom-avoids-using-webrtc/

Why is the web client (on Chrome, etc) so bad on Zoom ? I mean, people are building Google Earth on the browser .Its not just the bad video experience - even the product experience is seriously broken. For example, the default audio setting when you sign in to the web video client is to connect using PHONE AUDIO. In case you figure out how to click the tab to use computer audio...it breaks down a couple of time in asking for browser permissions (camera, mic). It is unusually bad for something that is supposed to be that good. there are all these articles about the comparisons - https://webrtchacks.com/zoom-avoids-using-webrtc/ https://bloggeek.me/webrtc-vs-zoom-video-quality/ hangouts still rules when it comes to web based video conferencing. And for countries with massive linux based usage (like India), Zoom is not a very viable option.

You can also consider tunneling over WebSockets. It's a lot easier than WebRTC especially you don't need the handshaking nonsense which often require self hosting STUN and TURN servers if you don't want to rely on third parties. IIRC the performance of WebSockets is good enough for companies like Xoom. https://webrtchacks.com/zoom-avoids-using-webrtc/ Some VNC services like noVNC and Xpra also uses WebSockets.