Superpowered Blog

Is Superpowered really a wise name choice? If you look at https://superpowered.com/ , you'll see that it's in use by another startup. They're doing something completely different (mobile sound SDKs), but still, it's not like it's a personal blog or niche webshop etc. that you may hope buying at a reasonable price one day.

Android is not exactly known for its buttery smooth interface... https://www.theandroidsoul.com/how-to-fix-galaxy-s9-lag-and-stutter-is… https://superpowered.com/android-audio-latency-problem-just-got-worse Not to mention all of the security vulnerabilities that don’t get patched. The advantage of updates is that it encourages app developers to take advantage of the latest features. For instance, the author of Overcast, my favorite podcast app could release a new version that required iOS 12 a month after it came out without having to lose any customers - anyone who could run iOS 11 could run iOS 12. As far as updates, just the integration of shortcuts and Workflow has done wonders in iOS 12.

Because there is none, this is for Apple devices. For android you have that SDK, though I didn't check the licensing terms: https://superpowered.com

And a cool follow-up article: http://superpowered.com/android-marshmallow-latency Last time I worked on Android Audio was around the 4.1-4.2 era, and it was absolutely brutal compared to iOS. Glad to see that it's improving finally!

You can watch this I/O session from a few years back: https://youtu.be/d3kfEeMZ65c It probably provides the best insight to the problem Google engineers were and probably is still facing in regards to high performance audio on Android. Even when the latency is brought down, there's still other audio related issues/glitches (like buffer underruns). Even now that Android is measurably better than it was previously, things like buffer underruns can still affect the overall usability. Increasing the buffer (to fix the underruns) render the device less suitable for such things as using it with external MIDI controller. You can see from Superpowered's audio latency measurements: http://superpowered.com/latency#cta that the buffer for iOS is half that of the best Android (the dual-core Nexus 9)- 64 instead of 128 for Android. I

Have you tried performing using an android app? I used a drum-pad app the other week and the latency makes it unplayable. There's a great explanation here as to why it's so bad: http://superpowered.com/androidaudiopathlatency

Android Web Audio latency is over 200-300ms on many devices and is deliberately introduced to kill WebGL browser games that could otherwise bypass the Play Store and its 30% tax. Google has plenty of engineers smart of enough to fix such a glaring latency issue that so many of us have been complaining about for so long but Play Store game profits are in the billions and Google isn't going to let a frictionless browser game experience kill that. The Superpowered Latency Database[1] has many measurements showing just how bad Android latency is. (A select few devices do OK.) Note that Apple latency measurements look pretty good, because Apple uses different strategies to kill WebGL browser games: (1) Make WebGL slow and lacking features; (2) Prevent browser apps from going fullscreen; (3) Don't even let browser games ask if they can use gyro/tilt control, just block this outright because "Privacy". After all, if Epic Games could just put Fortnite inside mobile Safari their case against Apple wouldn't be so desperate. There's a demo version of Minecraft for the browser that I've never seen work on mobile browsers[2]. [1] https://superpowered.com/latency [2] https://classic.minecraft.net/

Are Facebook and Instagram unusable on mobile Safari, which bypasses the App Store? (I haven't used FB/Insta for years, so I'm not asking a rhetorical question here: I really don't know and I'm genuinely curious.) I do know that viable mobile browser video games that bypass the App Stores are currently impossible to do on both mobile Safari and Android Chrome because I recently tried. Dark patterns are employed, including adding a typical 300ms audio latency to the Web Audio API on Android Chrome[1], and making WebGL perform terrible on iOS Safari, preventing WebGL from going full screen, preventing accelerometer/gyro input "for privacy reasons" etc. [1] Superpowered Android audio latency test database shows how awful Android web audio latency is: https://superpowered.com/latency

https://superpowered.com/latency has messy data, but it looks like typically single-digit latency.

The most common results on this page indicate that it varies with buffer size, but could be 9-11 ms for iPhones. https://superpowered.com/latency

> A $200 xiaomi phone is equivalent in specs to a $600 Samsung. Xiaomi phones have much higher audio latency than Samsung phones.[1] As a VoIP user, I would rather use an entry level Samsung phone (e.g., a $150 A02s) than a Xiaomi flagship. [1] https://superpowered.com/latency

Most Android devices aren't capable of truly low-latency audio (sub-10ms round-trip). This makes the total addressable market for pro audio apps very small, which means device manufacturers have little incentive to spend the engineering resources to provide low latency, which makes the total addressable market for pro audio apps very small. Unless Google start setting audio latency requirements, it's a lost cause. http://superpowered.com/latency

About 7-12 ms. Actual benchmarks by an audio SDK maker here: http://superpowered.com/latency

A brief skim of the article they link regarding Marshmallow [1] provides this quote: > As readers may recall, 10ms round-trip audio latency is the threshold that must be met by Android to be considered truly 'pro audio'. This wasn't possible before Marshmallow (per aforementioned article), and therefore pro-quality audio apps are still only truly possible on 10% of Android devices. So the market is opened up for 100 million devices. I'm not saying it's not a huge number of devices. But it's not as huge as the blog states. [1] http://superpowered.com/superpowered-android-media-server

This is a good explainer of the various latencies in the Android audio pipeline: http://superpowered.com/androidaudiopathlatency

Switching the DAC will only help with quality, not with latency, because the latency is a software issue. Samsung has implemented software workarounds to skip some parts of the audio processing path and reduce latency, but I'm not sure how low it gets. See http://superpowered.com/androidaudiopathlatency and https://github.com/felixpalmer/android-visualizer/issues/5 for more info on all this.

Direct link for the superpowered.com latency testing app: http://superpowered.com/latency

I completely agree with the general idea. Bloat is too high and has far reaching consequences. And it's actually even worse than that. 100ms is nowhere near enough to be sufficient for all applications. For audio applications you need 10ms [http://superpowered.com/androidaudiopathlatency] which incidentally is the reason why Android is unusable for any interactive audio applications. Same applies to Virtual Reality. You need sub 20ms latency for head tracking to eyes latency or you will feel sick (effectively less than 10ms from the cpu side. Display scanning is summed on top). The 100ms figure applies mostly to traditional UI workflow. Whenever you need a realtime interaction that's 10x too high.

GP is talking about Android's well-known problem of having horrendous audio latency, with no workable solution for app devs. http://superpowered.com/androidaudiopathlatency/#axzz3nj4FLEj1

Hi there -- latency ranking you ask? As Westley from The Princess Bride might reply, "As you wish" http://superpowered.com/latency :)

The title isn't clear but (sub-) 10ms is the target. The "problem" part is that Android devices generally don't come close, the lowest-latency Android device on the market today (barring Samsung's custom "Professional Audio SDK") has a 35ms roundtrip latency (ADC -> DAC), and many devices are way beyond 100ms (http://superpowered.com/latency/)

TL;DR the Linux layer (ALSA) and the Java layer (Audio Flinger) use widely compatible but high latency techniques, whereas Apple designed their API's and hardware such that these layers can be optimized to almost nothing. (From the article: http://superpowered.com/wp-content/uploads/2015/04/Android-Audio-Path-…

And here are some measurements for iOS as well (Superpowered Mobile Audio Latency Test App for Android and iOS.) The iOS devices come out at 6-18ms, Androids at 17-860ms. The faster Androids have Samsung's Professional Audio SDK. http://superpowered.com/latency/

Sadly, most audio APIs are not "half-decent" at zero-latency inter-app routing and minimal mixing/output latency (for a particularly bad example, see https://superpowered.com/androidaudiopathlatency). JACK on Linux supported this but few apps supported it, ALSA and PulseAudio loopback devices don't have it to my knowledge, and PipeWire is the first chance at bringing mainstream non-musician apps to this promised land (but only JACK and PipeWire apps, and Pulse and ALSA apps still have added latency). Additionally, conventional hardware input/output requires at least two periods of buffering for a stable duplex session (mic input and speaker playback simultaneously). On startup, you fill the output buffer with 2 periods of silence. Every period (when input buffer grows and output buffer shrinks by 1 period), you take 1 period of input audio, feed it through the audio graph (which can take up to 1 period otherwise your speakers run out of audio to play and stutter), then write 1 period of output audio to the speakers. Smaller periods (lower sample/frame count, lower duration in ms) reduce latency, but increase the risk of stutter if your system can't schedule and run all audio apps in 1 period of time.

Meanwhile, Android Chrome deliberately introduces a ~300 millisecond audio delay to make web games impossible and force game devs to use the Play Store so that Google gets its pound of flesh. The superpowered[1] database has many measurements showing how bad Android is. Those of you with Android devices can try this ancient web audio demo to see that the issue remains unfixed all these years later, except on certain select premium devices (certain Samsungs, etc.). https://webaudiodemos.appspot.com/TouchPad/index.html You should hear sound the instant a rectangle is touched/clicked and you will with a desktop browser but not with most Androids. Apple also employs dark patterns on iOS Safari to make web games impossible: 1. No full screen. ("But the user wouldn't know how to exit full-screen - unless they buy from the App Store! We also plan to make full screen PWAs impossible because of, like, Reasons and totally not because of our 30% App Store cut!") 2. Slow WebGL and no WebGL 2.0. ("But we need to run extra shader validation security checks every single frame even though we technically only need to do it once up front! But this extra Privacy and Security[TM] is totally unnecessary if you buy from the App Store!") 3. No device motion tilt control. ("Allowing the user to consent to tilt control would violate the user's Privacy and Security[TM] - unless they buy from the App Store!") 4. No JS debug console unless you register an Apple developer account, download 10+…

The description "real-time" varies in meaning depending on the application and acceptable average and maximum latency. The un-informed usually thinks "real-time" means "hard real-time", but that's seldom necessary, so one can save on additional expense and effort by using a regular linux distro and removing most of the daemons and file location indexing. I've done real-time development (for Space Shuttle, rocket and balloon projects), and largely all I care about is if a circular buffer can be emptied in time before it fills. That's one technique for avoiding latency variation issues. The versions of linux that you would normally encounter aim for music real-time, which is about 10 ms latency. 30 ms is considered to be bad. Most of the pro Yamaha synths use linux as the embedded OS, and some of the code is downloadable (they attempt to comply with the letter of the GPL.) So you can go down to Guitar Center and do a real-time test anytime yourself. :) The iPhone is pretty good for music, because it was designed to have low latency when playing. https://www.synthtopia.com/content/2018/02/17/10-years-later-android-a… https://superpowered.com/androidaudiopathlatency Looks like Wind River discontinued RTLinux, which was hard real-time (a real shame actually, as it removes one of the few hard real-time options): https://en.wikipedia.org/wiki/RTLinux

http://superpowered.com/latency Generally around 6-8ms.

There’s def web clients already out there. I’ve seen satchel, just haven’t come across an actively developed one. Jssip is a neat library but not a client: https://jssip.net/ Twilio also has a web client. Taking a cursory search of WebAssembly for voip shows some activity: Webrtc is a little old and using webassembly to introduce other codecs isn’t entirely new or novel: https://cloudtweaks.com/2020/08/deep-customization-webrtc/ Blazer allows using webassembly to make calls from twilio: https://www.twilio.com/blog/making-phone-calls-from-blazor-webassembly… Another neat use for webassembly and audio is audio production say for podcasts with multiple callers that makes editing a bit easier. https://superpowered.com/js-wasm-overview Noise cancellation and other dsp effects are something webassembly can meaningfully help with. https://news.ycombinator.com/item?id=30568164

Sure. Originally Android only had the Java media APIs, with a quite complex state. https://developer.android.com/reference/android/media/MediaPlayer.html Android 2.3 introduced OpenSL ES into the NDK and Android 4.0 got OpenMAX. All of them weren't up to the game of realtime audio due to the Android architecture. http://superpowered.com/androidaudiopathlatency https://nsr.cse.buffalo.edu/wp-content/uploads/2016/10/rtdroid-sound-j… https://www.androidauthority.com/android-6-0-marshmallow-audio-latency… There are plenty of other articles online when you search for audio issues on Android. There was even a Google IO 2013 talk about it. https://www.youtube.com/watch?v=d3kfEeMZ65c Meanwhile Samsung had customized their Android versions to properly support real time audio and offered an SDK for their handsets. http://developer.samsung.com/galaxy/professional-audio At Google IO 2016 the problem was again discussed, with the Samsung team from Professional Audio being called on stage to discuss their collaboration. https://www.youtube.com/watch?v=F2ZDp-eNrh4 Finally last year at Google IO 2017, the new real time audio was announced, as C only lib. https://www.youtube.com/watch?v=C0BPXZIvG-Q https://developer.android.com/ndk/guides/audio/index.html The announced C++ wrapper ended up being an external project on github, detached from the NDK, that you need to integrate yourself, without any guarantees how long Google will care to maintain it. Oh, to…