AssemblyAI Blog

Being able to handle and ask questions of audio data is a pretty big field. https://www.assemblyai.com/, for example, is a company entirely dedicated to audio intelligence. They have some great example use cases on their page.

It might be worth checking out https://www.assemblyai.com they let you build a custom audio model. One challenge with the audio from these radios is that it goes through some heavy compression. Traditional models will have a lot of challenges. Give a system that uses analog audio a try. The quality of the audio is a lot better.

Great question! There are three main parts to note: designing the videos, the backend for handling media, and the download renderer. The video designing process is a React app that leverages a Ruby on Rails based backend API. The React app handles the views, and if you look at the UI, you will see how the app steps are persisted in the URL. Our React app is based on Redux Toolkit, which is phenomenal. The Rails application is a normal API with Sidekiq (aka Redis) workers, which handle asynchronous tasks. Our UI renders a number of interesting elements which are respectively generated/prepared in various service oriented streams. The most important is our transcript API, which is from https://www.AssemblyAI.com - they are the best, cheapest, highest quality and best development experience transcription tool. We also have a series of Lambda functions that handle uploaded audio/video file prep, such that we can encode files in a unified format and parse out the audio data needed for visualizing things like the audio waveform or the animated audio frequency data. A few interesting tidbits are that we use Lambda Layers extensively. We have functions written Ruby and JavaScript where we move the vendor Gems or node_modules into a shared Lambda Layer, and then we also use EFS to run Python based functions that have dependencies that are too big for the Lambda itself. Our video rendering is also pretty neat, in that we leverage the browser as a rendering view, and batch process…

AssemblyAI is quite good, pretty cheap and easy with robust SDK support. https://www.assemblyai.com/

I don't know if this is the best resource for learning AI/ML. I'd sooner recommend Fast.AI, Caltech's open courses, etc.[1] Also, my understanding is that TensorFlow is becoming less and less popular, so you might want to focus on PyTorch. Curious if others agree? [1] See: https://phaseai.com/resources/free-resources-ai-ml-2024 [2] For example, scroll down to see the chart here: https://www.assemblyai.com/blog/pytorch-vs-tensorflow-in-2023/

FWIW, AssemblyAI has great trasncript quality in my experience, and they support streaming: https://www.assemblyai.com/docs/walkthroughs#realtime-streaming-transc…

AssemblyAI (YC S17) | Senior Backend Engineer | Full-time | Remote We're building the Top Rated Speech-to-Text API for Developers. Customers use our API to transcribe phone calls, meetings, videos, podcasts, and other types of media. Our accurate transcripts are used to power features like visual voicemail, call analytics, closed captioning, meeting summaries, and a slew of other features. We're growing fast, and processing millions of API calls per day in production. We're a fully remote company and our development team is currently spread out across the US. We're looking for a senior backend engineer with strong software and cloud engineering skills. You'll help us build highly scalable developer-centric APIs powered by state-of-the-art ML models. Job description: https://www.notion.so/Senior-Backend-Engineer-187f72231e4946f8a5b37e22… Website: https://www.assemblyai.com You can reach me directly at: [email protected] (Please put "HN" in your emails)

I imagine you could fix this by running a speaker diarization classifier periodically? https://www.assemblyai.com/blog/what-is-speaker-diarization-and-how-do…

AssemblyAI (YC S17) is currently the one that stands out in the WER and accuracy benchmarks (https://www.assemblyai.com/benchmarks). Though its models are accessed through a web API rather than locally hosted, and speaker diarization is enabled through a parameter in the API call (https://www.assemblyai.com/docs/speech-to-text/pre-recorded-audio/spea…

A comprehensive discussion of the topic: https://www.assemblyai.com/blog/emergent-abilities-of-large-language-m…

The thing I find most interesting about LLMs is the emergent skills that happen almost instantly(based on model size) that weren't inherently trained for. https://www.assemblyai.com/blog/emergent-abilities-of-large-language-m…

Its the seamless integration that counts. With the playground for example you can have it read and summarize a near 3 hour video https://www.youtube.com/watch?v=Se91Pn3xxSs with a few clicks and decide if you really want to watch https://www.assemblyai.com/playground/v2/transcript/6lu93wlww5-666c-49…

They have their own ASR Conformer-2[0] and support 9 languages (they count it as 12)[1] It looks like their synchronous transcribe is much slower than whisper, but if you need it fast, you need their realtime ASR (or amazon or google's). [0] Conformer-2 is trained on 1.1M hours of English https://www.assemblyai.com/blog/conformer-2/ [1] https://www.assemblyai.com/docs/Concepts/supported_languages

Not really, just look at https://sites.research.google/usm/ Or this: https://www.assemblyai.com/blog/conformer-1/

I'm going to just assume you're not being malicious. https://www.microsoft.com/en-us/research/publication/sparks-of-artific… https://www.assemblyai.com/blog/emergent-abilities-of-large-language-m… If you'd like you can define "emergent capabilities in an AGI sense".

What I find fascinating about RFDiffusion is that it puts together two very powerful yet distinct deep learning architectures: Diffusion models and Graph Neural Networks. I wrote about this here: https://www.assemblyai.com/blog/ai-trends-graph-neural-networks/

80x faster than Whisper is an incredible feat. How is Deepgram's transcription accuracy? Also, have you heard of Conformer-1 by Assembly-AI[1]? It released a few days ago and supposedly scored higher than Whisper on various benchmarks. [1]: https://www.assemblyai.com/blog/conformer-1/

poring over the docs for https://www.assemblyai.com/docs/core-transcription#profanity-filtering rn...

You can. You're just paying for compute and having it managed. Here's price estimates for 1,000 hours of audio on GCP: https://www.assemblyai.com/blog/how-to-run-openais-whisper-speech-reco… For reference, from OpenAI it would be $360 and it's the large-v2 model.

Your comment is actually pretty funny given that the thing that makes ChatGPT so much more successful than GPT-3 is the application of reinforcement learning. [0] [0] https://www.assemblyai.com/blog/how-chatgpt-actually-works/

In hugging face, almost 85% of models are exclusive to PyTorch, and even those that are not exclusive have about a 50% chance of being available in PyTorch as well. In contrast, only about 16% of all models are available for TensorFlow, with only about 8% being TensorFlow-exclusive. (source: https://www.assemblyai.com/blog/pytorch-vs-tensorflow-in-2022/)

Things are indeed moving really fast now, and that's really exciting, but I'd be cautious about extrapolating that pace too far into the future. Nearly all of the recent image generating AIs are all based on a single breakthrough technique called image diffusion[1]. Once we start hitting the limits of diffusion models progress will likely start to plateau a bit, at least until the next major breakthrough. [1]: https://www.assemblyai.com/blog/diffusion-models-for-machine-learning-…

https://alphacephei.com/vosk/install#usage-examples demonstrates the bare-bones vosk-transcriber sample, and there's also https://www.assemblyai.com/blog/getting-started-with-espnet I wasn't able to play with https://github.com/o-oconnell/mp4grep on ARM.

Here's a good explanation: https://www.assemblyai.com/blog/diffusion-models-for-machine-learning-…

They're currently hosted on Google in a way that you can download them via curl/wget. Here's a guide including the link: https://www.assemblyai.com/blog/how-to-run-stable-diffusion-locally-to…

Cool write-up. I'm currently interested in learning about Diffusion Models and found your other article [0] to be a nice introduction to the topic. [0] https://www.assemblyai.com/blog/diffusion-models-for-machine-learning-…

Yeah actually, good call. The position of the token matters, since these things use transformers to encode the embeddings. https://www.assemblyai.com/blog/how-imagen-actually-works/

The important part seems to be the diffusion model. Explanation linked from same page: https://www.assemblyai.com/blog/diffusion-models-for-machine-learning-…

These are good explanations: - https://www.assemblyai.com/blog/how-dall-e-2-actually-works/ - https://www.youtube.com/watch?v=F1X4fHzF4mQ

And in terms of relative popularity of TensorFlow and PyTorch: https://www.assemblyai.com/blog/pytorch-vs-tensorflow-in-2022/