Jean-Louis Quéguiner is the Founder and CEO of Gladia. He beforehand served as Group Vice President of Knowledge, AI, and Quantum Computing at OVHcloud, one in every of Europe’s main cloud suppliers. He holds a Grasp’s Diploma in Symbolic AI from the College of Québec in Canada and Arts et Métiers ParisTech in Paris. Over the course of his profession, he has held vital positions throughout varied industries, together with monetary information analytics, machine studying functions for real-time digital promoting, and the event of speech AI APIs.
Gladia offers superior audio transcription and real-time AI options for seamless integration into merchandise throughout industries, languages, and know-how stacks. By optimizing state-of-the-art ASR and generative AI fashions, it ensures correct, lag-free speech and language processing. Gladia’s platform additionally permits real-time extraction of insights and metadata from calls and conferences, supporting key enterprise use circumstances comparable to gross sales help and automatic buyer help.
What impressed you to sort out the challenges in speech-to-text (STT) know-how, and what gaps did you see available in the market?
Once I based Gladia, the preliminary purpose was broad—an AI firm that will make advanced know-how accessible. However as we delved deeper, it turned clear that voice know-how was probably the most damaged and but most important space to give attention to.
Voice is central to our day by day lives, and most of our communication occurs by way of speech. But, the instruments obtainable for builders to work with voice information had been insufficient by way of velocity, accuracy, and value—particularly throughout languages.
I wished to repair that, to unpack the complexity of voice know-how and repackage it into one thing easy, environment friendly, highly effective and accessible. Builders shouldn’t have to fret concerning the intricacies of AI fashions or the nuances of context size in speech recognition. My purpose was to create an enterprise-grade speech-to-text API that labored seamlessly, whatever the underlying mannequin or know-how—a real plug-and-play resolution.
What are a few of the distinctive challenges you encountered whereas constructing a transcription resolution for enterprise use?
In the case of speech recognition, velocity and accuracy—the 2 key efficiency indicators on this discipline—are inversely proportional by design. Because of this enhancing one will compromise the opposite, at the very least to some extent. The fee issue, to an enormous extent, outcomes from the supplier’s selection between velocity and high quality.
When constructing Gladia, our purpose was to seek out the right steadiness between these two elements, all whereas guaranteeing the know-how stays obtainable to startups and SMEs. Within the course of we additionally realized that the foundational ASR fashions like OpenAI’s Whisper, which we labored with extensively, are biased, skewering closely in direction of English as a result of their coaching information, which leaves a whole lot of languages under-represented.
So, along with fixing the speed-accuracy tradeoff, it was vital to us— as a European, multilingual workforce—to optimize and fine-tune our core fashions to construct a very world API that helps companies function throughout languages.
How does Gladia differentiate itself within the crowded AI transcription market? What makes your Whisper-Zero ASR distinctive?
Our new real-time engine (Gladia Actual Time) achieves an industry-leading 300 ms latency. Along with that, it’s in a position to extract insights from a name or assembly with the so-called “audio intelligence” add-ons or options, like named entity recognition (NER) or sentiment evaluation.
To our data, only a few opponents are in a position to present each transcription and insights at such excessive latency (lower than 1s end-to-end) – and do all of that precisely in languages apart from English. Our languages help extends to over 100 languages immediately.
We additionally put a particular emphasis on making the product actually stack agnostic. Our API is appropriate with all present tech stacks and telephony protocols, together with SIP, VoIP, FreeSwitch and Asterisk. Telephony protocols are particularly advanced to combine with, so we consider this product facet can carry great worth to the market.
Hallucinations in AI fashions are a major concern, particularly in real-time transcription. Are you able to clarify what hallucinations are within the context of STT and the way Gladia addresses this downside?
Hallucination normally happens when the mannequin lacks data or doesn’t have sufficient context on the subject. Though fashions can produce outputs tailor-made to a request, they will solely reference data that existed on the time of their coaching, and that will not be up-to-date. The mannequin will create coherent responses by filling in gaps with data that sounds believable however is inaccurate.
Whereas hallucinations turned identified within the context of LLMs first, they happen with speech recognition fashions— like Whisper ASR, a number one mannequin within the discipline developed by OpenAI – as effectively. Whisper’s hallucinations are like these of LLMs as a result of an identical structure, so it’s an issue that considerations generative fashions, which can be in a position to predict the phrases that comply with based mostly on the general context. In a manner, they ‘invent’ the output. This method may be contrasted with extra conventional, acoustic-based ASR architectures that match the enter sound to output in a extra mechanical manner
In consequence, chances are you’ll discover phrases in a transcript that weren’t really mentioned, which is clearly problematic, particularly in fields like drugs, the place a mistake of this type can have grave penalties.
There are a number of strategies to handle and detect hallucinations. One widespread method is to make use of a retrieval-augmented technology (RAG) system, which mixes the mannequin’s generative capabilities with a retrieval mechanism to cross-check info. One other technique includes using a “chain of thought” method, the place the mannequin is guided by way of a collection of predefined steps or checkpoints to make sure that it stays on a logical path.
One other technique for detecting hallucinations includes utilizing programs that assess the truthfulness of the mannequin’s output throughout coaching. There are benchmarks particularly designed to guage hallucinations, which contain evaluating totally different candidate responses generated by the mannequin and figuring out which one is most correct.
We at Gladia have experimented with a mix of strategies when constructing Whisper-Zero, our proprietary ASR that removes just about all hallucinations. It’s confirmed wonderful ends in asynchronous transcription, and we’re at the moment optimizing it for real-time to realize the identical 99.9% data constancy.
STT know-how should deal with a variety of complexities like accents, noise, and multi-language conversations. How does Gladia method these challenges to make sure excessive accuracy?
Language detection in ASR is an especially advanced process. Every speaker has a novel vocal signature, which we name options. By analyzing the vocal spectrum, machine studying algorithms can carry out classifications, utilizing the Mel Frequency Cepstral Coefficients (MFCC) to extract the principle frequency traits.
MFCC is a technique impressed by human auditory notion. It’s a part of the “psychoacoustic” discipline, specializing in how we understand sound. It emphasizes decrease frequencies and makes use of strategies like normalized Fourier decomposition to transform audio right into a frequency spectrum.
Nevertheless, this method has a limitation: it is based mostly purely on acoustics. So, in case you communicate English with a powerful accent, the system might not perceive the content material however as a substitute choose based mostly in your prosody (rhythm, stress, intonation).
That is the place Gladia’s revolutionary resolution is available in. We have developed a hybrid method that mixes psycho-acoustic options with content material understanding for dynamic language detection.
Our system does not simply take heed to the way you communicate, but in addition understands what you are saying. This twin method permits for environment friendly code-switching and does not let sturdy accents get misrepresented/misunderstood.
Code-switching—which is amongst our key differentiators—is a very vital function in dealing with multilingual conversations. Audio system might change between languages mid-conversation (and even mid-sentence), and the flexibility of the mannequin to transcribe precisely on the fly regardless of the change is essential.
Gladia API is exclusive in its means to deal with code-switching with this many language pairs with a excessive degree of accuracy and performs effectively even in noisy environments, identified to scale back the standard of transcription.
Actual-time transcription requires ultra-low latency. How does your API obtain lower than 300 milliseconds latency whereas sustaining accuracy?
Maintaining latency beneath 300 milliseconds whereas sustaining excessive accuracy requires a multifaceted method that blends {hardware} experience, algorithm optimization, and architectural design.
Actual-time AI isn’t like conventional computing—it’s tightly linked to the facility and effectivity of GPGPUs. I’ve been working on this area for almost a decade, main the AI division at OVHCloud (the largest cloud supplier within the EU), and discovered firsthand that it’s at all times about discovering the best steadiness: how a lot {hardware} energy you want, how a lot it prices, and the way you tailor the algorithms to work seamlessly with that {hardware}.
Efficiency in actual time AI comes from successfully aligning our algorithms with the capabilities of the {hardware}, guaranteeing each operation maximizes throughput whereas minimizing delays.
Nevertheless it’s not simply the AI and {hardware}. The system’s structure performs an enormous position too, particularly the community, which may actually affect latency. Our CTO, who has deep experience in low-latency community design from his time at Sigfox (an IoT pioneer), has optimized our community setup to shave off beneficial milliseconds.
So, it’s actually a mixture of all these elements—good {hardware} selections, optimized algorithms, and community design—that lets us persistently obtain sub-300ms latency with out compromising on accuracy.
Gladia goes past transcription with options like speaker diarization, sentiment evaluation, and time-stamped transcripts. What are some revolutionary functions you’ve seen your purchasers develop utilizing these instruments?
ASR unlocks a variety of functions to platforms throughout verticals, and it’s been wonderful to see what number of actually pioneering firms have emerged within the final two years, leveraging LLMs and our API to construct cutting-edge, aggressive merchandise. Listed below are some examples:
- Sensible note-taking: Many consumers are constructing instruments for professionals who have to rapidly seize and manage data from work conferences, pupil lectures, or medical consultations. With speaker diarization, our API can determine who mentioned what, making it straightforward to comply with conversations and assign motion objects. Mixed with time-stamped transcripts, customers can bounce straight to particular moments in a recording, saving time and guaranteeing nothing will get misplaced in translation.
- Gross sales enablement: Within the gross sales world, understanding buyer sentiment is all the pieces. Groups are utilizing our sentiment evaluation function to achieve real-time insights into how prospects reply throughout calls or demos. Plus, time-stamped transcripts assist groups revisit key components of a dialog to refine their pitch or handle shopper considerations extra successfully. For this use case specifically, NER can be key to figuring out names, firm particulars, and different data that may be extracted from gross sales calls to feed the CRM mechanically.
- Name heart help: Corporations within the contract heart area are utilizing our API to offer dwell help to brokers, in addition to flagging buyer sentiment throughout calls. Speaker diarization ensures that issues being mentioned are assigned to the best particular person, whereas time-stamped transcripts allow supervisors to evaluation essential moments or compliance points rapidly. This not solely improves the client expertise – with higher on-call decision fee and high quality monitoring – but in addition boosts agent productiveness and satisfaction.
Are you able to talk about the position of customized vocabularies and entity recognition in enhancing transcription reliability for enterprise customers?
Many industries depend on specialised terminology, model names, and distinctive language nuances. Customized vocabulary integration permits the STT resolution to adapt to those particular wants, which is essential for capturing contextual nuances and delivering output that precisely displays your online business wants. For example, it means that you can create a listing of domain-specific phrases, comparable to model names, in a selected language.
Why it’s helpful: Adapting the transcription to the particular vertical means that you can decrease errors in transcripts, attaining a greater consumer expertise. This function is particularly essential in fields like drugs or finance.
Named entity recognition (NER) extracts and identifies key data from unstructured audio information, comparable to names of individuals, organizations, areas, and extra. A standard problem with unstructured information is that this essential data isn’t readily accessible—it is buried inside the transcript.
To resolve this, Gladia developed a structured Key Knowledge Extraction (KDE) method. By leveraging the generative capabilities of its Whisper-based structure—just like LLMs—Gladia’s KDE captures context to determine and extract related data straight.
This course of may be additional enhanced with options like customized vocabulary and NER, permitting companies to populate CRMs with key information rapidly and effectively.
In your opinion, how is real-time transcription reworking industries comparable to buyer help, gross sales, and content material creation?
Actual-time transcription is reshaping these industries in profound methods, driving unbelievable productiveness positive aspects, coupled with tangible enterprise advantages.
First, real-time transcription is a game-changer for help groups. Actual-time help is essential to enhancing the decision fee due to quicker responses, smarter brokers, and higher outcomes (by way of NSF, deal with instances, and so forth). As ASR programs get higher and higher at dealing with non-English languages and performing real-time translation, contact facilities can obtain a very world CX at decrease margins.
In gross sales, velocity and spot-on insights are all the pieces. Equally to what occurs with name brokers, real-time transcription is what equips them with the best insights on the proper time, enabling them to give attention to what issues probably the most in closing offers.
For creators, real-time transcription is probably much less related immediately, however nonetheless stuffed with potential, particularly in the case of dwell captioning and translation throughout media occasions. Most of our present media prospects nonetheless favor asynchronous transcription, as velocity is much less essential there, whereas accuracy is essential for functions like time-stamped video enhancing and subtitle technology.
Actual-time AI transcription appears to be a rising pattern. The place do you see this know-how heading within the subsequent 5-10 years?
I really feel like this phenomenon, which we now name real-time AI, goes to be in every single place. Basically, what we actually discuss with right here is the seamless means of machines to work together with individuals, the best way we people already work together with each other.
And in case you have a look at any Hollywood film (like Her) set sooner or later, you’ll by no means see anybody there interacting with clever programs by way of a keyboard. For me, that serves as the last word proof that within the collective creativeness of humanity, voice will at all times be the first manner we work together with the world round us.
Voice, as the principle vector to combination and share human data, has been a part of human tradition and historical past for for much longer than writing. Then, writing took over as a result of it enabled us to protect our data extra successfully than counting on the group elders to be the guardians of our tales and knowledge.
GenAI programs, able to understanding speech, producing responses, and storing our interactions, introduced one thing utterly new to the area. It’s the very best of each phrases and the very best of humanity actually. It offers us this distinctive energy and vitality of voice communication with the good thing about reminiscence, which beforehand solely written media may safe for us. For this reason I consider it’s going to be in every single place – it is our final collective dream.
Thanks for the good interview, readers who want to be taught extra ought to go to Gladia.
