, Alexa, and Siri are the dominating voice assistants out there for on a regular basis use. These assistants have grow to be ubiquitous in virtually each dwelling, finishing up duties from dwelling automation, word taking, recipe steering and answering easy questions. In the case of answering questions although, within the age of LLMs, getting a concise and context-based reply from these voice assistants might be difficult, if not non-existent. For instance, if you happen to ask Google Assistant how the market is reacting to Jerome Powell’s speech in Jackson Gap on Aug 22, it can merely reply that it doesn’t know the reply and provides just a few hyperlinks you could peruse. That’s when you have the screen-based Google Assistant.
Typically you simply need a fast reply on present occasions, otherwise you wish to know if an Apple tree would survive the winter in Ohio, and sometimes voice assistants like Google and Siri fall wanting offering a satisfying reply. This acquired me enthusiastic about constructing my very own voice assistant, one that will give me a easy, single sentence reply primarily based on its search of the net.
Of the varied LLM powered search engines like google out there, I’ve been an avid person of Perplexity for greater than a 12 months now and I exploit it solely for all my searches besides for easy ones the place I nonetheless return to Google or Bing. Perplexity, along with its reside net index, which allows it to offer up-to-date, correct, sourced solutions, permits customers entry to its performance via a robust API. Utilizing this performance and integrating it with a easy Raspberry Pi, I supposed to create a voice assistant that will:
- Reply to a wake phrase and be able to reply my query
- Reply my query in a easy, concise sentence
- Return to passive listening with out promoting my knowledge or giving my pointless adverts
The {Hardware} for the Assistant
To construct our voice assistant, just a few key {hardware} parts are required. The core of the challenge is a Raspberry Pi 5, which serves because the central processor for our utility. For the assistant’s audio enter, I selected a easy USB gooseneck microphone. Any such microphone is omnidirectional, making it efficient at listening to the wake phrase from totally different elements of a room, and its plug-and-play nature simplifies the setup. For the assistant’s output, a compact USB-powered speaker offers the audio output. A key benefit of this speaker is that it makes use of a single USB cable for each its energy and audio sign, which minimizes cable litter.
This method of utilizing available USB peripherals makes the {hardware} meeting simple, permitting us to focus our efforts on the software program.
Getting the setting prepared
In an effort to question Perplexity utilizing customized queries and as a way to have a wake phrase for the voice assistant, we have to generate a few API keys. In an effort to generate a Perplexity API key one can join a Perplexity account, go to the Settings menu, choose the API tab, and click on “Generate API Key” to create and replica their private key to be used in functions. Entry to API key technology often requires a paid plan or fee methodology, so make sure the account is eligible earlier than continuing.
Platforms that provide wake phrase customization embody PicoVoice Porcupine, Sensory TrulyHandsfree, and Snowboy, with PicoVoice Porcupine offering a straightforward on-line console for producing, testing, and deploying customized wake phrases throughout desktop, cell, and embedded gadgets. A brand new person can generate a customized phrase for PicoVoice Porcupine by signing up for a free Picovoice Console account, navigating to the Porcupine web page, deciding on the specified language, typing within the customized wake phrase, and clicking “Prepare” to supply and obtain the platform-specific mannequin file (.ppn) to be used. Make sure that to check the wake phrase for efficiency earlier than finalizing, as this ensures dependable detection and minimal false positives. The wake phrase I’ve skilled and can use is “Hey Krishna”.
Coding the Assistant
The whole Python script for this challenge is on the market on my GitHub repository. On this part, let’s take a look at the important thing parts of the code to know how the assistant features.
The script is organized into just a few core features that deal with the assistant’s senses and intelligence, all managed by a central loop.
Configuration and Initialization
The primary a part of the script is devoted to setup. It handles loading the required API keys, mannequin recordsdata, and initializing the shoppers for the companies we’ll use.
# --- 1. Configuration ---
load_dotenv()
PICOVOICE_ACCESS_KEY = os.environ.get("PICOVOICE_ACCESS_KEY")
PERPLEXITY_API_KEY = os.environ.get("PERPLEXITY_API_KEY")
KEYWORD_PATHS = ["Krishna_raspberry-pi.ppn"] # My wake phrase pat
MODEL_NAME = "sonar"This part makes use of the dotenv library to securely load your secret API keys from a .env file, which is a greatest observe that retains them out of your supply code. It additionally defines key variables like the trail to your customized wake phrase file and the precise Perplexity mannequin we wish to question.
Wake Phrase Detection
For the assistant to be really hands-free, it must pay attention repeatedly for a selected wake phrase with out utilizing important system assets. That is dealt with by the whereas True: loop within the predominant perform, which makes use of the PicoVoice Porcupine engine.
# That is the primary loop that runs repeatedly
whereas True:
# Learn a small chunk of uncooked audio knowledge from the microphone
pcm = audio_stream.learn(porcupine.frame_length)
pcm = struct.unpack_from("h" * porcupine.frame_length, pcm)
# Feed the audio chunk into the Porcupine engine for evaluation
keyword_index = porcupine.course of(pcm)
if keyword_index >= 0:
# Wake phrase was detected, proceed to deal with the command...
print("Wake phrase detected!")This loop is the guts of the assistant’s “passive listening” state. It repeatedly reads small, uncooked audio frames from the microphone stream. Every body is then handed to the porcupine.course of() perform. This can be a extremely environment friendly, offline course of that analyzes the audio for the precise acoustic sample of your customized wake phrase (“Krishna”). If the sample is detected, porcupine.course of() returns a non-negative quantity, and the script proceeds to the lively part of listening for a full command.
Speech-to-Textual content — Changing person inquiries to textual content
After the wake phrase is detected, the assistant must pay attention for and perceive the person’s query. That is dealt with by the Speech-to-Textual content (STT) part.
# --- This logic is inside the primary 'if keyword_index >= 0:' block ---
print("Listening for command...")
frames = []
# File audio from the stream for a set length (~10 seconds)
for _ in vary(0, int(porcupine.sample_rate / porcupine.frame_length * 10)):
frames.append(audio_stream.learn(porcupine.frame_length))
# Convert the uncooked audio frames into an object the library can use
audio_data = sr.AudioData(b"".be a part of(frames), porcupine.sample_rate, 2)
strive:
# Ship the audio knowledge to Google's service for transcription
command = recognizer.recognize_google(audio_data)
print(f"You (command): {command}")
besides sr.UnknownValueError:
speak_text("Sorry, I did not catch that.")
As soon as the wake phrase is detected, the code actively data audio from the microphone for roughly 10 seconds, capturing the person’s spoken command. It then packages this uncooked audio knowledge and sends it to Google’s speech recognition service utilizing the speech_recognition library. The service processes the audio and returns the transcribed textual content, which is then saved within the command variable.
Getting Solutions from Perplexity
As soon as the person’s command has been transformed to textual content, it’s despatched to the Perplexity API to get an clever, up-to-date reply.
# --- This logic runs if a command was efficiently transcribed ---
if command:
# Outline the directions and context for the AI
messages = [{"role": "system", "content": "You are an AI assistant. You are located in Twinsburg, Ohio. All answers must be relevant to Cleveland, Ohio unless asked for differently by the user. You MUST answer all questions in a single and VERY concise sentence."}]
messages.append({"function": "person", "content material": command})
# Ship the request to the Perplexity API
response = perplexity_client.chat.completions.create(
mannequin=MODEL_NAME,
messages=messages
)
assistant_response_text = response.selections[0].message.content material.strip()
speak_text(assistant_response_text)
This code block is the “mind” of the operation. It first constructs a messages record, which features a essential system immediate. This immediate offers the AI its persona and guidelines, equivalent to answering in a single sentence and being conscious of its location in Ohio. The person’s command is then added to this record, and all the package deal is distributed to the Perplexity API. The script then extracts the textual content from the AI’s response and passes it to the speak_text perform to be learn aloud.
Textual content-to-Speech — Changing Perplexity response to Voice
The speak_text perform is what offers the assistant its voice.
def speak_text(text_to_speak, lang='en'):
# Outline a perform that converts textual content to speech, default language is English
print(f"Assistant (talking): {text_to_speak}")
# Print the textual content for reference so the person can see what's being spoken
strive:
pygame.mixer.init()
# Initialize the Pygame mixer module for audio playback
tts = gTTS(textual content=text_to_speak, lang=lang, sluggish=False)
# Create a Google Textual content-to-Speech (gTTS) object with the offered textual content and language
# 'sluggish=False' makes the speech sound extra pure (not slow-paced)
mp3_filename = "response_audio.mp3"
# Set the filename the place the generated speech might be saved
tts.save(mp3_filename)
# Save the generated speech as an MP3 file
pygame.mixer.music.load(mp3_filename)
# Load the MP3 file into Pygame's music participant for playback
pygame.mixer.music.play()
# Begin enjoying the speech audio
whereas pygame.mixer.music.get_busy():
pygame.time.Clock().tick(10)
# Preserve this system working (by checking if playback is ongoing)
# This prevents the script from ending earlier than the speech finishes
# The clock.tick(10) ensures it checks 10 instances per second
pygame.mixer.stop()
# Give up the Pygame mixer as soon as playback is full to free assets
os.take away(mp3_filename)
# Delete the momentary MP3 file after playback to scrub up
besides Exception as e:
print(f"Error in Textual content-to-Speech: {e}")
# Catch and show any errors that happen through the speech technology or playbackThis perform takes a textual content string, prints it for reference, then makes use of the gTTS (Google Textual content-to-Speech) library to generate a brief MP3 audio file. It performs the file via the system’s audio system utilizing the pygame library, waits till playback is completed, after which deletes the file. Error dealing with is included to catch points through the course of.
Testing the assistant
Beneath is an indication of the functioning of the customized voice assistant. To check its efficiency with Google Assistant, I’ve requested the identical query from Google in addition to from the customized assistant.
As you may see, Google offers hyperlinks to the reply relatively than offering a short abstract of what the person desires. The customized assistant goes additional and offers a abstract and is extra useful and informational.
Conclusion
On this article, we seemed on the technique of constructing a totally purposeful, hands-free voice assistant on a Raspberry Pi. By combining the ability of a customized wake phrase and the Perplexity API through the use of Python, we created a easy voice assistant machine that helps in getting data rapidly.
The important thing benefit of this LLM-based method is its capability to ship direct, synthesized solutions to complicated and present questions — a job the place assistants like Google Assistant usually fall quick by merely offering an inventory of search hyperlinks. As an alternative of performing as a mere voice interface for a search engine, our assistant features as a real reply engine, parsing real-time net outcomes to offer a single, concise response. The way forward for voice assistants lies on this deeper, extra clever integration, and constructing your individual is one of the simplest ways to discover it.
