The Conversations You Can't Afford to Get Wrong: Building Parlayy with Gradium

Imagine you're a freelance developer. You've spent days building a website for a client, and now it's time to discuss the price.

The client looks at your proposal and says, "Honestly, I think this is too expensive. Someone else offered to do it for half that price."

You know the work is worth what you're charging. You've thought through the costs, the time, and the value you're delivering. But knowing that and confidently defending your price in the moment are two different things.

What do you say next?

You could ask a friend to play the client. You could rehearse your answers in front of a mirror. You could ask an AI chatbot for advice.

But none of those options quite recreates the experience of being challenged in a real conversation, having to think on your feet, and finding the right words under pressure.

That's the problem I'm exploring with Parlayy.

The problem isn't always knowing what to say

Some conversations carry more weight than others.

A job interview can change your career. A difficult conversation can affect a relationship. A negotiation can determine whether months of work translate into a sustainable business. An investor meeting can force a founder to defend assumptions they haven't fully examined.

People often prepare for these moments by reading advice, writing down talking points, or imagining how the other person might respond.

The problem is that real conversations don't follow scripts.

People challenge assumptions. They raise objections you didn't anticipate. They ask follow-up questions. Sometimes, the answer you prepared falls apart the moment somebody pushes back.

I wanted to explore a different approach: what if you could practise the conversation itself, face meaningful resistance, make mistakes without real-world consequences, and learn from the exchange before it actually matters?

That question led me to build Parlayy.

What I'm building

Parlayy is a voice-first conversation rehearsal platform built around a simple idea: the best way to prepare for a conversation is to practise having it.

Instead of giving users a list of talking points and wishing them luck, Parlayy lets them step into a simulated conversation with an AI counterpart.

The user describes what they're preparing for. Parlayy sets up the other side of the conversation with a defined role, objectives, personality, and objections. The user then speaks naturally while the counterpart responds to what they actually say.

A skeptical client should be allowed to remain skeptical. An investor should be able to question a business assumption. An interviewer should be able to ask a difficult follow-up question instead of moving mechanically to the next item on a list.

The point isn't to make every interaction comfortable. It's to create a useful opportunity to practise responding when things don't go according to plan.

After the conversation, Parlayy evaluates the transcript and produces a coaching report covering strengths, missed opportunities, weak answers, responses to pushback, and suggested next steps.

I want the feedback to be specific enough that users can understand what they did well, where their responses could improve, and what they should try differently next time.

The experience follows three stages:

Prepare → Practise → Improve.

Although the initial scenarios cover things like negotiations, investor pitches, job interviews, and difficult conversations, I see an opportunity to make this useful for anyone preparing for a conversation they care about.

Why voice matters

I could have built Parlayy as a conventional text-based chatbot. That would have been simpler, but it would have missed an important part of the problem.

Real conversations involve more than the words we eventually put on a page. We have to listen, interpret a response, decide what to say next, and communicate our thoughts in real time.

Typing a carefully considered response is different from answering a skeptical client aloud while trying to maintain confidence and explain your reasoning.

Voice doesn't automatically make a simulation realistic, but it brings the practice closer to the experience users are preparing for. They have to formulate their thoughts, say them aloud, listen to the response, and decide what to do next.

That made voice a central part of Parlayy's design rather than a feature I wanted to add later.

It also introduced a significant engineering challenge: how do I coordinate live speech recognition, AI-generated responses, and audio playback so the user experiences a conversation rather than a collection of disconnected API calls?

That's where Gradium comes in.

How Gradium powers Parlayy

Gradium provides the speech-to-text and text-to-speech capabilities that underpin Parlayy's voice interaction. Its streaming speech APIs allow me to integrate speech recognition and audio generation into a browser-based application through WebSocket connections.

Rather than building speech recognition and speech synthesis from scratch, I can focus on integrating those capabilities into the product's conversation flow.

Here's how a typical interaction works:

1. The user speaks

Parlayy captures microphone audio in the browser.

2. Gradium Speech-to-Text

The spoken response is transcribed through Gradium's streaming speech recognition.

3. Parlayy's conversation engine

The application sends the transcript and conversation context to the language model to generate the counterpart's response.

4. Gradium Text-to-Speech

The generated response is converted into audio.

5. The user hears the response

Parlayy plays the audio and resumes listening for the next turn.

The cycle continues until the user ends the session.

Listening with Gradium Speech-to-Text

When a user speaks, Parlayy captures microphone audio and streams it to Gradium's speech-to-text service. The resulting transcript becomes the input for the next turn of the conversation.

This matters because the AI counterpart needs to respond to what the user actually says, not to a predetermined script.

Turn-taking also requires care. A pause doesn't necessarily mean someone has finished speaking. They might be thinking, taking a breath, or searching for the right words. If the application treats every pause as the end of a turn, it can interrupt users before they've finished expressing themselves.

I implemented turn-finalization logic around Gradium's speech events so Parlayy can distinguish an ongoing response from a completed one and submit the finalized transcript before requesting the AI's reply.

Giving the counterpart a voice with Gradium Text-to-Speech

Once the language model generates a response, Parlayy sends that text to Gradium's text-to-speech service.

The service returns audio in PCM chunks, which the browser decodes and schedules for playback. Parlayy manages the playback queue and waits for the response to finish before returning to listening.

That coordination is important. Generating the right sentence is only part of the experience. The audio must also arrive, decode correctly, play smoothly, and finish before the next user turn begins.

I use Gradium to handle the speech synthesis while concentrating my own engineering effort on making the complete interaction work reliably inside the application.

For Parlayy, Gradium is the bridge between an AI that can formulate a response and an experience in which a person can actually hear that response and continue the conversation.

Engineering a voice experience in Next.js

Parlayy is built with Next.js App Router, React, and TypeScript. I kept the application in this stack rather than introducing a separate Python backend just to integrate the speech services.

The application uses three principal server routes:

• /api/gradium/token — provides temporary credentials for the browser's Gradium WebSocket connections while keeping the permanent API key on the server.
• /api/chat — sends conversation context to the language model and returns the counterpart's next response.
• /api/evaluate — evaluates the completed transcript and returns a structured coaching report.

The browser connects to Gradium's speech services using temporary credentials, while privileged API keys remain on the server. This separation lets me use browser-native audio capabilities without exposing permanent service credentials to the client.

Managing the conversation, not just the audio

One of the biggest engineering challenges in a voice application is coordinating asynchronous events.

The microphone, speech recognition socket, language-model request, text-to-speech socket, and audio player all operate on different timelines. A transcript may still be arriving when a turn is ending. Audio can arrive in multiple chunks. A socket can close unexpectedly. A response can finish generating before its audio has finished playing.

Parlayy manages these transitions through an explicit session state machine:

idle → connecting → listening → thinking → speaking → listening

The implementation also includes transcript finalization, audio buffering, playback scheduling, speech-recognition reconnection handling, and cleanup when a session ends.

For example, the client uses a short guard period after playback before resuming microphone listening, helping reduce the risk of capturing the AI's own voice as a new user response. The audio player also buffers incoming audio before playback to reduce interruptions caused by uneven arrival times.

These details aren't particularly visible in a product screenshot. But they're part of the difference between an interface that looks like a voice application and one that can actually sustain a conversation.

Keeping the AI counterpart consistent

Speech alone isn't enough. If the AI agrees with everything the user says, forgets its earlier objections, or invents details about the situation, the practice becomes less useful.

To address this, I separate public scenario information from server-side scenario instructions. Those instructions define the counterpart's role, objectives, objections, knowledge boundaries, and behaviour at different difficulty levels.

The prompt builder also constrains responses to short, natural sentences suitable for spoken delivery.

Conversation history is included in subsequent requests so the counterpart can react to earlier statements and challenge inconsistencies. This is particularly important in a negotiation, where a user might make a commitment early in the conversation and later contradict it.

My goal isn't to create an adversary for its own sake. It's to create a counterpart with a coherent point of view, giving the user something meaningful to practise against.

Turning practice into actionable feedback

The final stage of Parlayy is the coaching report.

After a session ends, the application sends the transcript to the evaluation endpoint. The evaluator generates a structured report, and I validate that result against a Zod schema before displaying it.

The report covers:

• Strengths: what the user handled well.
• Missed opportunities: moments where a stronger response may have helped.
• Weak answers: responses that could be clearer, more persuasive, or better supported.
• Pushback: how the user responded when challenged.
• Next steps: concrete improvements to practise in another attempt.

I consider evidence discipline important here. Feedback should be grounded in what the user actually said, including relevant quotations where appropriate, rather than unsupported claims about their performance.

A useful coach should be able to explain why an answer was weak, identify where a better response was possible, and suggest what to try next.

I also designed the session flow to preserve the transcript if evaluation fails, allowing the report to be retried instead of losing the user's practice session.

Learning from the realities of deployment

Building a working prototype is one thing. Deploying a real-time voice application introduces another set of challenges.

During development, I encountered outbound network timeouts related to my development machine's IPv6 routing behaviour. I addressed that environment-specific problem by adjusting connection behaviour to prefer IPv4, which improved outbound request performance in that environment.

I also encountered a separate production-only voice issue: the AI's text appeared correctly on Vercel, but no audio was heard. The basic HTTP routes returned successful responses, yet that alone wasn't enough to establish that the text-to-speech audio had been delivered and played correctly.

I have since fixed the production issue. The experience reinforced an important lesson: successful API responses and successful builds do not, by themselves, prove that a real-time audio experience works end to end. The actual browser session matters.

It also reinforced why I treat the session, audio, and evaluation paths as distinct parts of the application. A failure in one area should be diagnosable without unnecessarily rewriting the others.

What comes next

Parlayy is already implemented and deployed, but I see several opportunities to improve the experience further.

I'm continuing to focus on custom scenario creation, a more polished live-session experience, end-to-end reliability, and more specific coaching reports.

Custom scenarios are particularly important to the direction I want to take. People shouldn't have to find an exact match in a predefined list to practise a conversation that matters to them. They should be able to describe their own situation and have that description turned into a structured scenario with a clear counterpart and meaningful objectives.

The broader opportunity is to make realistic conversation practice accessible whenever someone needs it, whether they're preparing to negotiate a contract, pitch a business, interview for a job, or navigate a difficult discussion.

I don't see Parlayy as a replacement for real-world experience. I see it as a way to prepare for that experience.

Why I built Parlayy

The idea behind Parlayy is simple: people should have an opportunity to practise important conversations before the stakes are real.

Advice can tell you what a strong answer might look like. Rehearsal lets you try to deliver it, respond to resistance, discover where your reasoning breaks down, and try again.

Building Parlayy has meant bringing together several pieces: a conversation engine that gives the AI counterpart a coherent role, a voice pipeline that connects spoken input with spoken responses, and an evaluation system that turns the exchange into actionable feedback.

Gradium plays a meaningful role in that architecture. Its speech-to-text and text-to-speech capabilities let me build around spoken interaction without having to develop those speech technologies myself. That allows me to focus on the product-specific challenges: natural turn-taking, consistent AI behaviour, reliable audio playback, and useful feedback after the conversation.

I'm building toward an experience that goes beyond an AI that simply answers questions. I want Parlayy to help people prepare, practise, and communicate better when it counts.

Parlayy — Get ready for the conversations that matter.

Try Parlayy

Live product: Parlayy

Built with: Next.js, React, TypeScript, Gradium Speech-to-Text, Gradium Text-to-Speech, and an OpenAI-compatible language model API through OpenRouter.

Learn more about Gradium: Speech-to-Text · Text-to-Speech · Developer documentation