Code your own caller

How to Build an Outbound AI Calling Agent in Python (No Vapi Needed)

By Shreyas Raj. Written from my YouTube video, about 24 minutes long.

Video: I Built an AI Voice Agent for ₹2/Min (No Vapi Needed) on the AI with Shreyas Raj YouTube channel.

Short answer: An outbound AI calling agent is a Python program that uses a phone provider, LiveKit, a speech to text service, a fast LLM and a text to speech service. In the video I collect each API key, fill in an environment file, set up an outbound trunk and place a test call.

What you will learn

Which components make up a code based voice agent and what each does
Where the cost of a call mostly goes
How to collect API keys and build the environment file
How to set up an outbound SIP trunk and buy a number
How to run the agent in a virtual environment and trigger a call

What this voice agent is made of

This is not a white label tool. The agent is coded in Python, and I provide the repository so you can follow along step by step. It can speak several languages, and in the demo I run a school receptionist that answers fee and location questions.

In the video I put the running cost at about 2.3 rupees a minute and say hosted platforms usually land between 7 and 9 cents a minute. Those are my numbers for my setup, so check current pricing before you quote anything to a client.

I should be clear about who this is for. In the video I admit it may be a bit too complex for part of my audience. If you have never touched a terminal, expect to go slowly, and lean on your AI editor whenever something looks unfamiliar. Following the steps in the same order as the video, without improvising, is the easiest way to get a working call.

The stack, piece by piece

Every component has one job. Together they replace what a hosted platform bundles for you.

Vobiz: the telephony provider that supplies the phone number and SIP trunk
LiveKit: open source infrastructure that streams the audio and orchestrates everything
Deepgram: speech to text, the ears of the agent
Groq: the LLM that acts as the brain, chosen for very fast inference and a free tier
Sarvam: text to speech, which handles Indian voices well
OpenAI and other providers: optional fallbacks you can leave empty

Where the cost goes

In my rough cost breakdown, the largest share goes to text to speech and the telephony provider. I say those two account for about 70 percent of the cost. Deepgram for speech to text and LiveKit for orchestration are smaller, and the LLM on Groq is a small part per call.

The only upfront cost I mention is the phone number, which I quote at roughly 500 to 600 rupees a month. That is why I call this a low investment way to learn voice agents.

Step 1: Download the code and open it in an editor

Download the repository as a zip, unzip it and open the folder in an AI powered editor. I use Google Antigravity, but Cursor or VS Code work as well. You will see a dashboard folder and several programs, which looks like a lot but you only need to configure a few files.

Create a new file named env. This is the environment file where sensitive values like API keys go. It is normally not pushed to GitHub, so everyone creates their own. The repository includes an example file showing the structure.

The repository also has a readme that explains the project structure and a documentation section. I do not read through all of it in the video, but it is worth skimming, especially if you want to sync the project with GitHub later.

Step 2: Collect your API keys

I think of the keys as collecting infinity stones. Start with the free ones and paste each into a notepad as you go.

Make sure you note which key belongs to which service as you go. When it comes time to fill the environment file, labels such as the Groq key, the Deepgram key and the Sarvam key must match exactly, or the agent will fail on startup with an authentication error.

Sarvam: sign in, open API keys under developers, create a key
LiveKit: create a project, go to settings, API keys, create a key and copy the URL and credentials
Deepgram: sign up free and create an API key in the console
Groq: start building, open API keys and create a token

Step 3: Set up Vobiz and an outbound trunk

On Vobiz, top up your balance and buy a number from the numbers section. In the video I mention a starting cost of about 500 to 600 rupees a month with some calling balance included.

Then go to SIP trunks, create an outbound trunk and give it a name. Create a credential with a username and password, attach it to the trunk, and optionally restrict access by IP. Copy the SIP domain, username, password and trunk ID into your notes, along with your outbound phone number.

Use strong credentials of your own and never reuse the example ones. I delete mine after recording.

Paste all your gathered values to the AI in your editor and ask it to place them in the right fields of the environment file. It saves time and avoids copy mistakes. Review the result and make sure each key matches its label.

Step 5: Install Python and run the agent

Install Python from the official site. On Windows, tick the option to add Python to PATH during installation, because skipping it causes pain later.

Install the requirements with pip install -r requirements.txt. Create a virtual environment with python3 -m venv venv and activate it with source venv/bin/activate (the command differs slightly on Windows). Then start the agent with python agent.py start.

I hit an error on the first run. When that happens, paste the error into your AI editor. In my case it was a missing certificate package, which it installed. After activating the environment again the worker started fine.

Two details are easy to miss. First, the install commands differ between Mac and Windows, so check the readme for your system. Second, the worker must be running before you trigger a call, otherwise the call script has nothing to connect to.

Step 6: Trigger a call and customize the agent

With the agent worker running, use the call script and enter the phone number to dial. The agent greeted me and answered questions as the school receptionist.

To change the persona, open config.py. There you can edit the system prompt, the initial greeting and other behavior. In the video I also check the call recordings in Vobiz and the sessions in LiveKit.

This first video covers local setup only. I say hosting it on a cloud server and handling inbound calls comes in later videos.

Once the call connects, listen for how it handles interruptions and silence. Small changes to the greeting and the prompt in the config file make a noticeable difference to how natural it sounds, so edit, restart the worker and call again until you are happy.

Watch the full 24 minute video to see me gather every key and place the live test call from start to finish.

Watch the full video See EducateXAI

Results vary and depend on your effort. Read the results disclaimer.

Questions people ask

Do I need to write the code myself?

No. In the video I provide a repository and an AI powered editor to fix errors. You mainly gather API keys, fill in the environment file and run a few commands in the terminal.

What is the environment file for?

It holds sensitive values such as API keys, URLs and trunk details that the program reads at startup. It stays private on your machine and is not meant to be uploaded publicly with your code.

Why use LiveKit?

LiveKit is an open source platform that streams the audio and ties speech to text, the LLM and text to speech together. Hosted voice platforms use similar technology under the hood, which is why building directly can cut cost.

What should I do if the agent will not start?

Copy the full error into your AI editor and ask it to fix it. In my run it was a missing dependency, and reactivating the virtual environment afterwards was enough to get it working.

More from the channel

Prefer short reads? Start with the EducateXAI guides.