AI voice agent handoff: rules for transfer, callback and escalation

1. Sort call reasons into three outcomes

Start with the reasons people already call: opening hours, a new enquiry, rescheduling, a complaint, a technical question or a request for a named employee. Do not begin with telephony features. Put each reason in one of three columns: the agent may complete it, the team must call back, or a person should take over now.

The first column is for bounded tasks with an approved answer or a reliable integration. The callback column is useful when the team needs records, judgement or preparation but does not need to interrupt current work. Immediate transfer is for an explicit request for a person and for cases where delay or an invented answer could cause meaningful harm. NIST's AI RMF Playbook recommends documenting system knowledge limits and how people oversee outputs; it also recommends involving operators in tests that resemble real use.

• Agent completes: approved information, enquiry capture or a confirmed action through a working integration.
• Callback: unresolved but non-urgent questions, quotations, diagnosis or a named employee who is unavailable.
• Transfer now: the caller asks for a person, an agreed escalation condition occurs, or the agent reaches a consequential limit.
• Stop path: the call does not fit the service, required information cannot be obtained, or the destination is closed.

2. Write observable triggers, not “when necessary”

A usable rule describes what the caller says or what state the system detects. Examples are “the caller asks to speak to a person,” “the requested appointment cannot be confirmed,” “the caller disputes a charge,” or “identity cannot be verified for an account change.” Avoid rules based only on mood, importance or a broad label such as “complex.” Those are difficult to test and may route similar calls differently.

For safety-critical or regulated situations, agree a separate script with the responsible professional. A general business voice agent should not diagnose, invent emergency advice or claim that a live specialist is available. The safe action may be to state its limit, give an approved contact route and end or record the request according to the business procedure. This guide is an operating checklist, not legal or sector-specific safety advice.

• Exact trigger: what phrase, missing fact or failed action changes the route?
• Destination: which person, role or queue owns this type of call?
• Availability: during which hours is an immediate transfer genuinely staffed?
• Caller message: what will the agent say before transfer or callback?

3. Design the receiving side, not only the AI side

A transfer is useful only if the receiving person knows why the call is arriving. Pass the minimum useful context: caller name, callback number when provided, reason for calling, facts already confirmed and the requested next step. Do not make the caller repeat everything, but do not include unrelated personal data in a transfer note.

ElevenLabs documents conditional transfers to configured phone or SIP destinations and distinguishes conference, blind and SIP-based transfer types. The available caller and operator messages depend on the telephony configuration. Twilio describes a warm transfer as a flow in which the first agent consults the receiving agent before leaving the call. Therefore, write the business requirement first—direct transfer, transfer with context, or callback—and verify that the chosen provider and phone setup can actually deliver it.

• Who answers the destination and how do they recognize an AI handoff?
• Which confirmed facts are included in the summary?
• Must the receiver accept before the caller is connected?
• What should happen if the destination is busy, rejects the call or disconnects?

4. Give every transfer a no-answer and after-hours fallback

Never promise “I am transferring you now” until the transfer path is available. Check business hours and the destination status where the phone system supports it. If no person answers, return to an agreed fallback: collect a callback number and reason, create a task, offer an approved alternative channel, or let the caller end without leaving details.

Set the expectation precisely. “A colleague will review your request during business hours” is safer than inventing a response time. If the business has a real service-level commitment, encode that exact commitment; otherwise do not create one in the agent script. Also define what happens when the callback number is missing or unclear and whether the agent should repeat it for confirmation.

• No answer after the agreed ring time.
• Destination closed or employee marked unavailable.
• Transfer starts but the call drops.
• Caller declines to leave a number or asks for another channel.

5. Test the decision and the phone path separately

Build a small test matrix from real call patterns. Include a straightforward enquiry, an explicit request for a human, an ambiguous request, an unavailable destination, a failed calendar or lookup tool, a caller who changes intent and a phrase that sounds urgent but is not covered by the service. For each case, record the expected outcome, destination, caller message, context fields and fallback.

ElevenLabs' official testing documentation separates multi-turn simulations, next-reply tests and tool-call tests. Tool-call tests can check that the transfer tool and expected parameters are used; simulations can check the full conversation outcome. Because generated conversations vary, repeat important scenarios rather than treating one successful call as proof. Use controlled test data and prevent test calls from creating real bookings or customer records unless that side effect is deliberately isolated.

• The correct route is selected for ordinary and awkward wording.
• The destination and transfer parameters are correct.
• The caller hears an accurate message before waiting.
• The fallback works when the receiver or an integration is unavailable.
• The summary contains enough context and no unnecessary sensitive data.

6. Launch narrowly and review missed escalations

Begin with a limited set of call reasons and a named owner for the handoff rules. During the pilot, review a representative sample of completed, callback and transferred calls. Count missed escalations, unnecessary transfers, failed connections, incomplete callback details and cases where the receiving person lacked context. A high transfer count is not automatically good or bad; compare it with the agreed call mix and outcomes.

Update a rule when evidence shows a recurring failure, then rerun the related tests. Keep a visible version of destinations, hours and fallback wording so operational changes do not remain only in an AI prompt. Human handoff is successful when callers know what will happen next and the team can act on the information—not merely when the phone system technically connects two call legs.