Testing and safety
Before an AI answers a business's phone, it should be tested against the calls that go wrong: gas smells, angry customers, discount tricks, and questions it cannot answer. This page shows our eval suite, the rules that run before the model, the switches that keep messages off until a business approves them, and how the AI tells callers it is an AI.
Aplos AI · Atlanta, Georgia · last checked 2026-11-09
Why this page exists
We have one paying client, and it is not an answering-service client. We have no managed-plan customers and no reviews. So instead of testimonials, this page shows how the AI is tested and where it is built to stop.
Everything below describes code in our own repositories as of November 9, 2026: the voice Worker that answers our phone line and customer lines, the follow-up Worker, and the eval suite that runs against both the rules and the model. When something is built but switched off, we say so. When a test exists but we did not run it for this page, we say that too.
If you want to check any of it yourself, call our line at (470) 971-3769. It answers as an AI and tells you so in the first sentence.
Rules before the model
Application code owns every action: transfers, hang-ups and safety instructions. The language model only phrases answers from facts the business approved, and it is only asked after fixed rules have looked at what the caller said.
If a caller mentions a gas smell or gas leak, fire, sparks, carbon monoxide or someone who passed out, a fixed script runs: leave the building, call 911 from outside, and for gas also call the gas company from outside. The model is not consulted. The call is flagged SAFETY in the owner summary.
Phrases like no heat, no AC, burst pipe, flooding, sewage backing up or no power, said with now, tonight or ASAP, go to the business's on-call number if one is set. If not, the request is marked priority and the AI asks for the address and a callback number.
Asking for a person, the owner, a manager or a technician triggers a transfer when the business has a transfer line. When it does not, the AI says so plainly and takes details for a callback.
The AI is told never to confirm an appointment. If the model's reply still says booked, scheduled or confirmed, code replaces it with: I can't confirm a time on this call, followed by the business's own booking note.
Prices, warranties, discounts, dates and technician names come only from the business's approved facts. Anything missing gets: the team will confirm it. Replies are kept to one or two short sentences, and the eval fails any reply over 50 words.
A phone number is taken from what the caller actually said, never from the model. A name or address the model extracts is kept only if it appears in the caller's words. Urgency set by the safety rules is never downgraded.
The eval suite
The Front Desk receptionist has an eval suite built on promptfoo, an open-source testing tool. It sends scripted callers through the real rules and the same Workers AI model the voice line uses, for a made-up Atlanta HVAC company.
Every reply in every case is checked for three things: it stays under 50 words, it contains no markdown (it is spoken aloud), and it never claims a booking. Then each case has its own check:
The rule we work by: every bad call seen in production becomes a new case. The suite runs before any change to the prompt, the rules or the model.
Spanish-speaking callers are answered in English. Speech recognition and the voice are English today, so fixing it is a phone-platform language change, not a prompt tweak. If a lot of your callers speak Spanish, Korean or another language, tell us on the call. A human answering service may suit you better for now.
We do not publish a pass rate for the eval suite. It runs against a live model, results vary run to run, and a single number would hide which case failed.
Code tests
Separate from the model evals, the rules and Workers have their own tests. These are the ones we ran on November 9, 2026, in a clean checkout of the site repository.
| Test | What it checks | Result on Nov 9 |
|---|---|---|
| Public line unit tests | AI and transcription disclosure in the greeting, spam and robocall pitches closed and never forwarded, team forward screens the reason and never speaks the owner's number, model failure falls back instead of going silent, quiet hours, after-hours messages, honest reporting of uncertain sends | 19 of 19 passed |
| Receptionist rules | Life-safety rule runs before any model, urgent transfer, human requests with and without an on-call line, booking claims overridden, phone taken from caller words only, urgency never downgraded, model failure fallback, owner summary with safety flag | Passed |
| Follow-up core rules | Quiet hours across time zones and daylight saving, Sunday skip, SMS only with consent, opt-out keywords and phrases, reply classification, stop on status change, SMS segment counting, conservative recovered-revenue report | Passed |
| Failure alerts | One alert per failure even when two fire at once, fixed recipient, uncertain sends held, a cap of ten alerts per hour | Passed |
| End-to-end Worker tests | Full Worker runs on Miniflare with fake Twilio, Jobber and email providers, for customer lines and estimate follow-up | Exist; not run for this page |
These tests use fake providers. They prove the code behaves as written, not that a carrier or CRM will. Live connections are tested per business before launch.
Off switches
Each part that can call, text, email or write into your software has a global switch and a per-business switch. Both have to be on.
The voice Worker refuses calls unless its master voice switch is on, and each business line has its own enabled flag. Before a line goes public it can be limited to an allowlist of test callers. Our own line has a separate switch so it can be turned off without touching customer lines.
Missed-call text-back needs the global SMS switch and the business's own SMS flag. A caller gets at most one text-back per window. In our checked-in configuration, the global SMS switch is off.
Estimate follow-up sends only when the global delivery switch and the business's delivery flag are both on. With either off, the whole pipeline still runs and every message is marked simulated, so you can review exactly what would have gone out. The global delivery switch is off today and stays off until a pilot business is accepted end to end.
Writing call requests into a business's Jobber account needs its own global switch plus the business's setting. It creates at most one request per call, and a lost response is treated as uncertain, never as success. That switch is off today.
If a text or email send comes back ambiguous, it is marked uncertain and not retried, so nobody gets the same message twice. The owner sees the uncertainty instead of a false success.
Plans include a fixed allowance. At the allowance, the approved fallback takes over. We agree any extra usage before it starts, and there is no automatic overage bill.
Hard limits
These are set in code, not left to the model.
Texting rules
These apply to estimate follow-up and review requests.
AI disclosure
Callers should never wonder whether they are talking to a person.
On our own line, the first thing every caller hears is:
Hi, thanks for calling Aplos AI. I'm an AI assistant, and this call is transcribed.
If a caller asks whether they are talking to a robot, it answers yes, in every branch of the call. A unit test fails if the greeting stops disclosing the AI or the transcription.
On a customer's line, the greeting is fixed in code:
Thanks for calling [business name]. This is the virtual assistant, and this call is transcribed. How can I help you today?
The owner's call summary labels what the caller said as unverified: the caller's own words as understood by the assistant, to confirm before dispatching.
Call transcripts are deleted after 7 days. Operational events store metadata only, with no audio, transcript, caller number or lead details, and are deleted after 30 days. See the privacy policy for the full picture.
Questions
Not on a managed plan yet. We have one paying client, an Atlanta sign shop, for a website and lead dashboard, not phone answering. That is why this page describes tests instead of results.
Yes. Call (470) 971-3769 and pick the demo. It plays the receptionist for a made-up HVAC company. You can also try it in your browser on the live demo page.
It says the team will confirm it and takes your details. It is told never to invent a price, warranty, discount or appointment time, and code overrides any reply that claims a booking.
No. Discounts are not something the model can grant, and a prompt-injection case in the eval suite checks that a caller demanding 50% off does not get one.
Not yet. Spanish-speaking callers are answered in English today. That is a known gap, listed in our own eval notes.
See it for yourself
Book a free 30-minute call and we will walk through the tests for your kind of calls, then give you a written automation plan for free. Or call our AI first: (470) 971-3769. It will tell you it is an AI.