AI chatbot trained on your own data: a plain guide for small businesses

Most owners who look for an AI chatbot trained on their own data want ChatGPT that knows their price list, returns policy and staff handbook. Before paying, you need straight answers on three things: is it safe to upload your files, will it make things up, and what will it really cost? Here are those answers in the providers’ own words, the arithmetic behind the bill, and two tools to scope your own project this week.

Accuracy is the shared worry. In the Federal Reserve’s 2026 survey of small employer firms, it was the top challenge for businesses already using AI (46%), and only 7% of users had fully built AI into how they work (Federal Reserve small business survey).

What an AI chatbot “trained on your own data” really is

In almost every product sold this way, nothing is trained. Before the model answers, the system looks up your documents and hands it the relevant passages. This is retrieval-augmented generation, or RAG, which AWS defines as making a model reference “an authoritative knowledge base outside of its training data sources before generating a response” (AWS: what is RAG).

Think of it as an open-book exam:

1. Your documents are cut into short passages and indexed by meaning.
2. A question comes in and the system finds the best-matching passages, typically three to ten.
3. The model gets only those passages, with an instruction: answer from these, cite them, and say so if the answer is not there.
4. Nothing is retrained. Remove a document and the assistant stops “knowing” it on the next question.

That helps a buyer. Updates are instant: change the PDF and the answers change. Answers can point to their source, the main defence against wrong answers. Access can be limited per person, so HR files reach HR only. The weak point is the documents: an out-of-date, contradictory or missing one becomes a wrong answer.

Three ways to get one

1. Built into tools you already pay for. Microsoft Copilot, Gemini in Google Workspace, or projects with uploaded files in ChatGPT Business and Claude Team. Internal use, per-seat pricing, quick to start.
2. A helpdesk AI agent. Intercom Fin or Zendesk AI agents answer customers from your help centre and charge per resolution.
3. A custom build on a provider’s API. Your own app with retrieval, permissions, logging and handoff, paid for in tokens plus build and upkeep. It fits when you need several sources, strict per-user access, your own branding, a set data region or links into other systems. That is the kind of assistant that answers from your own documents our team builds.

Where it earns its keep in a 5–50 person business

Customer support. Road, with a 20-person support team and 10,000 conversations a month, reports that Intercom’s Fin resolves 63%, after it rewrote its help articles and added daily review (Fin case study: Road). RB2B says Fin “handled 60% of our support tickets”, and two-person ChatPRD says it is “handling 70% of our volume” (Intercom small business page). These are self-reported vendor case studies, published because they went well. Your rate depends on your questions and content.

The internal handbook. HR policies, procedures, IT how-tos and onboarding notes, answered in plain words, with each answer pointing to the page it came from.

Sales and operations. Proposals, pricing rules, supplier contracts. We found no verified small-firm example, so this is our view: fewer questions, higher stakes, so cite the exact clause and lock down permissions.

A lesson from a big company. Klarna’s assistant handled two-thirds of its customer chats in its first month, doing “the equivalent work of 700 full-time agents” (Klarna press release). By May 2025 Klarna was investing in human support again, telling Bloomberg that “there will be always a human if you want” (Customer Experience Dive).

Is it safe to upload your documents?

None of the four big providers trains on business or API data by default; free and personal plans differ. In their own words:

All pages checked 4 October 2026.

“Not trained on” does not mean “not stored”

OpenAI keeps API abuse logs “for up to 30 days, unless longer retention is required by law”, with zero retention on eligible endpoints only (OpenAI data controls). Anthropic deletes commercial data “within 30 days” by default and offers zero retention by agreement (Anthropic retention policy). “Required by law” is real: in 2025 a US court order in the New York Times case made OpenAI keep ChatGPT logs it would normally delete, until October 2025 (Mashable via Yahoo News). We could not confirm ChatGPT Business’s default retention from OpenAI’s own pages, so ask in writing. If data must stay in one region, OpenAI lists residency regions including the UK, Europe, the US and India (OpenAI data controls); in a custom build, where data lives is a cloud hosting decision you control.

The real leak is staff on personal accounts

Paste a contract into a free or personal chatbot and consumer terms apply. Anthropic keeps consumer chats for five years if the user allows training (Anthropic). Google’s free Gemini API tier allows human review for US users and warns “Do not submit sensitive, confidential, or personal information to the Unpaid Services”; UK, EEA and Swiss users get the paid terms (Gemini API terms). In our view a custom GPT on a personal ChatGPT Plus account has the same problem: no central admin, no data processing agreement, no permissions. The fix is cheap: a company plan and a one-page policy on approved tools.

Over-shared folders become over-shared answers

Copilot “only surfaces organizational data to which individual users have at least view permissions” (Microsoft Learn), and Gemini “follows your existing Google Workspace permissions” (Google). The flip side: if the salary spreadsheet is shared with everyone, the assistant will quote it to everyone, so fix sharing first. For custom builds, OWASP calls for “permission-aware vector and embedding stores” (OWASP LLM08:2025).

Prompt injection, in one paragraph

Any document or email your assistant reads can carry hidden instructions. The UK’s National Cyber Security Centre says today’s models “simply do not enforce a security boundary between instructions and data inside a prompt”, and such attacks “may never be totally mitigated” (NCSC). So limit what the assistant can do: a customer-facing bot reads help articles, but cannot issue refunds or open other customers’ records.

Wrong answers are your responsibility

Air Canada’s chatbot told a customer that bereavement fares could be claimed after travel. The airline’s own policy page said they could not. Air Canada argued the chatbot was “a separate legal entity”. British Columbia’s Civil Resolution Tribunal called that “a remarkable submission”, said “It makes no difference whether the information comes from a static page or a chatbot”, and ordered the airline to pay C$812.02 (Moffatt v. Air Canada, 2024 BCCRT 149). Small money, clear principle: whatever your bot says, you said.

Your documents reduce errors; they do not remove them

Stanford researchers found that leading legal research tools built on retrieval still hallucinated on roughly 17% to 33% of 202 queries, and that vendors’ “hallucination-free” claims were “overstated” (Stanford RegLab). In April 2025 the coding tool Cursor’s support bot invented a one-device-per-subscription policy. Cursor had to post “We have no such policy… this is an incorrect response from a front-line AI support bot” as users cancelled in public (The Register).

Wrong answers come from four places: retrieval finds the wrong passage or none, two documents disagree, the answer is missing so the model fills the gap, or the question needs judgement the documents cannot supply.

Eight fixes, in order of value

1. Fix the documents first. One current, dated version of each policy. Road rewrote its articles “with subheadings and simplified language” before launch (Road case study).
2. Allow “I don’t know”, with a handoff to a person. Anthropic says letting the model admit uncertainty “can drastically reduce false information” (Anthropic: reduce hallucinations). It would have stopped the Cursor case.
3. Cite every answer. Anthropic’s Citations feature returns pointers “guaranteed to contain valid pointers to the provided documents” (Anthropic Citations). A citation is not proof, but checking takes seconds.
4. Answer only from your documents. Anthropic advises telling the model “to only use information from provided documents and not its general knowledge” (Anthropic).
5. Keep a person in the loop where mistakes cost money: draft-only mode for refunds, pricing exceptions and contracts, plus a daily sample review.
6. Draw permission lines. The assistant reads only what the asker may read, and a public bot never acts without a person; OWASP calls this risk “Excessive Agency” (OWASP Top 10 for LLM applications).
7. Test before launch and after every change with 50 to 100 real past questions. In our view, it is the step most often skipped.
8. Label the bot and offer a person. Several laws expect it, and it is where Klarna ended up.

Anthropic is candid that these techniques reduce hallucinations but “don’t eliminate them entirely” (Anthropic).

What it really costs

The model bill, worked out

Worked example. Assumptions (ours, for illustration): a 10–30 person firm asks 1,500 questions a month, about 70 a working day, across an internal assistant and a website bot. Each question sends about 6,000 input tokens (1,000 of instructions, 4,000 of retrieved passages, 1,000 of chat history and the question) and gets about 500 output tokens back, roughly 375 words.

Monthly input: 1,500 × 6,000 = 9.0M tokens. Monthly output: 1,500 × 500 = 0.75M tokens.

On Claude Haiku 4.5 at $1 per million input and $5 per million output: 9.0 × $1 = $9.00, plus 0.75 × $5 = $3.75. Model bill: $12.75 a month, under one cent per question.

The same arithmetic on other models:

Official API prices checked 4 October 2026 (OpenAI pricing, Anthropic pricing, Gemini pricing). Gemini 3.8 Flash doubles to $1.50 / $7.50 on 1 January 2027, taking its row to $19.13 and $76.50.

So the model bill is tens of dollars a month, not thousands, and model choice moves it about 25 times. Add a 30% buffer, because providers count tokens differently: Anthropic says its tokenizer for Claude 4.7 and later “produces approximately 30% more tokens for the same text” (Anthropic pricing). The extras are small: indexing 1,000 documents of 3,000 tokens with OpenAI’s text-embedding-3-small costs about $0.06, once (OpenAI pricing), and a custom build’s AWS Lightsail server and managed database run about $54 a month (Lightsail pricing).

Off-the-shelf tools

Prices checked 4 October 2026 (ChatGPT, Claude, Microsoft, Intercom, Zendesk). Support examples assume the AI resolves half of 1,500 conversations (Road reported 63%). Zendesk’s page mixed currencies, so confirm yours. Gemini joined Workspace Business plans in January 2025 (Google Workspace blog); we have not re-checked the current US price.

Per-resolution pricing is easy to start but grows with volume. At 6,000 conversations with half resolved, Fin costs about $3,030 a month (3,000 × $0.99 + $58 in seats), against $51 of Haiku 4.5 model fees plus $54 of hosting. But Fin includes a helpdesk, analytics and handoff, so add build cost and review time before you compare.

Custom builds

Clutch’s pricing guide, built from client reviews of every kind of AI project, puts the most common cost at $10,000–$49,999 and the average at $120,594.55 (Clutch AI pricing guide). That average covers all AI work and leans offshore, so treat it as a wide range. In our view a narrow assistant (one or two sources, one channel, citations, handoff, a simple admin page) sits at the low end. Each extra integration, permission model, language or channel pushes it up.

The costs nobody quotes

The real money is not the model. It is document clean-up, building the test set, and someone owning the content and reviewing flagged answers, perhaps one to three hours a week.

A one-page cost worksheet

Fill in the middle column; the example repeats the worked example above.

Prices as in the tables above, checked 4 October 2026. In the example, line H outweighs F and G combined at any hourly rate above about $8.

The rules in plain English

UK. If your documents hold personal data, do a short data protection impact assessment: the ICO lists “new technologies, or the novel application of existing technologies (including AI)” as a trigger when combined with other risk factors (ICO: high-risk processing). Your provider needs an Article 28 processor contract (ICO: processor contracts); the big providers publish one, a small start-up may not. Under the UK-US data bridge a US provider “must be certified to the UK Extension and appear on the DPF List”; otherwise use the IDTA or UK Addendum (GOV.UK data bridge factsheet). The ICO’s AI guidance is under review after the Data (Use and Access) Act (ICO AI guidance), and since 19 June 2026 every organisation needs a data protection complaints process (ICO). A support bot rarely makes solely automated decisions with legal effect, unless it approves refunds, credit or job applications by itself.

US. The FTC says “there is no AI exemption from the laws on the books” (FTC). DoNotPay paid $193,000 over untested “robot lawyer” claims (FTC: DoNotPay), so never advertise your bot as “always right” without evidence. California bans bots that mislead people about their artificial identity to drive a sale, with clear disclosure as a defence (California Business and Professions Code). Utah, per a Future of Privacy Forum summary, requires disclosure when a consumer asks, and up front for high-risk advice (FPF on Utah’s AI law).

EU, if you sell there. The AI Act covers firms outside the EU whose AI output is used in the Union (AI Act Article 2). From August 2026, Article 50 says people must be “informed that they are interacting with an AI system” unless that is obvious (AI Act Article 50); high-risk rules for Annex III areas start in December 2027 (European Commission) and do not cover a website support bot. Label it as AI, the simplest rule everywhere.

The wrong-answer risk matrix

Use this to decide how much freedom each use case gets. Ask two questions:

• How costly is a wrong answer? Low: an inconvenience. Medium: a refund, a credit or wasted staff time. High: legal, safety or contract exposure, or a large sum.
• Who reads it first? Staff, who can check before acting, or a customer, who acts straight away.

List the ten questions people ask most, from your inbox, helpdesk or team chat, and place each in a cell. Each cell sets four things: citations, “I don’t know” with handoff, human review, and what the assistant may read. Start in the low-cost row and move down only when your test results support it. If most questions sit in the top row, a tool you already pay for may be enough; if they cluster at the bottom, you need drafts and sign-off, not automation. The filled-in matrix and worksheet make a good brief, whether you build, buy or bring them to a call with our team.

Thirteen questions to ask a vendor

Ask for written answers.

1. Training. Is our data used to train any model? Where is that written?
2. Retention. How long is everything kept, and can we get zero retention?
3. Region. Which model, plan and region, and what covers UK or EU data transfers?
4. Contract. Will you sign an Article 28 data processing agreement and list sub-processors?
5. Permissions. Can answers be limited to what each user may see? Show us.
6. Citations. Does every answer link to its source?
7. “I don’t know”. What happens when the answer is missing? Show us live.
8. Testing. Will you report accuracy on our real questions, before launch and after changes?
9. Logs. Can we see, flag and export every conversation?
10. Actions. What can the bot do without a person’s approval?
11. Cost. What is the price at two and five times our volume, and is there a spend cap?
12. Exit. Do we own the documents, index, prompts and logs?
13. Claims. What evidence backs your accuracy figures? The FTC expects “competent and reliable evidence” for such claims (FTC: Workado).

Common mistakes

• Staff using personal chatbot accounts for company files.
• No “I don’t know” path, so the bot invents policy.
• Switching on Copilot or Gemini before fixing over-shared folders.
• Launching without a test set.

An assistant that answers from your own documents is a lookup system with a fluent writer on top, only as good as the files behind it and the rules around it. Pick one use case from a safe cell of the matrix, tidy its documents, test it on real questions, and let the results decide your next step.