Most website chatbots invent answers when they don't have one. This one is being designed to answer only from the documents of the business, to say where the answer came from, and to say "I don't know" when it can't find one. The name is provisional.
The story so far
-
2 Oct
Looking at what exists
I compared the chatbot plugins that already exist for WordPress and the hosted services. The complaints that repeat are two: answers that are made up, and doubts about privacy. None of the ones I saw makes a point of checking itself before answering.
-
2 Oct
Three bets to prove
Written down before building anything: a bot in Spanish that says "I don't know" and cites its source beats the current ones; a "gatekeeper" in front of and behind the search reduces invented answers; and there are businesses in Europe that want their data to stay there. None of the three is proven yet.
-
2 Oct
Testing the gatekeeper
I tried an open model that is meant to act as that gatekeeper, with 32 sentences of my own in Spanish. It saw 9 out of 10 times whether a text answered the question, and spotted all 10 attempts to trick it. Deciding whether it needed to search at all was weaker: 6 out of 12, and 9 out of 12 with a better wording. It is a small sample, a hint and not a measurement.
-
Next
The basic version first
Phase one has no gatekeeper: upload documents, search them, answer citing the source. Then a test with 50 to 100 real questions in Spanish. The gatekeeper is only added where that test shows it is needed, and the WordPress plugin comes last.
It's still being designed, so there's nothing to try yet. The name is provisional.