Trust, verification and limits of AI output
Summary
Treat every AI result as a well-organised first draft. The apps reduce the risk — they answer from your text, cite pages, and in one case check their own work — but none can promise correctness. Verify what matters (numbers, names, dates, obligations) against the source, and think about what you send before you send it.
Details
What the apps do to stay grounded
| App | Built-in safeguards | What remains your job |
|---|---|---|
| Ask PDF / PDF KB | Answers are written from retrieved passages only; the writing instruction forbids filling gaps from general knowledge and demands exact numbers. Citation markers pointing to nothing are dropped; each kept citation shows its quote on the page. A "how this answer was built" panel lists the steps | Open the citation; judge whether the passage really supports the sentence; remember summaries over many documents can miss a passage |
| Abstracts | Non-fiction abstracts are compared with the document afterwards and contradicted claims are corrected (when time allows). Fiction abstracts are not checked | Read the abstract against the document; check names and numbers |
| Cheat Sheet | With a document, items come from its text. With a topic only, from the model's general knowledge — nothing checks them | Verify formulas and facts, especially in topic-only mode |
| Checklist | Every item carries a source page and excerpts. Direct level stays with explicit statements | At Moderate and above items include inference; treat as suggestions |
Where AI goes wrong
- Sampling. Long documents are often read in part. Progress messages and the apps' FAQs say so; the result may omit what was not sampled.
- Invented or misplaced details, especially numbers, names and legal wording.
- Misread structure: tables, multi-column layouts and footnotes may be extracted out of order.
- Blind to images. Charts, photos and diagrams are not analysed.
- Fluent but wrong in other languages. Check translations by a speaker when it matters.
- Not advice. Legal, medical or financial decisions need a qualified person.
A quick verification routine
- Open the citation (Ask PDF, PDF KB) or the source page (Checklist) for every claim you will rely on.
- Re-check all numbers, dates and names against the page.
- Ask the opposite question ("Where does the document say otherwise?").
- Compare two runs with different options; items that appear in only one deserve a look.
- Mark the output as AI-drafted if you pass it on.
What is sent to the model
From reading the workers:
- Text, not images. The text of your pages (or of retrieved passages and extracted parts), your question or topic, and your option choices (style, language and so on). Page images used for highlighting citations are rendered on Documentize's side and are not analysed by the AI.
- Scans. In Abstracts, text recognition runs first as a processing step; its output text is what goes on.
- Long documents are split into parts, and several requests are made for one document.
- Searching also uses embeddings (numeric fingerprints of passages) from the same model endpoint.
- Which provider runs the model is a server setting that the code does not show; the book cannot state it. (verify on the live app / with the vendor)
How long things stay
The apps' own FAQ texts differ — read the one on the app you use:
- Ask PDF: PDFs and knowledge base kept about a day, then deleted.
- PDF Knowledge Base: kept about a day, deleted automatically, "not shared with anyone".
- Checklist: documents removed once the task completes.
- Cheat Sheet: "deleted immediately after generation". Documentize also keeps results for a short time for download; see Privacy and data lifecycle for the general rules. Do not rely on any of these as long-term storage; download what you need.
Safe habits for sensitive documents
- Remove what the AI does not need: Splitter to send only the relevant pages; Redactor for personal data.
- Do not upload what you are not allowed to share with a processing service; check your organisation's rules.
- For regulated material, consider self-hosting.