Document library

This chapter only applies to installations where the Library module is enabled. The assistant is a separate module, which may or may not be enabled.

The document library gathers the regulatory texts useful to your team: regulations, guidelines, ECB or EBA publications, internal policies. Documents are indexed for search. Once validated, they are also indexed for the assistant, if it is enabled.

It opens from the Resources › Regulatory library menu. The page is titled Regulatory library.

Common documents and own documents

Each document has an origin, shown by an icon in the list and by the Origin field of its page:

Origin Icon Description
Common globe The regulatory library, shared with every organisation. Its documents can be neither edited nor deleted from the application.
Own lock The documents added from the application. Only your organisation sees them.

The common library is fed by an automatic collection of the authorities’ publications, among them the EBA, ESMA, the ECB (including banking supervision), the BIS (including the Financial Stability Institute) and EIOPA.

  • Each PDF of a publication (report, annexes) becomes a separate document.
  • Only PDFs, Word documents, ODT and text files are collected. Zip archives, spreadsheets and HTML pages are left out.
  • After each collection that finds new publications, a “[Hubert] New Publications” e-mail (30 publications at most) is sent to the recipients configured for your installation.

The document list

For each document, the list shows:

  • the icon of its origin;
  • the title, which opens the document page;
  • the type, source, language and topics. Clicking one of these values filters the list on that value;
  • the Published date. A document uploaded without a date takes the creation date of the PDF;
  • the Status of its processing:
Status Meaning
Converting The document is being converted to Markdown.
To validate The Markdown is ready and waits to be validated before being embedded for the assistant.
No Markdown The conversion produced no Markdown. The plain text can be embedded instead.
Embedding The document is being embedded for the assistant.
Embedded The document can be used by the assistant.
Embedding failed The embedding failed. It can be tried again.
Not indexed The document is not indexed for the assistant yet.

The filters cover:

  • Search: title, file name, topic;
  • Library: All, Common or Own;
  • Type, Source, Language and Topics.

Clear filters resets them all.

Each row offers the Quick view icon, which opens a quick preview of the document, and an action menu: View, Quick view and, for an own document only, Delete.

At the top of the page:

  • Refresh reads the list again;
  • Index N own documents indexes at once for AI search all your own documents that are not indexed yet. A confirmation is requested: each indexing uses AI tokens. This button only appears when some documents remain to be indexed;
  • Ask the library opens the search;
  • Add documents opens the add page.

How to add documents

Click Add documents to open the Index documents page. It has two blocks, which can be used together:

  • Upload files: drop or choose files from your computer, in PDF, DOC, DOCX, ODT or TXT format, of 100 MB at most. Each file is checked when added (format, size, duplicate).
  • Index from the web: enter the document URLs, one per line. Only http:// and https:// addresses are accepted. Invalid lines and duplicates are flagged and will be skipped.

Then click Index N documents.

  • The progress of each file is shown.
  • The items that could not be sent are listed, with Retry.
  • At the end, the page offers Follow the indexing, Add more documents and Back to the library.
  • If you leave the page during an upload, a confirmation is requested: the files in progress would be lost.

Documents added from the application are always own documents of your organisation. Their conversion to Markdown and their indexing for search run in the background. The Status column of the list follows their progress.

The indexing tasks

The Admin › Settings › Import tasks menu opens the Indexing tasks page. It lists the additions of your own documents, with for each one:

  • the Timestamp;
  • the Indexing Status and its duration: once indexing is complete, the document can be found by search;
  • the Embedding Status and its duration: once vectorization is complete, the document can be used by the assistant;
  • the Document concerned: its title, its file name or its URL;
  • the error message in case of failure.

The Refresh button updates the page.

The document page

The document page offers at the top Library (back to the list), Download PDF and, for an own document, Delete. It has three tabs.

The Document tab

The Document tab shows the document preview (PDF or Word) and its Details: AI index, origin, type, source, language, topics, publication date, date added and file.

If an own document is not indexed, the Index for AI search button indexes it so that the AI search and the assistant find it. Indexing uses AI tokens.

The Summary block offers Summarise:

  • The AI reads the document and writes its key points, in the interface language. A long document can take a minute.
  • The summary carries the mention “Written by AI on {date}. Check it against the document.”
  • For a long document, the summary only covers its beginning, which the page states.
  • The summary is kept for the document and the language. The summary of a common document is shared by every organisation.

The Markdown tab

The Markdown tab shows the document’s text converted to Markdown by Docling, with its headings and tables, as well as its size in tokens and characters.

  • The conversion is prepared in the background. A long document takes a few minutes.
  • Only PDF and Word files are converted. Scanned documents go through character recognition (OCR).
  • If the conversion is not possible, the tab states “No Markdown version”.

For an own document, the Validation and embedding block shows where the document stands and offers the possible actions:

Status Actions
To validate Validate and embed: check the Markdown, then validate it to embed it for the assistant. Re-convert.
No Markdown Embed the plain text: the document’s plain text is embedded instead. Re-convert.
Embedded or Embedding failed Re-embed, Re-convert.

Re-convert asks for a confirmation: the new Markdown replaces the current one and has to be validated again. Embedding uses AI tokens. This is why a document is only embedded for the assistant on request.

Common documents are validated by the platform administrators.

The Questions tab

The Questions tab lets you ask the assistant about this document alone.

  1. Choose What the assistant reads:
    • Full document: the assistant reads the whole document;
    • Excerpts (search): the assistant looks for the useful passages in the document.
    • A document too long to be read whole switches to excerpts mode: “Too long to be read whole: questions search its excerpts”.
  2. Ask your question, or choose one of the three suggested questions.
  3. New conversation starts again from scratch.

The Conversations about this document panel lists your conversations about this document. You can reopen or delete them.

How to search the documents

Click Ask the library to open the Search the library page.

  1. Type one or more words in Words to look for, for example “liquidity coverage ratio”.
  2. Choose Look in: Titles and text or Titles only.
  3. Add criteria if needed: Library (common or own), type, source, topics, language, and publication period (Published from … to). Each criterion shows the number of documents per value.
  4. Click Search.

Search rules:

  • Documents in every language are searched.
  • Whole words are matched, not their beginnings: “liq” does not find “liquidity”.
  • The search also finds documents that cover the subject in other words (search by meaning).
  • A search on criteria alone, without any word, is possible. The most recent documents are then shown first.

Results are shown in pages of 20. For each document, the page shows the Closest passage, with the words found highlighted and its section. Open the document at this passage opens the document’s Markdown tab, at the highlighted passage. Duplicate excerpts are grouped: Show N more excerpt(s) shows the next ones.

The search and its criteria are kept in the page address, which can be shared.

The assistant

If the assistant is enabled, the speech-bubble button of the top bar (“Ask the assistant”) opens the Assistant panel beside the page. On a narrower screen, it opens over the page.

The first time it opens, the panel reopens your latest conversation.

To ask a question:

  1. Type your question in the “Your question” field, or choose one of the suggested questions.
  2. Press Enter or click Send. Shift+Enter adds a new line.
  3. While the answer is being written, the Stop waiting for the answer button interrupts it. If the answer fails, Try again asks for it again.

Answers are formatted. They cite the library documents with a link, and the Library documents list follows each answer:

  • A link opens a window on the cited passage, highlighted. Show the PDF shows the original document.
  • The arrow icon opens the document page in a new tab, on the Markdown tab, at the cited passage.

The assistant only relies on the documents embedded for it (Embedded status). It searches both by meaning and by exact words, such as article numbers, template codes or acronyms.

Warning

Answers are generated and may be wrong: the regulatory texts are the reference.

The assistant keeps your conversations:

  • History lists your conversations, most recent first. Only you can see them.
    • Click a conversation to reopen it.
    • The delete icon deletes it for good, after confirmation.
    • Show older conversations shows the next ones.
  • New conversation starts a conversation with no memory of the previous one.