Crawling & Indexing

The journey of a search query starts before you ever type a search, with crawling and indexing across thousands or millions of your structured and unstructured data sources (documents, emails, sql, Hadoop etc.).

These processes lay the foundation — they’re how we gather and organise information about your data so we can return the most useful results to you.

Your data is like an ever-growing library with many books and millions of pages. Frisk essentially gathers the pages during the crawl process and then creates an index, so we know exactly how to look things up. Much like the index in the back of a book, the Frisk index includes information about words and their locations. When you search, at the most basic level, our algorithms look up your search terms in the index to find the appropriate documents or data.

The search process can get more complex from there, taking into account Boolean options and other advanced filtering options based on references inferred between data sets.

The art of delivering you a powerful and effective search experience as all in the way we build and manage the index.