Index Search via JDito

Use the indexsearch module from @aditosoftware/jdito-types to build and execute search requests against ADITO index search. This page explains how to query the standard index with indexsearch.COLLECTION_SEARCH_INDEX, how to query the vector index with indexsearch.COLLECTION_VECTOR_INDEX, and how to run indexer jobs from JDito.

Import and Basic Concept

import { indexsearch } from "@aditosoftware/jdito-types";

The indexsearch module provides builders and constants for search requests against ADITO index search. New implementations should use indexsearch.createIndexQuery() to create the query and indexsearch.searchIndex(query, collection) to execute it. Older methods such as indexsearch.search(...) are legacy APIs and should only be used in existing logic.

The collection parameter determines which technical index is searched.

Collection	Purpose	Group definition
`indexsearch.COLLECTION_SEARCH_INDEX`	This collection contains the standard index for global search, lookups, and text-based index searches.	`INDEXGROUP_DEFINITION`
`indexsearch.COLLECTION_VECTOR_INDEX`	This collection contains vectorized data for semantic and similarity searches.	`VECTORGROUP_DEFINITION`

warning

Always pass the collection explicitly in new code. If the collection is omitted, field mappings, filters, or indexer jobs can run against the wrong index and produce unexpected results or errors.

Searching the Standard Index

The standard index is intended for full-text searches, field searches, filters, and group searches. If no search pattern is set, the query is treated as *:* and searches all documents in the selected groups.

warning

The standard index does not support vector-specific query settings such as setVector(...) or setTopK(...). Using these settings with indexsearch.COLLECTION_SEARCH_INDEX results in an error.

import { indexsearch, logging } from "@aditosoftware/jdito-types";

// Restrict the search to person records and return only the fields needed by
// the caller.
const query = indexsearch.createIndexQuery()
    .setPattern("Lisa Sommer")
    .setEntities("Person_entity")
    .setSearchFields("Person_entity.FIRSTNAME", "Person_entity.LASTNAME")
    .setResultIndexFields(
        indexsearch.FIELD_ID,
        indexsearch.FIELD_TITLE,
        indexsearch.FIELD_DESCRIPTION
    )
    .setRows(10)
    .setApplyTrailingWildcards(true);

const result = indexsearch.searchIndex(
    query,
    indexsearch.COLLECTION_SEARCH_INDEX
);

const hits = result[indexsearch.HITS] || [];
logging.log(JSON.stringify(hits));

The following settings are commonly used for standard index queries.

Setting	Usage
`setPattern(pattern)`	This method sets the main search pattern. If no pattern is set, ADITO uses `:`.
`setIndexGroups(...groups)`	This method restricts the search directly to index groups, for example `"Organisation"`.
`setEntities(...entities)`	This method restricts the search by entity. The index group is resolved when the search is executed.
`setSearchFields(...)`	This method searches in entity fields, for example `"Person_entity.LASTNAME"`.
`setSearchIndexFields(...)`	This method searches directly in technical index fields.
`setResultFields(...)`	This method returns entity fields in the result.
`setResultIndexFields(...)`	This method returns technical index fields in the result.
`setStart(...)` and `setRows(...)`	These methods configure paging. If `rows` is not set, ADITO uses the project preference `indexsearchDefaultResultCount`.
`addFilter(pattern)`	This method adds a filter that operates on the result of the main pattern or on the previous filter.
`setUser(user)`	This method executes the search from the perspective of the specified user.
`setDisablePermissions(true)`	This method disables permission checks for special cases, such as technical admin processes.

Fields, Groups, and Mappings

With setEntities(...), setSearchFields(...), and setResultFields(...), entity names and entity fields are translated into technical index groups and index fields. This translation happens when searchIndex(...) is called. The search fails if no matching active index group can be found for a specified entity, or if a field cannot be mapped to an index field.

Entity fields must start with the defining entity, for example "Organisation_entity.NAME". You can optionally distinguish between value and displayValue, for example "Organisation_entity.NAME.displayValue". Without a suffix, ADITO uses value.

Resolve technical index fields with lookupIndexField(...) when you need the native field name.

import { indexsearch } from "@aditosoftware/jdito-types";

const nameField = indexsearch.lookupIndexField(
    "Organisation_entity",
    "NAME",
    indexsearch.COLLECTION_SEARCH_INDEX
);

lookupIndexField(...) returns null if index search is disabled or if a legacy configuration is used.

Search Patterns and Filters

For programmatically built search patterns, use the pattern builder and build the pattern in the collection context in which it will be used.

import { indexsearch } from "@aditosoftware/jdito-types";

const patternConfig = indexsearch.createPatternConfig()
    .and(
        indexsearch.createTerm("ADITO")
            .setEntityField("Organisation_entity.NAME")
    );

const filterPattern = indexsearch.buildPattern(
    patternConfig,
    indexsearch.COLLECTION_SEARCH_INDEX
);

const query = indexsearch.createIndexQuery()
    .setIndexGroups("Organisation")
    .addFilter(filterPattern);

If an ADITO filter definition is already available as JSON, use indexsearch.convertFilterToPattern(...) instead of manually concatenating Lucene syntax. The method also requires the correct entity or index group context so that filter extensions can be resolved.

Unknown filter extensions are handled according to the selected policy.

Policy	Behavior
`FILTER_EXTENSION_SYSTEM_DEFAULT`	This policy uses the global setting, which defaults to `FAIL`.
`FILTER_EXTENSION_FAIL`	This policy aborts the conversion with an exception.
`FILTER_EXTENSION_SKIP`	This policy skips unknown extensions.

When working with manually entered search terms, keep the following behavior in mind.

setDefaultOperator(indexsearch.OPERATOR_OR) is the implicit default for the main pattern. OPERATOR_AND makes every term mandatory.
The default operator only affects the main pattern. Filter patterns use OR internally.
setApplyTrailingWildcards(true) can turn foo bar into foo* bar*.
setApplyLeadingWildcards(true) creates leading wildcards such as *foo. This can be expensive and should be used deliberately.
Raw wildcard characters in user-provided search strings are treated as normal characters unless wildcard behavior is added deliberately through the query configuration. Do not rely on input such as *term or term? to express partial matching.
setAutomaticEscape(true) can escape syntax characters. Do not apply it after generating a pattern with a pattern builder or converted filter JSON, because it may modify an already valid generated pattern.

Result Structure

The result of searchIndex(...) is a map. Access result values via constants.

const totalHits = result[indexsearch.TOTALHITS];
const groups = result[indexsearch.GROUPS] || [];
const hits = result[indexsearch.HITS] || [];

The following constants are commonly used when reading the result.

Constant	Meaning
`TOTALHITS`	This value contains the total number of hits.
`HITS`	This value contains the returned hit list.
`GROUPS` and `TOTALGROUPS`	These values contain hit counts per index group.
`SUBGROUPS`	This value contains hit counts per subgroup.
`TAGS`	This value contains global search tags if tags are enabled.
`FIELD_ID`	This field contains the technical hit ID.
`FIELD_TITLE`	This field contains the result title.
`FIELD_DESCRIPTION`	This field contains the result description.
`FIELD_TAGS`	This field contains the tags on the hit.
`FIELD_SCORE`	This field contains the ranking score if it was requested and is available.

Vector Search with `COLLECTION_VECTOR_INDEX`

Vector search uses the same IndexQuery builder as standard search, but the query must be executed against indexsearch.COLLECTION_VECTOR_INDEX. This collection contains vectorized documents from index groups configured with VECTORGROUP_DEFINITION in an IndexRecordContainer.

warning

The vector index does not support setPattern(...), setExcludedTags(...), or setIncludedTags(...). Use addFilter(...) if a vector search must be restricted by additional criteria.

import { indexsearch } from "@aditosoftware/jdito-types";

// The configured KNN_VECTOR field expects exactly 1024 numeric values.
const queryVector = JSON.stringify([/*... <1024 dimensional vector> ...*/]);

const query = indexsearch.createIndexQuery()
    .setVector(queryVector)
    .setTopK(10)
    .setRows(10)
    .setIndexGroups("Organisation")
    .setResultIndexFields(
        indexsearch.FIELD_ID,
        indexsearch.FIELD_TITLE,
        indexsearch.FIELD_DESCRIPTION,
        indexsearch.FIELD_SCORE
    );

const result = indexsearch.searchIndex(
    query,
    indexsearch.COLLECTION_VECTOR_INDEX
);

The following settings are specific to vector search.

Topic	Consideration
Collection	Always pass `indexsearch.COLLECTION_VECTOR_INDEX` so that the query runs against the vector index.
Vector format	`setVector(...)` expects a string in JSON array format. The array must match the configured vector dimension.
`setTopK(k)`	This setting defines how many nearest vectors are considered by the k-nearest-neighbor search. The default value is `32`.
`setRows(n)`	This setting defines how many documents are returned. In practice, `rows` should not be greater than `topK`.
Groups	`setIndexGroups(...)` refers to vector groups, not to standard-index groups.
Field mapping	`lookupIndexField(...)`, `buildPattern(...)`, `toFilterPattern(...)`, and `convertFilterToPattern(...)` must be called with `indexsearch.COLLECTION_VECTOR_INDEX` when they are used for vector filters.
Filters	`addFilter(...)` can further restrict the vector result. The filter pattern must be valid for the vector index.
Result fields	Request `indexsearch.FIELD_VECTOR` and `indexsearch.FIELD_SCORE` only if those fields are stored and required for the use case.

A typical semantic search consists of the following steps.

Convert the input text into an embedding vector.
Build an IndexQuery with setVector(...), setTopK(...), the vector index groups, and the required result fields.
Execute searchIndex(query, indexsearch.COLLECTION_VECTOR_INDEX).

Building Vector Filters Correctly

If a vector search must be restricted by additional filters, build the filter in the context of the vector collection.

import { indexsearch } from "@aditosoftware/jdito-types";

const queryVector = JSON.stringify([/*... <1024 dimensional vector> ...*/]);

const activeOrganisationFilter = indexsearch.buildPattern(
    indexsearch.createPatternConfig()
        .and(
            indexsearch.createTerm("ACTIVE")
                .setEntityField("Organisation_entity.STATUS")
        ),
    indexsearch.COLLECTION_VECTOR_INDEX
);

const query = indexsearch.createIndexQuery()
    .setVector(queryVector)
    .setTopK(20)
    .setRows(10)
    .setIndexGroups("Organisation")
    .addFilter(activeOrganisationFilter);

const result = indexsearch.searchIndex(
    query,
    indexsearch.COLLECTION_VECTOR_INDEX
);

warning

If a filter is built with indexsearch.COLLECTION_SEARCH_INDEX, or without a collection, field names and filter extensions can be mapped to the standard index. The vector collection has its own record containers and field definitions, so the resulting filter can fail or return unexpected results.

User and Permission Options

Server-Side User Impersonation

When a search runs in the server context, the request can impersonate a user. ADITO then executes the search with the user's permissions and processes pattern extensions in the user's context.

Set the user title with setUser(user).

const query = indexsearch.createIndexQuery()
    .setUser("search.service");

warning

User impersonation is only supported in server-side processes. If the process runs in the client or session context, the search fails.

Disabling Permissions

Use setDisablePermissions(true) to disable permission checks for a search request. If this setting is not specified, permissions are enabled.

const query = indexsearch.createIndexQuery()
    .setDisablePermissions(true);

Disabling Pattern Extensions

Use setDisablePatternExtension(true) to disable pattern extensions for a search request. If this setting is not specified, pattern extensions are enabled.

const query = indexsearch.createIndexQuery()
    .setDisablePatternExtension(true);

Updating the Index

Both the standard index and the vector index can be addressed through the collection parameter.

Run the Indexer

Use runIndexer(...) to run a full rebuild or a group-specific rebuild.

import { indexsearch } from "@aditosoftware/jdito-types";

indexsearch.runIndexer(null, indexsearch.COLLECTION_SEARCH_INDEX);
indexsearch.runIndexer(["Organisation"], indexsearch.COLLECTION_VECTOR_INDEX);

Run the Incremental Indexer

Use runIncrementalIndexer(...) to rebuild individual records.

import { indexsearch } from "@aditosoftware/jdito-types";

const config = indexsearch.createIncrementalIndexerRunConfig()
    .group("Organisation")
    .addUid("46d9a82e-3b7b-46e0-a5e0-94c985f93bb8");

indexsearch.runIncrementalIndexer(
    config,
    indexsearch.COLLECTION_VECTOR_INDEX
);

Import and Basic Concept​

Searching the Standard Index​

Fields, Groups, and Mappings​

Search Patterns and Filters​

Result Structure​

Vector Search with COLLECTION_VECTOR_INDEX​

Building Vector Filters Correctly​

User and Permission Options​

Server-Side User Impersonation​

Disabling Permissions​

Disabling Pattern Extensions​

Updating the Index​

Run the Indexer​

Run the Incremental Indexer​

Import and Basic Concept

Searching the Standard Index

Fields, Groups, and Mappings

Search Patterns and Filters

Result Structure

Vector Search with `COLLECTION_VECTOR_INDEX`

Building Vector Filters Correctly

User and Permission Options

Server-Side User Impersonation

Disabling Permissions

Disabling Pattern Extensions

Updating the Index

Run the Indexer

Run the Incremental Indexer