Optimized Record Field Loading

A RecordContainer usually does't load the values of every declared field when its records are loaded. For improved performance, it tries to retrieve only the data actually needed for a request or the current UI. This chapter explains:

• How a dbRecordContainer optimizes its query
• How to use the $local.fieldhints variable in a JDitoRecordContainer to optimize field loading in a contentProcess
• The conditions for an EntityField to be considered "required"

dbRecordContainer

A dbRecordContainer automatically limits the database columns and SQL expressions in the SELECT statement to the fields that are required for the current situation. For example, a recordFieldMapping may define an expression for a calculated column; that expression is evaluated only when the mapped EntityField is requested.

JDitoRecordContainer

In a JDitoRecordContainer the contentProcess is responsible for returning the records. The system provides information about which fields are actually needed in the JDito variable $local.fieldhints. You can use these field hints to omit fields that require expensive processing.

The value of $local.fieldhints includes the names of the required record fields in the following structure:

{
    recordFields: ["UID.value", "ORGANISATION_NAME.value", "EMAIL.displayValue"],
    aggregateFields: [...]
}

This example shows how the field hints can be used to skip the processing of one field in a contentProcess:

let contactData = loadContacts();

let fieldHints = vars.get("$local.fieldhints");
let includeOrgName = fieldHints.recordFields.includes("ORGANISATION_NAME.value");

let records = contactData.map(contact =>
{
    let organisationName = "";
    if (includeOrgName) 
    {
        organisationName = getOrgName(contact.orgId);
    }
    return [
        contact.id,
        contact.firstName,
        contact.lastName,
        organisationName
    ];
});

result.object(records);

How the RecordContainer decides which fields are required

An EntityField can be considered "required" for two reasons:

1. Direct request: the field is included in the current View, REST call, or JDito request.
2. Dependency: a required field's JDito process references another field (for example, a valueProcess reads $field.FOO), so that other field must also be loaded.

How dependencies are discovered

The loading of records from the RecordContainer happens before individial EntityField JDito processes (valueProcess etc) are executed. Because of this, dependencies on RecordContainer fields that arise from JDito code must be discoverable without executing the code. The system performs a static analysis over the processes of the EntityFields to find $field references and mark those fields as required. It also walks through chains of field references, so if a referenced field depends on another field, that nested dependency is picked up, too.

Important limitations of that approach:

• Only processes declared directly in the Entity (for example valueProcess/displayValueProcess) are scanned. Imported libraries are not scanned.
• The static analysis looks for literal $field.FIELDNAME occurrences. Indirect or computed uses are not reliably detected.

Examples:

Will be detected:

let name = vars.get("$field.FIRSTNAME");
let title = vars.get("$field.TITLE.displayValue");
let addressFieldTitle = vars.get("$property.ADDRESS.title");

Not detected:

// Using a library — the analyzer will not inspect the source of MyUtility
let name = MyUtility.getFieldValue("FIRSTNAME");

// Computed name — analyzer cannot resolve dynamic string
let field = "$field." + fieldName;
let val = vars.get(field);

Recommendation: reference required RecordContainer fields directly in the Entity's processes (not inside libraries), so the static analysis can detect them.

info

This static-analysis approach only applies to fields that the RecordContainer loads. Fields that compute values entirely in valueProcess (and do not rely on RecordContainer-provided values) can still be evaluated dynamically. See Understanding Variable Dependencies for more detail.