Plain and real-time index settings

Defining index schema in a configuration file

index <index_name>[:<parent index name>] {
...
}

Example of a plain index in a configuration file

index <index_name> {
  type = plain
  path = /path/to/index
  source = <source_name>
  source = <another source_name>
  [stored_fields = <comma separated list of full-text fields that should be stored>]
}

Example of a real-time index in a configuration file

index <index name> {
  type = rt
  path = /path/to/index

  rt_field = <full-text field name>
  rt_field = <another full-text field name>
  [rt_attr_uint = <integer field name>]
  [rt_attr_uint = <another integer field name, limit by N bits>:N]
  [rt_attr_bigint = <bigint field name>]
  [rt_attr_bigint = <another bigint field name>]
  [rt_attr_multi = <multi-integer (MVA) field name>]
  [rt_attr_multi = <another multi-integer (MVA) field name>]
  [rt_attr_multi_64 = <multi-bigint (MVA) field name>]
  [rt_attr_multi_64 = <another multi-bigint (MVA) field name>]
  [rt_attr_float = <float field name>]
  [rt_attr_float = <another float field name>]
  [rt_attr_bool = <boolean field name>]
  [rt_attr_bool = <another boolean field name>]
  [rt_attr_string = <string field name>]
  [rt_attr_string = <another string field name>]
  [rt_attr_json = <json field name>]
  [rt_attr_json = <another json field name>]
  [rt_attr_timestamp = <timestamp field name>]
  [rt_attr_timestamp = <another timestamp field name>]

  [stored_fields = <comma separated list of full-text fields that should be stored>]

  [rt_mem_limit = <RAM chunk max size, default 128M>]

}

Common plain and real-time indexes settings

type

type = plain

type = rt

Index type: “plain” or “rt” (real-time)

Value: plain (default), rt

path

path = path/to/index

Absolute or relative path without extension where to store the index or where to look for it

Value: path to the index, mandatory

stored_fields

stored_fields = title, content

By default when an index is defined in a configuration file, full-text fields’ original content is not stored, but just indexed. If this option is set, values from the fields will be both indexed and stored.

Value: comma separated list of full-text fields that should be stored. Default is empty (i.e. does not store original field text) for Plain mode, but is enabled for every field for RT mode as long as it’s declared as just text.

Note, in case of a real-time index the fields listed in stored_only_fields should be also declared as rt_field.

Note also, that you don’t need to list attributes in stored_fields, since their original values are stored anyway. stored_fields is only for full-text fields.

See also docstore_block_size, docstore_compression for document storage compression options.

SQL:

CREATE TABLE products(title text stored indexed, content text stored indexed, name text indexed, price float)

POST /cli -d "
CREATE TABLE products(title text stored indexed, content text stored indexed, name text indexed, price float)"

$params = [
    'body' => [
        'columns' => [
            'title'=>['type'=>'text', 'options' => ['indexed', 'stored']],
            'content'=>['type'=>'text', 'options' => ['indexed', 'stored']],
            'name'=>['type'=>'text', 'options' => ['indexed']],
            'price'=>['type'=>'float']
        ]
    ],
    'index' => 'products'
];
$index = new \Manticoresearch\Index($client);
$index->create($params);

Python:

utilsApi.sql('mode=raw&query=CREATE TABLE products(title text stored indexed, content text stored indexed, name text indexed, price float)')

Javascript:

res = await utilsApi.sql('mode=raw&query=CREATE TABLE products(title text stored indexed, content text stored indexed, name text indexed, price float)');

Java:

utilsApi.sql("mode=raw&query=CREATE TABLE products(title text stored indexed, content text stored indexed, name text indexed, price float)");

index products {
  stored_fields = title,content

  type = rt
  path = idx
  rt_field = title
  rt_field = content
  rt_field = name
  rt_attr_uint = price
}

stored_only_fields

stored_only_fields = title,content

A list of fields that will be stored in the index but will be not indexed. Similar to stored_fields except when a field is specified in stored_only_fields it is only stored, not indexed and can’t be searched with fulltext queries. It can only be returned with search results.

Value: comma separated list of fields that should be stored only, not indexed. Default is empty. Note, in case of a real-time index the fields listed in stored_only_fields should be also declared as rt_field.

Note also, that you don’t need to list attributes in stored_only_fields, since their original values are stored anyway. If to compare stored_only_fields to string attributes the former (stored field): * is stored on disk and doesn’t require memory * is stored compressed * can be only fetched, you can’t sort/filter/group by the value

The latter (string attribute) is: * stored on disk and in memory * stored uncompressed * can be used for sorting, grouping, filtering and anything else you want to do with attributes.

Real-time index settings:

rt_field

rt_field = subject

Full-text fields to be indexed. The names must be unique. The order is preserved; and so field values in INSERT statements without an explicit list of inserted columns will have to be in the same order as configured.

Value: at least one full-text field should be specified in an index, multiple records allowed.

rt_attr_uint

rt_attr_uint = gid

Unsigned integer attribute declaration

Value: field_name or field_name:N, can be multiple records. N is the max number of bits to keep.

rt_attr_bigint

rt_attr_bigint = gid

BIGINT attribute declaration

Value: field name, multiple records allowed

rt_attr_multi

rt_attr_multi = tags

Multi-valued attribute (MVA) declaration. Declares the UNSIGNED INTEGER (unsigned 32-bit) MVA attribute. Multi-value (ie. there may be more than one such attribute declared), optional.