Index rotation is a procedure in which the searchd server looks upon new versions of defined indexes in the configuration. Rotation is subject only to Plain mode of operation.
There can be two cases:
In the first case, indexer cannot put the new version of the index
online as the running copy is locked and loaded by searchd.
In this case indexer needs to be called with –rotate
parameter. If rotate is used, indexes creates new index files with
.new. in their name and sends a HUP signal to
searchd informing it about the new version. The
searchd will perform a lookup and will put in place the new
version of the index and discard the old one. In some cases it might be
desired to create the new version of the index but not perform the
rotate as soon as possible. For example it might be desired to check
first the health of the new index versions. In this case,
indexer can accept --nohup parameter which
will forbid sending the HUP signal to the server.
New indexes can be loaded by rotation, however the regular handling
of HUP signal is to check for new indexes only if configuration has
changed since server startup. If the index was already defined in the
configuration, the index should be first created by running
indexer without rotation and perform RELOAD
INDEXES statement instead.
There are also two specialized statements can be used to perform rotations on indexes:
RELOAD INDEX idx [ FROM '/path/to/index_files' ];RELOAD INDEX allows you to rotate indexes using SQL.
It has two modes of operation. First one (without specifying a path) makes Manticore server check for new index files in directory specified in path. New index files must have a idx.new.sp? names.
And if you additionally specify a path, server will look for index files in specified directory, move them to index path, rename from index_files.sp? to idx.new.sp? and rotate them.
mysql> RELOAD INDEX plain_index;
mysql> RELOAD INDEX plain_index FROM '/home/mighty/new_index_files';RELOAD INDEXES;Works same as system HUP signal. Initiates index rotation. Unlike
regular HUP signalling (which can come from kill or indexer
), the statement forces lookup on possible indexes to rotate even if the
configuration has no changes since the startup of the server.
Depending on the value of seamless_rotate setting, new queries might be shortly stalled; clients will receive temporary errors. Command is non-blocking (i.e., returns immediately).
mysql> RELOAD INDEXES;
Query OK, 0 rows affected (0.01 sec)The rotate assumes old index version is discarded and new index version is loaded and replace the existing one. During this swapping, the server needs also to serve incoming queries made on the index that is going to be updated. To not have stalls of the queries, the server implements by default a seamless rotate of the index as described below.
Indexes may contain some data that needs to be precached in RAM. At
the moment, .spa, .spb, .spi and
.spm files are fully precached (they contain attribute
data, blob attribute data, keyword index and killed row map,
respectively.) Without seamless rotate, rotating an index tries to use
as little RAM as possible and works as follows:
searchd waits for all currently running queries to
finish;searchd resumes serving queries from new index.However, if there’s a lot of attribute or dictionary data, then preloading step could take noticeable time - up to several minutes in case of preloading 1-5+ GB files.
With seamless rotate enabled, rotation works as follows:
Seamless rotate comes at the cost of higher peak
memory usage during the rotation (because both old and new copies of
.spa/.spb/.spi/.spm data need to be in RAM while preloading
new copy). Average usage stays the same.
Example:
seamless_rotate = 1