Migrating from sphinx to manticore real-time index#

This documentation applies to Piler enterprise edition 1.7.2

Revision #1

Publication date: May 5, 2023

Introduction#

This document describes to how migrate sphinx index data to manticore real-time index on Ubuntu Linux.

Upgrade to 1.7.2+#

Follow the upgrade docs for the details

Uninstall sphinxsearch#

systemctl stop pilersearch
rm -f /usr/bin/index* /usr/bin/searchd /usr/bin/wordbreaker

Install manticore#

wget https://repo.manticoresearch.com/manticore-repo.noarch.deb
dpkg -i manticore-repo.noarch.deb
apt-get update
apt-get install -y manticore manticore-columnar-lib manticore-extra
systemctl stop manticore
systemctl disable manticore
touch /etc/piler/MANTICORE

Create the default tenant in manticore#

mkdir /var/piler/manticore/piler
chown piler:piler /var/piler/manticore/piler

For multi-tenant installations: create the tenant dirs in manticore, eg.

mkdir /var/piler/manticore/customer1
chown piler:piler /var/piler/manticore/customer1
...

Fix config-site.php#

Add the following to /etc/piler/config-site.php

$config['RT_INDEX'] = 1;

Start pilersearch service#

systemctl start pilersearch

Verify the following files in /var/piler/manticore/piler#

-rw------- 1 piler piler    0 May  5 12:25 main.lock
-rw------- 1 piler piler  962 May  5 12:25 main.meta
-rw------- 1 piler piler   68 May  5 12:25 main.ram
-rw------- 1 piler piler    0 May  5 12:25 note1.lock
-rw------- 1 piler piler  648 May  5 12:25 note1.meta
-rw------- 1 piler piler   20 May  5 12:25 note1.ram
-rw------- 1 piler piler    0 May  5 12:25 tag1.lock
-rw------- 1 piler piler  647 May  5 12:25 tag1.meta
-rw------- 1 piler piler   20 May  5 12:25 tag1.ram

For multi-tenant installations: check files in the tenant dirs, eg. /var/piler/manticore/customer1

Fix piler.conf#

Edit /etc/piler/piler.conf, and add (or set) the following:

rtindex=1

Restart the piler service#

systemctl restart piler

Reindex emails#

cd /var/piler/imap
reindex -a -p

For multi-tenant deployments: reindex for all tenants

Remove the indexer jobs in piler's crontab#

When using real-time (rt) index, there's no need to run the period indexer cron jobs.

sed -i 's/.*indexer.*\.sh/###/g' /var/spool/cron/crontabs/piler

Remove sphinx data#

rm -rf /var/piler/sphinx/