NativeSearchContrib

Accelerated search for use with mod_perl

By default the RCS-based store in Foswiki searches content by forking the standard grep program in a sub-process to search through the text files in the database cache. This is usually pretty fast, but unfortunately in some environments (notably when running with Apache mod_perl) forking a sub-process is very expensive, and another approach is needed.

NativeSearchContrib contains a compiled code module that can be plugged into Foswiki to replace the default searching algorithm. It publishes an XS interface that allows it to be called directly from Perl, without having to fork a sub-process. It has comparable performance to the standard forking search in most environments, but is often significantly faster than forked search when Foswiki is run under a CGI accelerator. The effect is most pronounced with mod_perl, but is also there for SpeedyCGI.

You require shell access to the server to install this module.

Tested with Foswiki 1.0.9 and 1.1

Installation Instructions

Before you start, there are some prerequisites.

1. you must have ExtUtils::MakeMaker installed (available from CPAN)
2. you must have a C compiler (gcc has been tested). Most Unix/Linux systems will have this already. On Win32 MinGW is recommended.
3. you will need make (a standard development tool). Unix/Linux users will probably have this already; others may have to search. You are recommended to use nmake on Win32.
4. finally you will need the pcre library (required for perl compatible regular expressions). This is a standard Linux library and will be available in an RPM or Debian package or whatever for your system. It is also available from GnuWin32 for Windows. You need both the runtime library (binary) and also the development header files, which are usually shipped in a seperate package.

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server.

Open configure, and open the "Extensions" section. Use "Find More Extensions" to get a list of available extensions. Select "Install".

If you have any problems, or if the extension isn't available in configure, then you can still install manually from the command-line. See http://foswiki.org/Support/ManuallyInstallingExtensions for more help.

Building the c-code

cd to the root directory in your Foswiki installation. Now, assuming you have administrator (root) access to the server:

$ cd tools/native_search
$ perl Makefile.PL
$ make install
$ perl test.pl -i -l FoswikiNativeSearch test.pl Makefile.PL FoswikiNativeSearch.xs

(on Win32 you should use nmake)

Make sure that perl test.pl returns something sane, and NOT a Segmentation fault. (test.pl is simply a stub interface to the native search, so it behaves like grep. The test above simply greps for the string "NativeSearch" in the files Makefile.PL and FoswikiNativeSearch.xs)

If you do not have administrator (root) access to the server, you can still install. Follow the steps above, but use perl Makefile.PL INSTALLBASE=/your/dir to install to a different directory. Exactly where you install depends on your system configuration.

Now, in configure, in the "Store" section, you should be able to select the search algorithm Foswiki::Store::SearchAlgorithms::Native.

Run a Foswiki search, and watch the web server error log for any problems.

Because performance can differ widely between two apparently similar webserver configurations, you are recommended to do your own benchmarking and select the search algorithm that performs best for you.

You may also want to read up on other search engines such as Foswiki:Extensions.KinoSearchContrib, which may address your requirements better.

Info

Another great Foswiki extension from the WikiRing - working together to improve your wiki experience!

Many thanks to the following sponsors for supporting this work:

• Wind River Systems http://windriver.com

Author(s):	Crawford Currie http://c-dot.co.uk
Copyright:	© Crawford Currie 2007-2010
License:	GPL (Gnu General Public License)
Version:	11235 (2011-03-26)
Release:	1.4.1
Change History:
1.4.1	Foswiki:Tasks/Item10531: duplicate search results fixed
1.4	Support for trunk
1.3	Restored missing 1.0 support :-8
1.2	Changes for Foswiki 1.1
1.1	Initial release
Dependencies:	Name Version Description ExtUtils::MakeMaker >=0 Required for building makefiles
Contrib Home:	http://foswiki.org/Extensions/NativeSearchContrib