SWISH-RUN - Running Swish-e and Command Line Switches
The Swish-e program is controlled by command line arguments (called
switches). Often, it is run manually from a shell (command prompt), or
from a program such as a CGI script that passes the command line arguments to
swish.
Note: A number of the command line switches may be specified in the Swish-e
configuration file specified with the "-c" command line argument.
Please see SWISH-CONFIG for a complete description of available configuration
file directives.
There are two basic operating modes of Swish-e: indexing and searching. There
are command line arguments that are unique to each mode, and others that apply
to both (yet may have different meaning depending on the operating mode).
These command line arguments are listed below, grouped by:
INDEXING -- describes the command line arguments used while indexing.
SEARCHING -- lists the command line arguments used while searching.
OTHER SWITCHES -- lists switches that don't apply to searching or indexing.
Beginning with Swish-e version 2.1, you may embed its search engine into your
applications. Please see SWISH-LIBRARY.
Swish-e indexing is initiated by passing
command line arguments to swish.
The command line arguments used for
searching are described in
SEARCHING. Also, see SWISH-SEARCH for examples of searching with Swish-e.
Swish-e usage:
swish-e [-i dir file ... ] [-c file] [-f file] [-l] \
[-v (num)] [-S method(fs⎪http⎪prog)] [-N path]
The "-h" switch (help) will list the available Swish-e command line
arguments:
swish-e -h
Typically, most if not all indexing settings are placed in a configuration file
(specified with the "-c" switch). Once the configuration file is
setup indexing is initiated as:
swish-e -c /path/to/config/file
See SWISH-CONFIG for information on the configuration file.
Security Note: If the swish binary is named
swish-search then swish will
not allow any operation that would cause swish to write to the index file.
When indexing it may be advisable to index to a temporary file, and then after
indexing has successfully completed rename the file to the final location.
This is especially important when replacing an index that is currently in use.
swish-e -c swish.config -f index.tmp
[check return code from swish or look for err: output]
mv index.tmp index.swish-e
Indexing Command Line Arguments
- -i *directories and/or files* (input file)
- This specifies the directories and/or files to index.
Directories will be indexed recursively. This is typically specified in
the configuration file with the IndexDir directive instead of on
the command line. Use of this switch overrides the configuration file
settings.
- -S [fs⎪http⎪prog] (document source/access
mode)
- This specifies the method to use for accessing documents to
index. Can be either "fs" for local indexing via the file system
(the default), "http" for spidering, or "prog" for
reading documents from an external program.
Located in the "conf" directory are example configuration files
that demonstrate indexing with the different document source methods.
See the SWISH-FAQ for a discussion on the different indexing methods, and
the difference between spidering with the http method vs. using the file
system method.
- fs - file system
- The "fs" method simply reads files from a local
(or networked) drive. This is the default method if the "-S"
switch is not specified. See SWISH-CONFIG for configuration directives
specific to the "fs" method.
- http - spider a web server
- The "http" method is used to spider web servers.
It uses an included helper program called swishspider. See
SWISH-CONFIG for configuration directives specific to the "http"
method.
Security Note: Under Windows swish passes the URLs fetched from remote
documents through the shell (swish uses the system() command for
running swishspider under Windows), and this may be considered an
additional security risk.
The "http" method is deprecated (or at least not very well
appreciated). Consider using the "prog" method described below
for spidering. There's a spider program available in the prog-bin
directory for use with the "prog" method. Here's a number of
limitation with this method that are solved with the "prog"
method:
- *
- swishspider only spiders standard <a
href="..."> links. Frames and other links are not
followed.
- *
- By default, this method of spidering only indexes files
that have a content type of "text/*" (e.g. text/plain,
text/html, text/xml). You should use "DefaultContents" and
"IndexContents" to map file extensions to parsers used by swish
(e.g. "IndexContents HTML* .html .htm"), but this will fail
where a document does not have a file extension.
- *
- Swish-e's "FileFilter" directive can be used with
the "http" access method, although it requires a separate
process (in addition to the swsihspider process) for each document
filtered.
- *
- The SWISH::Filter modules can be used with the swishspider
program. SWISH::Filter provides a general purpose filtering system (see
SWISH::Filter documentation). To use SWISH::Filter set PERL5LIB to point
to the location of the SWISH module name space (typically
/usr/local/lib/swish-e under Unix). For example:
export PERL5LIB=/usr/local/lib/swish-e # bash, bourne shells
setenv PERL5LIB /usr/local/lib/swish-e # csh, tcsh
or under Windows
set PERL5LIB=c:\program files\swish-e2.4\lib\swish-e
SWISH::Filter is not enabled by default due to the overhead of loading the
modules for every document fetched.
The Swish-e distribution includes perl modules in the SWISH::Filters::*
namespace to make converting non-text documents into a format that Swish-e
can parse easy. As mentioned above, the helper script swishspider
will use these modules if can be found via PERL5LIB. These modules only
provide an interface to programs that do the conversion. For example, you
will need to download and install the "catdoc" program to
convert MSWord documents into text for indexing. Please see
filters/README to see how to use this filter system.
- prog - general purpose access method
- The "prog" method is new to Swish-e version 2.2.
It's designed as a general purpose method to feed documents to swish from
an external program.
For example, the external program can read a database (e.g. MySQL), spider a
web server, or convert documents from one format to another (e.g. pdf to
html). Or, you can simply use it to read the files of the file system
(like "-S fs"), yet provide you with full control of what files
are indexed.
The external program name to run is passed to swish either by the IndexDir
directive, or via the "-i" option.
The program specified should be an absolute path as swish-e will attempt to
stat() the program to make sure it exists. Swish does this to help
in error reporting.
If the program specified with -i or IndexDir is not an absolute path (i.e.
does not include "/" ) then swish-e will append the
"libexecdir" directory defined during configuration. Typically,
libexecdir is set to "$prefix/lib/swish-e"
(/usr/local/lib/swish-e), but is platform and installation dependent.
Running swish-e -h will report the directory.
For example, the -S prog program "spider.pl" is a Perl helper
program for use with -S prog and is installed in libexecdir.
IndexDir spider.pl
SwishProgParameters default http://localhost/index.html
and swish-e will find spider.pl in libexecdir.
Additional parameters may be passed to the external program via the
SwishProgParameters directive. In the example above swish-e will pass two
parameters to spider.pl, "default" and
"http://localhost/index.html".
A special name "stdin" may be used with "-i" or IndexDir
which tells swish to read from standard input instead of from an external
program. See example below.
The external program prints to standard output (which swish captures) a set
of headers followed by the content of the file to index. The output looks
similar to an email message or a HTTP document returned by a web server in
that it includes name/value pairs of headers, a blank line, and the
content.
The content length is determined by a content-length header supplied to
swish by the program; there is no "end of record" character or
flag sent between documents. Therefore, it is critical that the
content-length header is correct. This is a common source of errors.
One advantage of this method (over using filters, for example) is that the
external program is run only once for the entire indexing job, instead of
once for every document. This avoids forking and creating a new process
for every document, and makes a huge difference when your external program
is something like perl that has a large startup cost.
Here's a simple example written in Perl:
#!/usr/local/bin/perl -w
use strict;
# Build a document
my $doc = <<EOF;
<html>
<head>
<title>Document Title</title>
</head>
<body>
This is the text.
</body>
</html>
EOF
# Prepare the headers for swish
my $path = 'Example.file';
my $size = length $doc;
my $mtime = time;
# Output the document (to swish)
print <<EOF;
Path-Name: $path
Content-Length: $size
Last-Mtime: $mtime
Document-Type: HTML*
EOF
print $doc;
The external program passes to swish a header. The header is separated from
the body of the document with a blank line. The available headers
are:
- Path-Name:
- This is the name of the file you are indexing. This can be
any string, so for example it could be an ID of a record in a database, a
URL or a simple file name.
This header is required.
- Content-Length:
- This header specifies the length in bytes of the document
that follows the header. This length must be exactly the length of the
document -- do not make the mistake of adding an extra line feed at the
end of the document.
This header is required.
- Last-Mtime:
- Thi parameter is the last modification time of the file,
and must be a time stamp (seconds since the Epoch on your platform).
This header is not required.
- Document-Type:
- You may override swish's determination of document type
("Indexcontents") by using the "Document-Type:"
header. The document type is used to select which parser Swish-e uses to
parse the document's contents.
For example, a spider program might map the content-type returned from a web
server to one of the types Swish-e understands. For example,
my $doc_type = 'HTML*' if $response->content_type =~ m!text/html!'
This header is not required.
- Update-Mode:
- When updating an incremental index this header can be used
to select the mode for updating the index. There are three possible
values:
Update
Remove
Index
"Update" will update the index with the given file if the date of
the given file is newer than the date of the file already in the index.
Setting to "Update" is the same as using -u on the command line.
"Remove" mode will remove the file specified by the Path-Name
header. Setting "Remove" is the same as using -r on the command
line.
"Index" will add the file to the index. NOTE: swish-e will not
check to see if the file already exists.
If this header is not specified, the default is the mode specified on the
command line (-u, -r, or none).
This option is still experimental and is subject to change in the future.
Ask on the Swish-e list before using.
The above example program only returns one document and exits, which is not very
useful. Normally, your program would read data from some source, such as files
or a database, format as XML, HTML, or text, and pass them to swish, one after
another. The "Content-Length:" header tells swish where each
document ends -- there is not any special "end of record" character
or marker.
To index with the above example you need to make sure that the program is
executable (and that the path to perl is correct), and then call swish telling
to run in "prog" mode, and the name of the program to use for input.
% chmod 755 example.pl
% ./swish-e -S prog -i ./example.pl
Programs can and should be tested prior to running swish. For example:
% ./example.pl > test.out
A few more useful example programs are provided in the swish-e distribution
located in the
prog-bin directory. Some include documentation:
% cd prog-bin
% perldoc spider.pl
Others are small examples that include comments:
% cd prog-bin
% less DirTree.pl
The
spider.pl program can be used as a replacement for the
-S http
method. It is far more feature-rich and offers much more control over
indexing.
If you use the special program name "stdin" with "-i" or
IndexDir then swish-e will read from standard input instead of from a program.
For example:
% ./example.pl --count=1000 /path/to/data ⎪ ./swish-e -S prog -i stdin
This is basically the same as using a swish-e configuration file of:
SwishProgParameters --count=1000 /path/to/data
IndexDir ./example.pl
in a config file and running
% ./swish-e -S prog -c swish.conf
This gives an easy way to run swish without a configuration file with a "-S
prog" program that requires parameters. It also means you can capture
data to a file and then index more once with the same data:
% ./example.pl /path/to/data --count=1000 > docs.txt
% cat docs.txt ⎪ ./swish-e -S prog -i stdin -c normal_index
% cat docs.txt ⎪ ./swish-e -S prog -i stdin -c fuzzy_index
Using "stdin" might also be useful for programs that call swish
(instead of swish calling the program).
(The reason "stdin" is used instead of the more common "-"
dash is due to the rotten way swish parses the command line. This should be
fixed in the future.)
The "prog" method bypasses some of the configuration parameters
available to the file system method -- settings such as "IndexOnly",
"FileRules", "FileMatch" and "FollowSymLinks"
are ignored when using the "prog" method. It's expected that these
operations are better accomplished in the external program before passing the
document onto swish. In other words, when using the "prog" method,
only send the documents to swish that you want indexed.
You may use swish's filter feature with the "prog" method, but
performance will be better if you run filtering programs from within your
external program. See also
filters/README for an example how to easily
add document converstion and filtering into your Perl-based programs.
Notes when using -S prog on MS Windows
Windows does not use the shebang (#!) line of a program to determine the program
to run. So, when running, for example, a perl program you may need to specify
the perl.exe binary as the program, and use the
"SwishProgParameters" to name the file.
IndexDir e:/perl/bin/perl.exe
SwishProgParameters read_database.pl
Swish will replace the forward slashes with backslashes before running the
command specified with "IndexDir". Swish uses the
popen(3)
command which passes the command through the shell.
- -f *indexfile* (index file)
- If you are indexing, this specifies the file to save the
generated index in, and you can only specify one file. See also
IndexFile in the configuration file.
If you are searching, this specifies the index files (one or more) to search
from. The default index file is index.swish-e in the current
directory.
- -c *file ...* (configuration files)
- Specify the configuration file(s) to use for indexing. This
file contains many directives that control how Swish-e proceeds. See
SWISH-CONFIG for a complete listing of configuration file directives.
Example:
swish-e -c docs.conf
If you specify a directory to index, an index file, or the verbose option on
the command-line, these values will override any specified in the
configuration file.
You can specify multiple configuration files. For example, you may have one
configuration file that has common site-wide settings, and another for a
specific index.
Examples:
1) swish-e -c swish-e.conf
2) swish-e -i /usr/local/www -f index.swish-e -v -c swish-e.conf
3) swish-e -c swish-e.conf stopwords.conf
- 1
- The settings in the configuration file will be used to
index a site.
- 2
- These command-line options will override anything in the
configuration file.
- 3
- The variables in swish-e.conf will be read, then the
variable in stopwords.conf will be read. Note that if the same variables
occur in both files, older values may be written over.
- -e (economy mode)
- For large sites indexing may require more RAM than is
available. The "-e" switch tells swish to use disk space to
store data structures while indexing, saving memory. This option is
recommended if swish uses so much RAM that the computer begins to swap
excessively, and you cannot increase available memory. The trade-off is
slightly longer indexing times, and a busy disk drive.
- -l (symbolic links)
- Specifying this option tells swish to follow symbolic links
when indexing. The configuration file value FollowSymLinks will
override the command-line value.
The default is not to follow symlinks. A small improvement in indexing time
my result from enabling FollowSymLinks since swish does not need to stat
every directory and file processed to determine if it is a symbolic
link.
- -N path (index only newer files)
- The "-N" option takes a path to a file, and only
files newer than the specified file will be indexed. This is
helpful for creating incremental indexes -- that is, indexes that contain
just files added since the last full index was created of all files.
Example (bad example)
swish-e -c config.file -N index.swish-e -f index.new
This will index as normal, but only files with a modified date newer than
index.swish-e will be indexed.
This is a bad example because it uses index.swish-e which one might
assume was the date of last indexing. The problem is that files might have
been added between the time indexing read the directory and when the
index.swish-e file was created -- which can be quite a bit of time
for very large indexing jobs.
The only solution is to prevent any new file additions while full indexing
is running. If this is impossible then it will be slightly better to do
this:
Full indexing:
touch indexing_time.file
swish-e -c config.file -f index.tmp
mv index.tmp index.full
Incremental indexing:
swish-e -c config.file -N indexing_time.file -f index.tmp
mv index.tmp index.incremental
Then search with
swish-e -w foo -f index.full index.incremental
or merge the indexes
swish-e -M index.full index.incremental index.tmp
mv index.tmp index.swish-e
swish-e -w foo
- -r
-
**incremental index format only** The "-r"
option puts swish-e into "removal" mode. Any input files (given
with "-i" or the "IndexDir" parameter) are removed
from an existing index.
Example:
swish-e -r -i file.html
would remove file.html from the existing index.
- -u
-
**incremental index format only** The "-u"
option puts swish-e into "update" mode. The timestamp of each
input file is compared against the corresponding file in the existing
index. If swish-e encounters an input file that either does not exist yet
in the index or exists with a timestamp older than the input file, the
input file is updated in the index. Any words in the input file that have
been added or removed are reflected as such in the index.
Example:
swish-e -i file.html -u
would update the index.swish-e index with the contents of file.html. If
file.html was new, it would be added. If file.html already existed in the
index, its contents would be updated in the index.
- -v [0⎪1⎪2⎪3] (verbosity level)
- The "-v" option can take a numerical value from 0
to 3. Specify 0 for completely silent operation and 3 for detailed
reports.
If no value is given then 1 is assumed. See also IndexReport in the
configuration file.
Warnings and errors are reported regardless of the verbosity level. In
addition, all error and warnings are written to standard out. This is for
historical reasons (many scripts exist that parse standard out for error
messages).
- -W (0⎪1⎪2⎪3) (parser warning
level)
- If using the libxml2 parser, the default parser warning
level is set at 2. Use the "-W" option to override that default.
Most often, you might want to turn it off altogether:
swish-e -W0 -i path/to/files
would fail silently if the parser encountered any errors.
The following command line arguments are available when searching with Swish-e.
These switches are used to select the index to search, what fields to search,
and how and what to print as results.
This section just lists the available command line arguments and their usage.
Please see SWISH-SEARCH for detailed searching instructions.
Warning: If using Swish-e via a CGI interface, please see CGI Danger!
Security Note: If the swish binary is named
swish-search then swish will
not allow any operation that would cause swish to write to the index file.
Searching Command Line Arguments
- -w *word1 word2 ...* (query words)
- This performs a case-insensitive search using a number of
keywords. If no index file to search is specified (via the "-f"
switch), swish-e will try to search a file called index.swish-e in the
current directory.
swish-e -w word
Phrase searching is accomplished by placing the quote delimiter (a
double-quote by default) around the search phrase.
swish-e -w 'word or "this phrase"'
Search would should be protected from the shell by quotes. Typically, this
is single quotes when running under Unix.
Under Windows command.com you may not need to use quotes, but you
will need to backslash the quotes used to delimit phrases:
swish-e -w \"a phrase\"
The phrase delimiter can be set with the "-P" switch.
The search may be limited to a MetaName. For example:
swish-e -w meta1=(foo or baz)
will only search within the meta1 tag.
Please see SWISH-SEARCH for a description of MetaNames
- -f *file1 file2 ...* (index files)
- Specifies the index file(s) used while searching. More than
one file may be listed, and each file will be searched. If no
"-f" switch is specified then the file index.swish-e in
the current directory will be used as the index file.
- -m *number* (max results)
- While searching, this specifies the maximum number of
results to return. The default is to return all results.
This switch is often used in conjunction with the "-b" switch to
return results one page at a time (strongly recommended for large
indexes).
- -b *number* (beginning result)
- Sets the begining search result to return (records
are numbered from 1). This switch can be used with the "-m"
switch to return results in groups or pages.
Example:
swish-e -w 'word' -b 1 -m 20 # first 'page'
swish-e -w 'word' -b 21 -m 20 # second 'page'
- -t HBthec (context searching)
- The "-t" option allows you to search for words
that exist only in specific HTML tags. Each character in the string you
specify in the argument to this option represents a different tag in which
to search for the word. H means all HEAD tags, B stands for BODY tags, t
is all TITLE tags, h is H1 to H6 (header) tags, e is emphasized tags (this
may be B, I, EM, or STRONG), and c is HTML comment tags
search only in header (<H*>) tags
swish-e -w word -t h
- -d *string* (delimiter)
- Set the delimiter used when printing results. By default,
Swish-e separates the output fields by a space, and places double-quotes
around the document title. This output may be hard to parse, so it is
recommended to use "-d" to specify a character or string used as
a separator between fields.
The string "dq" means "double-quotes".
swish-e -w word -d , # single char
swish-e -w word -d :: # string
swish-e -w word -d '"' # double quotes under Unix
swish-e -w word -d \" # double quotes under Windows
swish-e -w word -d dq # double quotes
The following control characters may also be specified: "\t \r \n
\f".
Warning: This string is passed directly to sprintf() and therefore
exposes a securty hole. Do not allow user data to set -d format strings
directly.
- -P *character*
- Sets the delimiter used for phrase searches. The default is
double quotes """.
Some examples under bash: (be careful about you shell metacharacters)
swish-e -P ^ -w 'title=^words in a phrase^'
swish-e -P \' -w "title='words in a pharse"'
- -p *property1 property2 ...* (display properties)
- This causes swish to print the listed property in the
search results. The properties are returned in the order they are listed
in the "-p" argument.
Properties are defined by the ProperNames directive in the
configuration file (see SWISH-CONFIG) and properties must also be defined
in MetaNames. Swish stores the text of the meta name as a
property, and then will return this text while searching if this
option is used.
Properties are very useful for returning data included in a source documnet
without having to re-read the source document while searching. For
example, this could be used to return a short document description. See
also see Document Summeries and PropertyNames in SWISH-CONFIG.
To return the subject and category properties while indexing.
swish-e -w word -p subject category
Properties are returned in double quotes. If a property contains a double
quote it is HTML escaped ("). See the "-x" switch for a
more advanced method of returning a list of properties.
NOTE: it is necessary to have indexed with the proper PropertyNames
directive in the user config file in order to use this option.
- -s *property [asc⎪desc] ...* (sort)
- Normally, search results are printed out in order of
relevancy, with the most relevant listed first. The "-s" sort
switch allows you to sort results in order of a specified property,
where a property was defined using the MetaNames and
PropertyNames directives during indexing (see SWISH-CONFIG).
The string passed can include the strings "asc" and
"desc" to specify the sort order, and more than one property may
be specified to sort on more than one key.
Examples:
sort by title property ascending order
-s title
sort descending by title, ascending by name
-s title desc name asc
Note: Swish limits sort keys to 100 characters. This limit can be changed by
changing MAX_SORT_STRING_LEN in src/config.h and rebuilding swish-e.
- -L limit to a range of property values (Limit)
-
This is an experimental feature!
The "-L" switch can be used to limit search results to a range of
property values
Example:
swish-e -w foo -L swishtitle a m
finds all documents that contain the word "foo", and where the
document's title is in the range of "a" to "m",
inclusive. By default, the case of the property is ignored, but this can
be changed by using PropertyNamesCompareCase configuation directive.
Limiting may be done with user-defined properties, as well.
For example, if you indexed documents that contain a created timestamp in a
meta tag:
<meta name="created_on" content="982648324">
Then you tell Swish that you have a property called "created_on",
and that it's a timestamp.
PropertyNamesDate created_on
After indexing you will be able to limit documents to a range of timestamps:
-w foo -L created_on 946684800 949363199
will find documents containing the word foo and that have a created_on date
from the start of Jan 1, 2000 to the end of Jan 31, 2000.
Note: swish currently does not parse dates; Unix timestamps must be used.
Two special formats can be used:
-L swishtitle <= m
-L swishtitle >= m
Finds titles less than or equal, or grater than or equal to the letter
"m".
This feature will not work with "swishrank" or
"swishdbfile" properties.
This feature takes advantages of the pre-sorted tables built by swish during
indexing to make this feature fast while searching. You should see in the
indexing output a line such as:
6 properties sorted.
That indicates that six pre-sorted tables were built during indexing. By
default, all properties are presorted while indexing. What properties are
pre-sorted can be controlled by the configuration parameter
"PreSortedIndex".
Using the "-L" switch on a property that was not pre-sorted will
still work, but may be much slower during searching.
Note that the PropertyNamesSortKeyLength setting is used for sorting
properties. Using too small a PropertyNamesSortKeyLength could result in
-L selecting the wrong properties due to incomplete sorting.
This is an experimental feature, and its use and interface are subject to
change.
- -x formatstring (extended output format)
- The "-x" switch defines the output format string.
The format string can contain plain text and property names (including
swish-defined internal property names) and is used to generate the output
for every result. In addition, the output format of the property name can
be controlled with C-like printf format strings. This feature overrides
the cmdline switches "-d" and "-p", and a warning will
be generated if "-d" or "-p" are used with
"-x".
Warning: The format string (fmt) is passed directly to sprintf() and
therefore exposes a securty hole. Do not allow user data to set -x format
strings directly.
For example, to return just the title, one per line, in the search results:
swish-e -w ... -x '<swishtitle>\n' ...
Note: the "\n" may need to be protected from your shell.
See also ResultExtFormatName for a way to define named format strings
in the swish configuration file.
Format of "formatstring":
"text<propertyname>text<propertyname fmt=propfmtstr>text..."
Where propertyname is:
- *
- the name of a user property as specified with the config
file directive "PropertyNames"
- *
- the name of a swish Auto property (see below). These
properties are defined automatically by swish -- you do not need to
specify them with PropertyNames directive. (This may change in the
future.)
propertynames must be placed within "<" and ">".
User properties:
Swish-e allows you to specify certain META tags within your documents that can
be used as
document properties. The contents of any META tag that has
been identified as a document property can be returned as part of the search
results. Doucment properties must be defined while indexing using the
PropertyNames configuration directive (see SWISH-CONFIG).
Examples of user-defined PropertyNames:
<keywords>
<author>
<deliveredby>
<reference>
<id>
Auto properties:
Swish defines a number of "Auto" properties for each document indexed.
These are available for output when using the "-x" format.
Name Type Contents
-------------- ------- ----------------------------------------------
swishreccount Integer Result record counter
swishtitle String Document title
swishrank Integer Result rank for this hit
swishdocpath String URL or filepath to document
swishdocsize Integer Document size in bytes
swishlastmodified Date Last modified date of document
swishdescription String Description of document (see:StoreDescription)
swishdbfile String Path of swish database indexfile
The Auto properties can also be specified using shortcuts:
Shortcut Property Name
-------- --------------
%c swishreccount
%d swishdescription
%D swishlastmodified
%I swishdbfile
%p swishdocpath
%r swishrank
%l swishdocsize
%t swishtitle
For example, these are equivalent:
-x '<swishrank>:<swishdocpath>:<swishtitle>\n'
-x '%r:%p:%t\n'
Use a double percent sign "%%" to enter a literal percent sign in the
output.
Formatstrings of properties:
Properties listed in an "-x" format string can include format control
strings. These "propertyformats" are used to control how the
contents of the associated property are printed. Property formats are used
like C-language printf formats. The property format is specified by including
the attribute "fmt" within the property tag.
Format strings cannot be used with the "%" shortcuts described above.
General syntax:
-x '<propertyname fmt="propfmtstr">'
where "subfmt" controls the output format of "propertyname".
Examples of property format strings:
date type: <swishlastmodified fmt="%d.%m.%Y">
string type: <swishtitle fmt="%-40.35s">
integer type: <swishreccount fmt=/%8.8d/>
Please see the manual pages for
strftime(3) and
sprintf(3) for an
explanation of format strings. Note: some versions of strftime do not offer
the %s format string (number of seconds since the Epoch), so swish provides a
special format string "%ld" to display the number of seconds since
the Epoch.
The first character of a property format string defines the delimiter for the
format string. For example,
-x "<author fmt=[%20s]> ...\n"
-x "<author fmt='%20s'> ...\n"
-x "<author fmt=/%20s/> ...\n"
Standard predefined formats:
If you ommit the sub-format, the following formats are used:
String type: "%s" (like printf char *)
Integer type: "%d" (like printf int)
Float type: "%f" (like printf double)
Date type: "%Y-%m-%d %H:%M:%S" (like strftime)
Text in "formatstring" or "propfmtstr":
Text will be output as-is in format strings (and property format strings).
Special characters can be escaped with a backslash. To get a new line for each
result hit, you have to include the Newline-Character "\n" at the
end of "fmtstr".
-x "<swishreccount>⎪<swishrank>⎪<swishdocpath>\n"
-x "Count=<swishreccount>, Rank=<swishrank>\n"
-x "Title=\<b\><swishtitle>\</b\>"
-x 'Date: <swishlastmodified fmt="%m/%d/%Y">\n'
-x 'Date in seconds: <swishlastmodified fmt=/%ld/>\n'
Control/Escape charcters:
you can use C-like control escapes in the format string:
known controls: \a, \b, \f, \n, \r, \t, \v,
digit escapes: \xhexdigits \0octaldigits
character escapes: \anychar
Example,
swish -x "%c\t%r\t%p\t\"<swishtitle fmt=/%40s/>\"\n"
Examples of -x format strings:
-x "%c⎪%r⎪%p⎪%t⎪%D⎪%d\n"
-x "%c⎪%r⎪%p⎪%t⎪<swishdate fmt=/%A, %d. %B %Y/>⎪%d\n"
-x "<swishrank>\t<swishdocpath>\t<swishtitle>\t<keywords>\n
-x "xml_out: \<title\><swishtitle>\>\</title\>\n"
-x "xml_out: <swishtitle fmt='<title>%s</title>'>\n"
- -H [0⎪1⎪2⎪3⎪<n>] (header
output verbosity)
- The "-H n" switch generates extened header
output. This is most useful when searching more than one index file at a
time by specifying more than one index file with the "-f"
switch. "-H 2" will generate a set of headers specific to each
index file. This gives access to the settings used to generate each index
file.
Even when searching a single index file, "-H n" will provided
additional information about the index file, how it was indexed, and how
swish is interperting the query.
-H 0 : print no header information, output only search result entries.
-H 1 : print standard result header (default).
-H 2 : print additional header information for each searched index file.
-H 3 : enhanced header output (e.g. print stopwords).
-H 9 : print diagnostic information in the header of the results (changed from: C<-v 4>)
- -R [0⎪1] (Ranking Scheme)
-
This is an experimental feature!
The default ranking scheme in SWISH-E evaluates each word in a query in
terms of its frequency and position in each document. The default scheme
is 0.
New in version 2.4.3 you may optionally select an experimental ranking
scheme that, in addition to document frequency and position, uses Inverse
Document Frequency (IDF), or the relative frequency of each word across
all the indexes being searched, and Relative Density, or the normalization
of the frequency of a word in relationship to the number of words in the
document.
NOTE: IgnoreTotalWordCountWhenRanking must be set to no or
0 in your index(es) for -R 1 to work.
Specify -R 1 to turn on IDF ranking. See the API documentation for how to
set the ranking scheme in your Perl or C program.
- -V (version)
- Print the current version.
- -k *letter* (print out keywords)
- The "-k" switch is used for testing and will
cause swish to print out all keywords in the index beginning with that
letter. You may enter "-k '*'" to generate a list of all words
indexed by swish.
- -D *index file* (debug index)
- The -D option is no longer supported in version 2.2.
- -T *options* (trace/debug swish)
- The -T option is used to print out information that may be
helpful when debugging swish-e's operation. This option replaced the
"-D" option of previous versions.
Running "-T help" will print out a list of available
*options*
In previous versions of Swish-e indexing would require a very large amount of
memory and the indexing process could be very slow. Merging provided a way to
index in chunks and then combine the indexes together into a single index.
Indexing is much faster now and uses much less memory, and with the
"-e" switch very little memory is needed to index a large site.
Still, at times it can be useful to merge different index files into one file
for searching. This could be because you want to keep separate site indexes
and a common one for a global search, or you have separate collections of
documents that you wish to search all at one time, but manage separately.
- -M *index1 index2 ... indexN out_index
- Merges the indexes specified on the command line -- the
last file name entered is the output file. The output index must not exist
(otherwise merge will not proceed).
Only indexes that were indexed with common settings may be merged. (e.g.
don't mix stemming and non-stemming indexes, or indexes with different
WordCharacter settings, etc.).
Use the "-e" switch while merging to reduce memory usage.
Merge generates progress messages regardless of the setting of
"-v".
- -c *configuration file*
- Specify a configuration file while indexing to add
administrative information to the output index file.
$Id: SWISH-RUN.pod 1741 2005-05-17 02:22:40Z karman $
.