textindexer-0.2

These are quick design notes.

Classes (in the order we consider them)

Word is just a thin wrapper around the string class. Its main purpose is the implementation of the input operator operator>>(istream &, Word) which is such that

word w;
istream is;
is >> w;

succeeds only if w contains a nonempty word that consist of the next maximal sequence of alphabetic characters from the stream. The test program is the file test-word.C, included in the class documentation.

The wordstream_iterator class behaves like an input iterator. It is initialized using a stream and a set of words to be ignored. Thus

set<string> ignore;
istream is;
copy(wordstream_iterator(is, ignore), wordstream_iterator(),
  ostream_iterator<string>(cout,"\n") )

will print all the words (see Word) from is that are not on the ignore list.

In order to save space, we want to use a StringPool class to keep the names of the files that have been indexed. Thus a word in the index will be associated with a set of pointers to filenames instead of a set of filenames.

The stringpool is convenient to use:

StringPool pool;
string s;
string *p;
if (p=pool(s)) // s found in pool, p points to the string in the pool
..
p = pool[s]; // if s found in pool, same as pool(s) otherwise, s is
// added to the pool and then pool(s) is returned.

The Index class is the main index structure it contains:

a stringpool for the filenames,
a mapping from (shortened) words to sets of (pointers to) filenames.
a mapping from (pointers to) filenames to dates (for each filename, the date of the last index operation is kept),
a set of strings representing words to be ignored.

Index::query(set<string> query, set<string*> filenames)

can be used to obtain the set of filenames that are associated with each of the words in query. There are also facilities for reading and writing an Index object to a text file.

The Query class represents a more complicated query using phrases as well as single words. It is associated, at construction time, with an Index structure on which it can be executed. The execution simply calls Index::query() for the list of words in the query and then checks the resulting filenames for exact occurrences of all the phrases in the query (where a word by itself counts as a short phrase).

The Server class uses both Query and Index to implement the server interface. It does not know about sockets but simply handles input and outpu through an iostream. The main functions are the constructor which processes the configuration file and Server::serve(iostream& ios) which repeatedly reads commands from ios and writes the proper response to the same stream. There are a few auxiliary functions, notably a function Server::ok_to_index(const string& path) that returns true iff path refers to a file that can be indexed.

The file indexer.C contains the server program: it simply creates a server object, opens a server socket and calls Server::serve(socket) for each socket that results from an accept by the server socket.

The file ffiles.C contains the client program and a single auxiliary function that recursively adds files, based on a single path: if the path refers to a directory, all the files in the directory will be added. The specs were unclear on the exact response of ffiles to an add command. In this implementation, ffiles will print the number of files actually added.

A simple text indexer server and example client.

Installation

Design