@apl{xsbdoc} is an @em{automatic program documentation generator} for
XSB applications and modules.

@apl{xsbdoc} is a rewriting of the Ciao system's @apl{lpdoc} to
generate a reference manual automatically from one or more XSB source
files @cite{ciao-man}).  The target format of the documentation can be
Postscript, HTML, PDF, or nicely formatted ASCII text.  @apl{xsbdoc}
can be used to automatically generates a description of full
applications, library modules, README files, etc.  A fundamental
advantage of using @apl{xsbdoc} to document programs is that it is
much easier to maintain a true correspondence between the program and
its documentation, and to identify precisely to what version of the
program a given printed manual corresponds.  Naturally, the
@apl{xsbdoc} manual generated by @apl{xsbdoc} itself.

@section{Overview of this document}

This first part of the document provides basic explanations on how to
generate a manual from a set of files that already contain
@concept{assertions} and @concept{comments}. Examples are given using
the files in the @tt{doc} directory that generate this manual.  These
instructions assume that @apl{xsbdoc} is installed as an XSB package.

Other parts of this document provide:

@begin{itemize}

@item Documentation on the @concept{formatting commands} that can be
embedded in @em{comments}.

@item Documentation on the @em{assertions} that @apl{xsbdoc} uses
(those defined in the @apl{XSB} @pred{assertions} library.

@end{itemize}

@comment{
-----------------------------------------------------------------
This document is also an @concept{internals manual}, providing
information on how the different internal parts of @apl{xsbdoc} are
connected.  These chapters need not be read by an @apl{xsbdoc} user,
rather they are intended for those who wish to maintain and improve
@apl{xsbdoc}.  To this end, the document also provides:

@begin{itemize}

@item The documentation for the @lib{autodoc} @concept{automatic
documentation library}, which provides the main functionality of
@apl{xsbdoc}.

@item Documentation on the predicates that define the
@concept{conversion formats} used (@apl{texinfo}, and others), and
which are in the @lib{autodocformats} library.

@end{itemize}
-----------------------------------------------------------------
}

All of the above have been generated automatically from the comments
and assertions in the corresponding sources and can also be seen as
examples of the use of @apl{xsbdoc}.

@section{ xsbdoc operation - source and target files}

The main input used by @apl{xsbdoc} in order to generate a manual are
@concept{Prolog source files}.  Basically, @apl{xsbdoc} generates a
file in the GNU @apl{texinfo} format (with a @tt{.texi} ending) for
each @apl{Prolog} file (see ``The GNU Texinfo Documentation System''
manual for more info on this format). Using these @tt{texi} files,
@apl{xsbdoc} calls various applications to generate Postscript, PDF,
HTML, or ASCII output.

The @apl{Prolog} files must have a @tt{.P} ending, although
information is also taken from files ending in @tt{.H} if the
corresponding @tt{.P} file is present.  Depending on declarations from
the @concept{format file} used to generate the documentation a given
file may be documented as a @concept{library} or as an
@concept{application}.  In the first case,
 the @tt{.texi} file generated for it will contain information on the
interface (e.g., the predicates exported and imported by the file,
etc.).  In the second case, if it is an application, no 
 no description of the interface will be generated

If needed, @cindex{texinfo files} files written directly in
@apl{texinfo} can also be used as input files for @apl{xsbdoc}. These
files @em{must have a} @tt{.src} @em{(instead of} @tt{.texi} @em{)
ending}. This is needed to distinguish them from any automatically
generated @tt{.texi} files. Writing files directly in @apl{texinfo}
has the disadvantage that it may be difficult to adhere to all the
conventions used by @apl{xsbdoc}.  For example, these files will be
typically used as chapters and must be written as such. Also, the set
of indices used must be the same as specified in the @apl{xsbdoc}
format file for the application.  Finally, no bibliographic citations
can be used in @tt{.src} files.  Because of this, and because in the
future @apl{xsbdoc} may be able to generate documentation in formats
other than @apl{texinfo} directly (in which case these files would not
be useful), writing files in @apl{texinfo} directly is
discouraged. This facility was added mainly to be able to reuse parts
of manuals which were already written in @apl{texinfo}.  Note that if
a stand-alone file needs to be written (i.e., a piece of documentation
that is not associated to any @tt{.P} file) it can always be written
as a ``dummy'' @tt{.P} file (i.e., one that is not used as code), but
which contains machine readable comments).

@subsection{Documenting Libraries and/or Applications}

A manual can be generated either from a single source file (@tt{.P} or
@tt{.src}) or from a set of source files.  In the latter case, then
one of these files should be chosen to be the @index{main file}, and
the others will be the @index{component files}.  The main file is the
one that will provide the title, author, date, summary, etc. to the
entire document.  In principle, any set of source files can be
documented, even if they contain no assertions or comments. However,
the presence of assertions or comments will greatly improve the
documentation (see @ref{Enhancing the documentation being generated}).

If the manual is generated from a single main file (i.e., there are no
component files) then a flat document containing no chapters will be
generated.  If the manual is generated from a main file and one or
more components, then the document will contain chapters. The comments
in the main file will be used to generate the introduction, while each
of the component files will be used to generate a separate
chapter. The contents of each chapter will be controlled by the
contents of the corresponding component file.

@comment{
As mentioned before, @apl{xsbdoc} typically generates @apl{texinfo}
files.  From the @apl{texinfo} files, @apl{xsbdoc} can generate
printed and on-line manuals in several formats (@tt{dvi}, @tt{ps},
@tt{ascii}, @tt{html}, @tt{info}, etc.)  automatically, using
different (publicly available) packages.  @apl{xsbdoc} can also
generate directly includes generating (parts of) a @concept{master
index} of documents which can be placed in an installation directory
and which will provide pointers to the individual manuals
generated. Using this feature, @tt{xsbdoc} can maintain global
@apl{html} and/or @apl{info} documentation sites automatically (see
@ref{Installing a generated manual in a public area}).

}

@section{Generating a manual} 

To use @apl{xsbdoc}, the package must be first loaded via the command

@begin{verbatim}
         [xsbdoc].
@end{verbatim}

The top-level command to generate documentation is 

@begin{verbatim}
?- xsbdoc(FormatFile,GenerationType).
@end{verbatim}

Where @tt{FormatFile} is a file specifying input files and formatting
options to use, while @tt{GenerationType} is specifies the graphical
format to use: postscript, html, etc.  We consider each of the
arguments in turn.

@subsection{The Format File}

The @apl{doc} library directory includes the @em{format file}
@tt{'xsbdoc_format.P'} used to generate this manual, and which can be
used as a example of how to generate other manuals.  Typically, the
format file is put into a separate directory (say a subdirectory) from
the code that it documents, and relative or absolute paths to the main
file and the components are given so that they can be accessed.  The
main file to be used is indicated by the predicate
@pred{xsbdoc_main/1}, while component files are indicated by the
predicate @pred{xsbdoc_component/1} and are documented in the order in
which they occur in the format file.  For instance, in
@tt{'xsbdoc_format.P'}, the main file is indicated by the fact:

@begin{verbatim}
xsbdoc_main('../xsbdoc1.P').
@end{verbatim}

while the two few chapters are indicated by the facts

@begin{verbatim}
xsbdoc_component('comments.P').
xsbdoc_component('assertions_props.P').
@end{verbatim}

@apl{xsbdoc} uses as default directory search paths whatever
directories have been asserted via the XSB predicate
@pred{library_directory/1} (See the XSB manual for more details).

Finally, if there are any citations in the manual, any bibtex files
will have to be indicated through @pred{xsbdoc_bibfile/1}.  All the
references will appear together in a @em{References} appendix at the
end of the manual.  If you are not using citations, then add the fact
@tt{xsbdoc_option('-norefs')} in the format file, which will prevent
an empty 'References' appendix from appearing in the manual.

@subsection{Miscellaneous options}

Like @apl{lpdoc}, @apl{xsbdoc} can use default settings for many of
the other options.  However, if you wish to control the behavior of
@apl{xsbdoc}, you can do so through a large array of options.  

@begin{itemize}

@item Options that don't require parameters are indicated through
@pred{xsbdoc_option/1} facts.  @pred{xsbdoc_option/1} facts can be
used to configure whether various pieces of information are included
or not, along with many particulars of how the generated documentation
looks.  None of these options were used in generating this manual.
The current set of values for @pred{xsbdoc_option/1} are:

@begin{description}

@item @tt{xsbdoc_option('-noauthors')} 

Omits all author names.

@item @tt{xsbdoc_option('-twosided')} Formats

 the documentation for
printing on two-sided paper by starting each chapter and appendix on
an odd-numbered page.

@item @tt{xsbdoc_option('-shorttoc')} 

Produces a shorter table of
contents that does not include entries for definitions of individual
predicates.

@item @tt{xsbdoc_option('-noversion')} 

Omits version information from
various places in the manual --- such as the title page and summary.

@item @tt{xsbdoc_option('-nosysmods')} 

Does not include system modules
(defined as subdirectories of the root XSB directory) in list of
libraries used for modules.

@item @tt{xsbdoc_option('-nochangelog')} 

Does not include a change log
at the end of a chapter (cf. @ref{Version/Change Log (xsbdoc1)}).

@item @tt{xsbdoc_option('-nopatches')}

Does not include comments for patches within the change log
(i.e. only major & minor revisions are included)

@item @tt{xsbdoc_option('-nobugs')} 

Does not include information on known bugs and planned improvements
(cf.  @ref{Known bugs and planned improvements (xsbdoc1)}).

@item @tt{xsbdoc_option('-norefs')} 

Omits a @em{References} section regardless of whether citations are
present.

@item @tt{xsbdoc_option('-nopropsepln')} 

Does not put each property in a separate line when @tt{pred/1}
assertions are commented for individual predicates (cf. the assertions
for @tt{comment/2} in @ref{Documentation on exports (comments)}).

@item @tt{xsbdoc_option('-nopropnames')} 

Do not include property names when @tt{pred/1} assertions are
commented for individual predicates (cf. the assertions for
@tt{comment/2} in @ref{Documentation on exports (comments)}).

@item @tt{xsbdoc_option('-v')} 

Generates verbose output and can be useful for debugging document
speficications.

@end{description}

@item Facts of the form @pred{xsbdoc_index/1} determine the list of
indices to be included at the end of the document.  These can include
indices for defined predicates, modules, concepts, etc.

@begin{description}

@item @tt{xsbdoc_index(pred)} 

Index of defined predicates.

@item @tt{xsbdoc_index(op)} 

Index of defined operators.

@item @tt{xsbdoc_index(concept)} 

Index of cited and defined concepts.

@item @tt{xsbdoc_index(global)} 

Global index of cited and defined predicates, operators, libraries,
applications, concepts, etc.

@item @tt{xsbdoc_index(all)} 

Includes all currently defined indices.  @em{Limitations in the number
of simultaneous indices} in some texinfo installations may
occasionally prevent this option from working.

@end{description}


@cindex{page numbering, changing}

@item The predicate @pred{xsbdoc_startpage/1} changes the page number
of the first page of the manual. This can be useful if the manual is
to be included in a larger document or set of manuals.  Typically,
this should be an @em{odd} number.  The default number is @tt{1}.

@cindex{page size, changing}
@cindex{page style, changing}

@item @pred{xsbdoc_papertype/1} allows selection between several paper
sizes for the printable outputs (@tt{dvi}, @tt{ps}, etc.). The
currently supported outputs (most of them inherited from
@apl{texinfo}) are:

  @begin{description}

  @item{@tt{afourpaper}} 

     The default, usable for printing on @index{A4 paper}. Rather busy, but
     saves trees.

  @item{@tt{afourwide}} 

     This one crams even more stuff than @tt{afourpaper} on an A4
     page. Useful for generating manuals in the least amount of space.
     Saves more trees.

  @item{@tt{afourlatex}} 

     This one  is a little less compressed than @tt{afourpaper}.

  @item{@tt{smallbook}} 

     Small pages, like in a handbook. 

  @item{@tt{letterpaper}} 

     For printing on American @index{letter size paper}.

  @item{@tt{afourthesis}} 

     A @index{thesis-like style} (i.e., double spaced, wide margins
     etc.). Useful -- for inserting @apl{xsbdoc} output as appendices 
     of a thesis or similar document. Does not save trees. 

  @end{description}

The default paper type is @tt{letterpaper}.

@end{itemize}

@subsection{Graphical Formatting}

The second argument of @pred{xsbdoc/2} indicates the target graphical
format.  Use of all options depends on having @apl{tex} installed on
your system.

The current options are: 

@begin{itemize}

@item @tt{dvi}, which generates a dvi format file, viewable by
@apl{xdvi}, @apl{yap} and other viewers.  

@item @tt{ps} which generates a viewable and printable postscript
file.  Use of this option depends on having @apl{dvips} installed on
your machine.

@item @tt{pdf} which generates a viewable and printable postscript
file.  Use of this option depends on having @apl{dvips} and
@apl{ps2pdf} installed on your machine.

@item @tt{html} which generates html files in a subdirectory of the
current working directory.  The directory is named @tt{<Main>_html}
where @tt{Main} is the base name of the file (i.e. the filename
without a directory path or extension) specified by the fact
@pred{xsbdoc_main/1} in the format file.  Use of this option depends
on having @apl{texi2html} installed on your machine.

@item @tt{ascii} which generates ascii files which are surprisingly
nicely formatted to denote chapter and section headings, indices, and
so on.  Use of this option depends on having @apl{makeinfo} installed
on your machine.

@end{itemize}

Future versions will probably also include support for generating
@tt{rtf} files. 

@section{Enhancing the documentation being generated} 

The quality of the documentation generated can be greatly enhanced by
including within the program text:

@begin{itemize}

@item @bf{Import/Export Information}. @apl{xsbdoc} uses information on
exported and imported predicates in order to document the interface of
a module.  Because some users consider the XSB module system is
awkward to use while developing code, @apl{xsbdoc} also recognizes the
directives

@begin{itemize} 

@item @pred{document_export/1}, and 

@item @pred{document_import/1}

@end{itemize} 

which are treated by @apl{xsbdoc} exactly as @pred{export/1} and
@pred{import/1}, but which are ignored by the XSB compiler.

In principle, only the (document_)exported predicates are documented,
although any predicate can be included in the documentation by
explicitly requesting it (see the documentation for the
@decl{comment/2} declaration).

@item @bf{Assertions} are directives that are included in the source program
and provide the compiler with information regarding characteristics of
the program. Typical assertions include standard compiler directives
(such as @decl{dynamic/1}, @decl{op/3},
@decl{use_variant_tabling/1}...), etc.  However, the @decl{pred/2}
assertion can also be used to declare global, call, and success types
and modes (see @ref{The XSB Assertion Library}).

When documenting a module, @apl{xsbdoc} will use the assertions
associated with the module interface to construct a textual
description of this interface.  Judicious use of these assertions
combines documentation of the program with improved debugging
behavior.  Improvement in debugging is possible because some
assertions provide information on the intended meaning or behaviour of
the program (i.e., the specification) which can be checked at
compile-time (by a suitable preprocessor/static analyzer) and/or at
run-time (via checks inserted by a preprocessor).  (TLS: this portion
is still under development for XSB).

@item @bf{Machine-readable comments} are also declarations included in
the source program but which contain additional information intended
to be read by humans (i.e., this is an instantiation of the
@index{literate programming} style of Knuth @cite{knuth-lit}). In
addition to predicate-level comments, typical comments include title,
author(s), bbugs, changelog, etc.

@end{itemize}

@section{Other Usage Information}

@comment{
@subsection{Splitting large documents into parts}

TLS test and revise.  As mentioned before, in @apl{xsbdoc} each
documented file (each component) corresponds to a chapter in the
generated manual. In large documents, it is sometimes convenient to
build a super-structure of parts, each of which groups several
chapters. There is a special value of the second argument of the
@tt{:- comment(filetype,...).}  declaration mentioned above designed
for this purpose. The special @em{filetype} value @tt{part} can be
used to flag that the file in which it appears should be documented as
the start of one of the major @index{parts in a large document}. In
order to introduce such a part, a @tt{.pl} file with a declaration
@tt{:- comment(filetype,part).}  should be inserted in the sequence of
files that make up the @tt{COMPONENTS} variable of the @file{SETTINGS}
file at each point in which a major part starts. The @tt{:-
comment(title,""..."").}  declaration of this file will be used as the
part title, and the @tt{:- comment(module,""..."").}  declaration text
will be used as the introduction to the part.
}

@subsection{Separating the documentation from the source file}

Sometimes it is convenient to include long introductory comments in a
separate file from a source code file.  This can be by using the
@@include command. For example, the declaration:

@begin{verbatim}
:- comment(module,""@@include@{Intro.xsbdoc@}"").
@end{verbatim}

@noindent includes the contents of the file @tt{Intro.xsbdoc} as the
module description.

Alternatively, sometimes it may be convenient to generate the
documentation from a completely different file. Assuming that the
original module is @tt{m1.P}, documentation can be kept in
@tt{m1_doc.P}, and included as a component in the format file.
@apl{xsbdoc} recognizes and treats such @tt{_doc} files specially so
that the name without the @tt{_doc} part is used in various parts of
the documentation, in the same way as if the documentation were placed
in file @tt{m1}.

Finally, there are several other mechanisms of including image files,
predicate definitions and so on in machine readable comments, as is
explained in @ref{Enhancing Documentation with Machine-Readable
Comments}.

@subsection{Writing comments to document version/patch changes}

Version comments (@tt{:- comment(version(...), ""..."").}) can provide
useful documentation on how a given library or application has been
changed or has evolved.  Sometimes one would like to write version
comments that are internal, i.e., not meant to appear in the
manual. This can, of course be done with standard Prolog comments
(which @apl{xsbdoc} will not read). An alternative and quite useful
solution is to put such internal comments in @em{patch} changes (e.g.,
1.1#2 to 1.1#3), and put the more general comments, which describe
major changes to the user and should appear in the manual, in
@em{version} changes (e.g., 1.1#2 to 1.2#0). Selecting the appropriate
options in @apl{xsbdoc} then allows including in the manual the
version changes but not the patch changes (which might on the other
hand be included in an @index{internals manual}).

@subsection{Cleaning up the documentation directory} 

Typing the command: 
@begin{verbatim}
?- make_distclean.
@end{verbatim}
deletes all intermediate files and the
generated @tt{.texic} files, leaving only the targets (i.e., the
@tt{.ps}, @tt{.dvi}, @tt{.ascii}, @tt{.html}, etc. files). This is the
option normally used when building software distributions in which the
manuals come ready made in the distribution itself and will not need
to be generated during installation.

@subsection{Ensuring Compatibility with All Supported Target Formats}

@apl{xsbdoc} allows manuals to be generated in several different
formats.  Because these formats each have widely different
requirements it is sometimes a little tricky to get things to work
successfully for all formats. The following recommendations may help:

@begin{itemize} 

@comment{
@item @apl{texinfo} supports only a limited number of indices. Thus,
if you select too many indices in the format file you may
exceed @apl{texinfo}'s capacity (which it will signal by saying
something like ``No room for a new @@write'').
}

@item The GNU info format requires all @em{nodes} (chapters, sections,
etc.) to have different names. This is ensured by @apl{xsbdoc} for
the automatically generated sections (by appending the module or file
name to all section headings). However, care must be taken when
writing section names manually to make them different. For example,
use ``xsbdoc usage'' instead of simply ``Usage'', which is much more
likely to be used as a section name in another file being documented.

@item Also due to a limitation of the @apl{info} format, (used in
generating formatted ASCII files) do not use @tt{:} or @tt{,} or
@tt{-}@tt{-} in section, chapter, etc. headings.

@item The character ``@tt{_}'' in names may sometimes give problems in
indices, since current versions of @apl{texinfo} do not always handle
it correctly.

@end{itemize}

@comment{
@subsection{Documenting files which are not modules}

TLS: revise -- Sometimes it is difficult for @apl{xsbdoc} to
distinguish @concept{include files} and CIAO @concept{packages} from
normal @em{user} files (i.e., normal code files but which are not
modules). The distinction is important because the former are quite
different in their form of use (they are loaded via @decl{include/1}
or @decl{use_package/1} declarations instead of
@decl{ensure_loaded/1}) and effect (since they are included, they
'export' operators, declarations, etc.), and should typically be
documented differently.  There is a special @decl{comment/2}
declaration which provides a way of defining the intended use of the
file. This declaration is normally not needed in modules, include
files, or packages, but should be added in user files (i.e., those
meant to be loaded using @decl{ensure_loaded/1}).
}

@subsection{Generating auxiliary files (e.g., READMEs)}

Using @apl{xsbdoc} it is often possible to use a common source for
documentation which should appear in several places. For example, if a
file @file{INSTALL.xsbdoc} contains text (with @tt{xsbdoc} formatting
commands) describing an application. This text can be included in a
section of the main file documentation as described above:

@begin{verbatim}
:- comment(module,""
   ...
   @@section@{Installation instructions@}
   @@include@{INSTALL.xsbdoc@}
   ...
   "").
@end{verbatim}

@noindent At the same time, this text can be used to generate a nicely
formatted @tt{INSTALL} file in ascii.  To this end, an @tt{INSTALL.P}
file as follows can be constructed:

@begin{verbatim}
:- comment(title,""Installation instructions"").
:- comment(module,""@@include@{INSTALL.xsbdoc@}"").
@end{verbatim}

Then, the ascii @tt{INSTALL} file can be generated by simply running
@tt{xsbdoc(format,ascii)} using a file @tt{format.P} with the fact
 @tt{xsbdoc_main('INSTALL')}.

@section{Troubleshooting}

These are some common errors which may be found using @apl{xsbdoc}
and the usual fix:

@begin{itemize}

@item Messages of the type:
@begin{verbatim}

 ! No room for a new @@write .

@end{verbatim} while converting from @tt{.texi} to @tt{.dvi} (i.e.,
while running @apl{tex}). These messages are @apl{tex}'s way of saying
that an internal area (typically for an index) is full. This is
normally because more indices were selected in the @tt{INDICES}
variable of the @tt{SETTINGS} file than the maximum number supported
by the installed version of @apl{tex}/@apl{texinfo} installations, as
mentioned in @ref{Generating a manual}. The easiest fix is to reduce
the number of indices generated.  Alternatively, it may be possible to
recompile your local @apl{tex}/@apl{texinfo} installation with a
higher number of indices.

@item Missing links in @apl{info} files (a section which exists in the
printed document cannot be accessed in the on-line document) can be
due to the presence of a colon (@tt{:}), a comma (@tt{,}), a double
dash (@tt{--}), or other such separators in a section name. Due to
limitations of @apl{info} section names cannot contain these symbols.

@item Menu listings in @apl{info} which @em{do not work} (i.e., the
menu listings are there, but they cannot be followed): see if they are
indented. In that case it is due to an @tt{itemize} or @tt{enumerate}
which was not closed.

@end{itemize}