=pod

=head1 NAME

B<rwset> - Generate binary IPset files of unique IP addresses

=head1 SYNOPSIS

  rwset [--sip-file=FILE] [--dip-file=FILE] [--nhip-file=FILE]
        [--compression-method=COMP_METHOD] [--print-filenames]
        [--copy-input=PATH] [--site-config-file=FILENAME] [FILES...]


DEPRECATED:

  rwset [{ --daddress | --saddress }]
        { --print-ips
          | --integer-ips
          | --set-file=<output-filename>}
        [FILES...]

=head1 DESCRIPTION

B<rwset> reads SiLK Flow data from the standard input, a named pipe,
or named input files and generates binary IPset file(s) containing the
list of the unique source IP addresses, destination IP addresses,
and/or next-hop IP addresses.

=head1 OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an
exact match for an option.  A parameter to an option may be specified
as B<--arg>=I<param> or B<--arg> I<param>, though the first form is
required for options that take optional parameters.

One of the following output switches is required:

=over 4

=item B<--sip-file>=I<FILE>

Store the unique source IP addresses in the binary IPset file I<FILE>.
B<rwset> will write the IPset file to the standard output when I<FILE>
is C<stdout> and the standard output is not a terminal.

=item B<--dip-file>=I<FILE>

Store the unique destination IP addresses in the binary IPset file
I<FILE>.  B<rwset> will write the IPset file to the standard output
when I<FILE> is C<stdout> and the standard output is not a terminal.

=item B<--nhip-file>=I<FILE>

Store the unique next-hop IP addresses in the binary IPset file
I<FILE>.  B<rwset> will write the IPset file to the standard output
when I<FILE> is C<stdout> and the standard output is not a terminal.

=back

Only one of the above switches my use C<stdout> as the name of the
file.

B<rwset> supports these additional switches:

=over 4

=item B<--compression-method>=I<COMP_METHOD>

Set the compression method of the output to I<COMP_METHOD>.  Some SiLK
tools can use an external library to compress their binary output.
The list of available compression methods and the default method are
set when SiLK is compiled (the B<--help> and B<--version> switches
print the available and default compression methods) and depend on
which supported libraries are found.  SiLK can support:

=over 4

=item none

Do not compress the output using an external library

=item zlib

Use the B<zlib(3)> library for compressing the output

=item lzo1x

Use the I<lzo1x> algorithm from the LZO real time compression library
for compression

=item best

Use whichever available method gives the C<best> compression in
general, though not necessarily the C<best> for this particular
output.

=back

=item B<--print-filenames>

Prints to the standard error the names of input files as they are
opened.

=item B<--copy-input>=I<PATH>

Copy all binary input to the specified file or named pipe.  I<PATH>
can be C<stdout> to print flows to the standard output as long as no
IPset files are being written there.

=item B<--site-config-file>=I<FILENAME>

Read the SiLK site configuration from the named file I<FILENAME>.
When this switch is not provided, the location specified by the
C<$SILK_CONFIG_FILE> environment variable is used if that variable is
not empty.  The value of C<$SILK_CONFIG_FILE> should include the name
of the file.  Otherwise, the application looks for a file named
F<silk.conf> in the following directories: the directory specified in
the F<$SILK_DATA_ROOTDIR> environment variable; the data root
directory that is compliled into SiLK (use the B<--version> switch to
view this value); the directories F<$SILK_PATH/share/silk/>,
F<$SILK_PATH/share/>, or F<$SILK_PATH>; or in the F<share/silk/> and
F<share/> directories parallel to the application's directory.

=back

The above output switches allow the user to create multiple IPset
files in a single data pass.  If none of those switches are given,
B<rwset> will fall back to its legacy behavior and generate a single
set of IPs, as determined by the following two deprecated switches.
If neither switch is given, the source addresses are used:

=over 4

=item B<--saddress>

Generate the set from the source addresses.  DEPRECATED: Use the
B<--sip-file> switch instead.

=item B<--daddress>

Generate the set from destination addresses.  DEPRECATED: Use the
B<--dip-file> switch instead.

=back

The following legacy switches determine what to do with the IP list
generated by the B<--saddress> or B<--daddress> switch:

=over 4

=item B<--print-ips>

Print IP addresses to stdout.  DEPRECATED: Use B<rwsetcat(1)> to print
the contents of the IPset.

=item B<--integer-ips>

Enable the B<--print-ips> option, but print the IPs as integers
instead of as dotted decimal.  DEPRECATED: Use B<rwsetcat(1)> to print
the contents of the IPset.

=item B<--set-file>=I<FILE>

Write the IP addresses to a binary IPset file named FILE.
DEPRECATED: Use the B<--sip-file> or B<--dip-file> switch
instead.

=back

=head1 EXAMPLES

rwset is intended to work tightly with rwfilter.  For example,
consider generating two sets: one of all IP addresses which have 3
packets or less in their sessions, and one with four or more packets.

The first set, for TCP traffic on 03/01/2003 can be generated
with:

  rwfilter --start-date=2003/03/01:00 --end-date=2003/03/01:23 \
        --proto=6 --packets=1-3 --pass=stdout | \
        rwset --sip-file=low_packet_tcp.set

The second set with:

  rwfilter --start-date=2003/03/01:00 --end-date=2003/03/01:23
        --proto=6 --packets=4-9999999 --pass=stdout | \
        rwset --sip-file=high_packet_tcp.set

=head1 NOTES

IPset files are in a binary format.  To create a binary IPset file
from a list of dotted decimal IP addresses, use B<rwsetbuild(1)>.

Use B<rwsetcat(1)> to see the IP addresses in a binary set file.

B<rwsetintersect(1)> and B<rwsetunion(1)> will allow you to perform
set operations on binary IPset files.

The command:

  rwset --sip-file=stdout | rwsetcat

will be faster than B<rwuniq>, but cannot do the thresholding that
B<rwuniq> does.

B<rwset> uses B<rwfilter> as an input, so anything that B<rwfilter>
passes will become a member of a set (in other words, there's no
guarantee that a member of F<low_packet_tcp.set> from above didn't
have 4 packets or more in a session, just that there exists at least 1
session for which it had 3 packets or less).

=head1 SEE ALSO

B<rwfilter(1)>, B<rwuniq(1)>, B<rwsetbuild(1)>, B<rwsetcat(1)>,
B<rwsetintersect(1)>, B<rwsetunion(1)>

=cut

$SiLK: rwset.pod 8272 2007-08-03 19:22:40Z mthomas $

Local Variables:
mode:text
indent-tabs-mode:nil
End: