=pod =head1 NAME B - Configuration file for sensors and probes =head1 DESCRIPTION To allow for easier configuration, the B and B daemons read a text file containing information regarding the sensors and probes the daemon is processing. This manual page describes the syntax of this Sensor Configuration file. The Sensor Configuration file tries to satisfy the following goals: =over 4 =item 1 to tell the collection daemon (B or B) how to collect the flow data (e.g., from a network socket, a UNIX domain socket, or a file) =item 2 to tell B how to categorize the data (e.g., as incoming or outgoing) once it has been collected =back A I represents an object that generates a stream of flow-data. The object could be a YAF sensor listening to live traffic, a Cisco router producing S, or software creating flow information from packet capture (B-like) data. Probes can be grouped into a logical I. A sensor is one or more probes that, for the purpose of analysis, all have the same name. While probes are defined only in the Sensor Configuration file, sensors should be defined in the I Configuration file, B. While it is possible to define sensors in the Sensor Configuration file, the analysis tools (e.g., B) will not be aware of these sensors. The syntax of the Sensor Configuration file is simple. Blank lines are ignored. Comments, which begin with the C<#> character and continue to the end of the line, are also ignored. Every other line should contain a key-value pair, where the key and value are separated by white space. For a key that accepts multiple values, the values must be separated by white space and/or a comma. Repeating a key will overwrite the key's previous value and generate a warning message. Lines are grouped together into blocks. There are two types of blocks: B, which defines a new sensor, and B, which defines a new probe. The keywords C and C introduce a new block which continues until the beginning of the next block. Each key-value pair within the block sets an attribute on the sensor or probe. =head2 Sensor Block The following is an example of a sensor block: sensor S01 class all The sensor block creates a temporary sensor to be used by this invocation of B or B. It contains the sensor name and its class. As stated previously, the analysis tools are not aware of sensors that are created in the Sensor Configuration file, and their use is discouraged. A sensor block is most useful during testing. The following are legal in a sensor block: =over 4 =item C The C keyword closes the previous sensor or sensor-probe block and defines a new sensor. It is followed by the name of the sensor being defined; this name will be used to generate the names of the data files. =item C The C keyword sets the class that the sensor belongs to. In the above example, sensor C belongs to the class C. Saying a sensor I a class means data collected by that sensor will be considered whenever the name of the class is passed to B. The name of the class must be one of the classes defined in the B Site Configuration file. (In addition, the B and B daemons must have been aware of the class when they were compiled.) =back =head2 Sensor-Probe Block A sample sensor-probe block showing all possible attributes (but that is illegal because of conflicting attributes) is shown here. (For valid sensor-probe blocks, see the L section below.) sensor-probe S01 probe-name netflow probe-type netflow priority 8 log-flags none all bad missing listen-on-port 9999 protocol udp listen-as-host 192.168.1.1 listen-on-unix-domain-socket /tmp/sock read-from-file /var/tmp/flow-file accept-from-host 172.16.22.22 null-index 0 isp-ip 10.10.10.10,10.10.10.11 external-index 1,2,3,4 internal-index 8,9 null-ipblock 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16 internal-ipblock 128.2.0.0/16 external-ipblock remainder The first set of values describes the probe: =over 4 =item C The C keyword closes the previous sensor or sensor-probe block and begins a new sensor-probe block. The value is the name of the sensor for which this probe is a flow collector. The sensor name must be known, either through definition in the B Site Configuration file or from a sensor block previously defined in this Sensor Configuration file. =item C If this attribute is not present, the name of the probe will be the same as the name of the sensor (i.e., the value given on the sensor-probe line). Note that the probes belonging to a single sensor must each have a unique name. We recommend setting the probe-name to the probe-type. This attribute is used by B to generate file names. When B reads these data files, the attribute is used to map from the file name back to the probe. When B reads NetFlow records directly, this attribute is only used in error messages. =item C The following probe-types are defined: =over 4 =item I A NetFlow probe processes NetFlow V5 protocol data units (PDU) that the daemon reads from a UDP port or from a file. NetFlow may be generated by a router or by software that reads packet capture (B) data and generates NetFlow V5 records. =item I An IPFIX probe processes Internal Protocol Flow Information eXchange records that the daemon reads from an IPFIX source such as YAF. To support IPFIX probes, SiLK must be built with support for the external library libfixbuf. Both YAF and libfixbuf are available from L. =cut =pod =back When the C attribute is not specified, the probe is considered a C probe. =item C The priority is an integer value between 1 and 100, inclusive. Values of 1-50 are considered low priority, 51-100 are high. B accumulates flow data from probes into files, and these files are closed after a certain amount of time. Closed high-priority files are always sent to a flowcap client before closed low-priority files. B does not use this attribute. =item C This attribute gives specific logging instructions for the probe. Some probe-types will keep track of the sequence numbers in the packets they receive; these probes will write messages to the log file about missing or bad packets. By default, all messages are written by the probes that keep track of this information, but sometimes it is useful to reduce the verbosity of this information or eliminate it entirely. The C attribute accepts a list of the following values: =over 4 =item I Log nothing. =item I Log all the conditions listed below; this is the default. =item I Write messages about bad packets. An example of a bad packet is a NetFlow v5 PDU that reports no flow records. =item I Examine the sequence numbers and write messages about missing packets. =back =back Next, we specify how the probe collects data. For each probe, one and only one the following must be specified: =over 4 =item C This attribute instructs B or B to listen on the specified network port in order to collect flow data. When listening to NetFlow from a Cisco router, this is the I given by the Cisco ISO command ip flow-export [ip-address] [port] =item C =cut =pod The value contains the path name to a UNIX domain socket. Currently no probes use this collection method. =cut =pod =item C When this attribute is given, B will read PDU records from the specified path on the file system. B does not use this value. The file's length should be an integer multiple of 1464 bytes, where 1464 is the maximum length of the NetFlow v5 PDU. Each 1464 block should contain the 24-byte NetFlow v5 header and space for thirty 48-byte flow records, even if fewer NetFlow records are valid. You can override this attribute by giving the B<--netflow-file> switch to B, but the attribute must still be present in the Sensor Configuration file. In this case, we recommend you use C as the file name. =back When the daemon is listening on an IP socket (C), the following attributes may be specified: =over 4 =item C This attribute specifies whether the port on which to listen is an C, C, or C port. The attribute must be specified for IPFIX probe-types, as they have no default protocol. The default (and only permitted value) for all other probe-types is C. =item C This attribute contains the IP or name of the host that is allowed to connect to the socket. When this attribute is not present, any host may connect. When listening for NetFlow, this attribute would be the IP address of the router as seen from the machine running B or B. =item C This attribute is used on a multi-homed machine to specify which address to listen on. Its value is a network IP address in dotted-decimal notation. If not present, the program will listen on all the machine's addresses. For listening to NetFlow, the value would be the I given by Cisco ISO command ip flow-export [ip-address] [port] =back The following attributes are used by B to determine in which file a record is stored (i.e., whether the flow is categorized as incoming, outgoing, or dropped [null]). The details are described below in the L section. Even though B does not use these attributes, any required attributes must be present so that the probe may be verified. You may categorize any flow, regardless of its probe-type, based on the IP addresses it contains and the IP space of the monitored network. Flows collected by a NetFlow probe may instead be categorized using the SNMP interface information in the flow. Every probe requires you to specify either the IP space outside the monitored network (C) or, for NetFlow probes, the SNMP interfaces that face outside the network (C). In addition to specifying the external side, you may specify the values on the internal side (C or C) and the values that represent nonrouted flows (C or C). It is an error to specify both the IP space and SNMP interfaces on a probe. The keyword C may be used for I CIDR block list or SNMP interface list. When C is specified, that CIDR block list or SNMP interface list is set to all values not seen in the other two lists. =over 4 =item C The value of this attribute is a list of CIDR blocks specifying the IP-space outside of an organization, or the keyword C. Every sensor-probe must have either this attribute or the C attribute. =item C The value of this attribute is a list of CIDR blocks specifying the internal IP space of an organization, or the keyword C. If not explicitly specified, it is set all CIDR blocks not listed in the C and C attributes. =item C The value of this attribute is a list of CIDR blocks denoting non-routed packets, or the keyword C. =item C The value for this attribute is a list of positive integers that represent the SNMP interface index(es) that connect to external routers or machines, where I means outside the administrative domain being instrumented, or the keyword C. For a border router, the external-indexes would be those connected to the ISP. Either this attribute or the C attribute is required. =cut =pod =item C The value for this attribute is a list of positive integers that represent the SNMP interface index(es) that connect to the internal network, or the keyword C. If not explicitly specified, it is set all SNMP interfaces not listed in the C and C attributes. =cut =pod =item C The value for this attribute is the SNMP index of the output interface used by the router to denote a not-routed packet. A packet may be not-routed because the packet is meant for the router itself (e.g., a BGP message) or because it matches an ACL violation. If this attribute is not provided, 0 is used as the default, which is the value used by Cisco routers to denote a null flow. =item C =cut =pod This attribute is currently unused. =cut =pod =back =head1 PACKING LOGIC B uses the above values to determine where on disk to store a flow. This packing logic depends on the site that was used when SiLK was configured. This section describes the how a flow's category or I is set for the default SiLK site, B, using IP-based splitting. (For SNMP-based splitting, the logic is identical, except the SNMP interfaces are checked.) =over 4 =item ext2ext Contains flows where both the source IP and destination IP are listed in the external-ipblock. =item innull Contains flows where the source IP is listed in the external-ipblock and the destination IP is listed in the null-ipblock. =item in,inweb Contains flows where the source IP is listed in the external-ipblock and the destination IP is listed in the internal-ipblock. A flow's type is B if the protocol is TCP and either the source port or destination port is one of 80, 443, 8080; otherwise it is B. =item int2int Contains flows where both the source IP and destination IP are listed in the internal-ipblock. =item outnull Contains flows where the source IP is listed in the internal-ipblock and the destination IP is listed in the null-ipblock. =item out,outweb Contains flows where the source IP is listed in the internal-ipblock and the destination IP is listed in the external-ipblock. =item other Contains flows where either =over 4 =item * the source IP is not listed in the external-ipblock or internal-ipblock (note that the source could be listed in the null-ipblock) =item * the destination IP is not listed in the external-ipblock, internal-ipblock, or null-ipblock =back =back =head1 EXAMPLES =head2 Example A A probe that listens on a UDP port NetFlow from a router and uses the SNMP interfaces to categorize the flows: sensor-probe NF-UDP-SNMP probe-name netflow # optional but recommended probe-type netflow # default value priority 8 # optional log-flags all # default value listen-on-port 9999 accept-from-host 172.16.22.22 # optional; default: all hosts listen-as-host 192.168.1.1 # optional; default: all addrs null-index 0 # default value external-index 1,2,3,4 internal-index 8,9 # line is ignored =head2 Example B The same probe but categorizing the flows based on the IP addresses: sensor-probe NF-UDP-CIDR probe-name netflow # optional but recommended probe-type netflow # default value priority 8 # optional listen-on-port 9999 accept-from-host 172.16.22.22 # optional; default: all hosts listen-as-host 192.168.1.1 # optional; default: all addrs internal-ipblock 128.2.0.0/16 external-ipblock remainder =head2 Example C Instead of listening on a UDP port, suppose those probes processed files containing NetFlow v5 PDUs: sensor-probe NF-FILE-SNMP probe-name netflow # optional but recommended probe-type netflow # default value priority 8 # optional log-flags bad # ignore missing pkts read-from-file /dev/null # use --netflow-file= null-index 0 # default value external-index 1,2,3,4 internal-index 8,9 # line is ignored sensor-probe NF-FILE-CIDR probe-name netflow # optional but recommended probe-type netflow # default value priority 8 # optional log-flags bad # ignore missing pkts read-from-file /dev/null # use --netflow-file= internal-ipblock 128.2.0.0/16 external-ipblock remainder =head2 Example D A IPFIX probe can listen on a TCP or UDP socket; there is no SNMP information, so CIDR blocks must be used to categorize the flows: sensor-probe IPFIX-TCP-CIDR probe-name ipfix # optional but recommended probe-type ipfix priority 8 # optional listen-on-port 9999 protocol tcp accept-from-host 172.16.22.22 # optional; default: all hosts listen-as-host 192.168.1.1 # optional; default: all addrs internal-ipblock 128.2.0.0/16 external-ipblock remainder =cut =pod =head1 SEE ALSO B, B, B =cut $SiLK: sensor.conf.pod 6982 2007-04-19 15:07:52Z mthomas $ Local Variables: mode:text indent-tabs-mode:nil End: