dccproc(8) Distributed Checksum Clearinghouse dccproc(8) NNAAMMEE ddccccpprroocc -- Distributed Checksum Clearinghouse Procmail Interface SSYYNNOOPPSSIISS ddccccpprroocc [--VVddAAQQCCHHEERR] [--hh _h_o_m_e_d_i_r] [--mm _m_a_p] [--ww _w_h_i_t_e_c_l_n_t] [--TT _t_m_p_d_i_r] [--aa _I_P_-_a_d_d_r_e_s_s] [--ff _e_n_v___f_r_o_m] [--tt _t_a_r_g_e_t_s] [--xx _e_x_i_t_c_o_d_e] [--cc _t_y_p_e_,[_l_o_g_-_t_h_o_l_d_,]_r_e_j_-_t_h_o_l_d] [--gg [_n_o_t_-]_t_y_p_e] [--SS _h_e_a_d_e_r] [--ii _i_n_f_i_l_e] [--oo _o_u_t_f_i_l_e] [--ll _l_o_g_d_i_r] [--BB _d_n_s_b_l_-_o_p_t_i_o_n] [--LL _l_t_y_p_e_,_f_a_c_i_l_i_t_y_._l_e_v_e_l] DDEESSCCRRIIPPTTIIOONN DDccccpprroocc copies a complete SMTP message from standard input or a file to standard output or another file. As it copies the message, it computes the DCC checksums for the message, reports them to a DCC server, and adds a header line to the message. Another program such as procmail(1) can use the added header line to filter mail. Dccproc does not support any thresholds of its own, because equivalent effects can be achieved with regular expressions and you can apply dccproc several times using differ- ent DCC servers and then score mail based what all of the DCC servers say. Error messages are sent to stderr as well as the system log. Connect stderr and stdout to the same file to see errors in context, but direct stderr to /dev/null to keep DCC error messages out of the mail. The --ii option can also be used to separate the error messages. DDccccpprroocc sends reports of checksums related to mail received by DCC clients and queries about the total number of reports of particular checksums. A DCC server receives no mail, address, headers, or other information, but only cryptographically secure checksums of such informa- tion. A DCC server cannot determine the text or other information that corresponds to the checksums it receives. It only acts as a clearing- house of counts of checksums computed by clients. For the sake of privacy for even the checksums of private mail, the checksums of senders of purely internal mail or other mail that is known to not be unsolicited bulk can be listed in a whitelist to not be reported to the DCC server. When sendmail(8) is used, dccm(8) is a better DCC interface. Dccifd(8) is more efficient than ddccccpprroocc because it is a daemon, but that has costs in complexity. See dccsight(8) for a way to use previously computed checksums. OOPPTTIIOONNSS The following options are available: --VV displays the version of the DCC procmail(1) interface. --dd enables debugging output from the DCC client library. Additional --dd options increase the number of messages. One causes error messages to be sent to STDERR as well as the system log. --AA adds to existing X-DCC headers (if any) of the brand of the current server instead of replacing existing headers. --QQ only queries the DCC server about the checksums of messages instead of reporting and then querying. This is useful when ddccccpprroocc is used to filter mail that has already been reported to a DCC server by another DCC client such as dccm(8). No single mail message should be reported to a DCC server more than once per recipient. It is better to use _M_X_D_C_C lines in the global _w_h_i_t_e_c_l_n_t file for your MX servers --CC outputs only the X-DCC header and the checksums for the message. --HH outputs only the X-DCC header. --EE adds lines to the start of the log file turned on with --ll and --cc describing what might have been the envelope of the message. The information for the inferred envelope comes from arguments including --aa and headers in the message when --RR is used. No lines are gener- ated for which no information is available, such as the envelope recipient. --RR says the first Received lines have the standard "helo (name [address])..." format and the address is that of the SMTP client that would otherwise be provided with --aa. The --aa option should be used if the local SMTP server adds a Received line with some other format or does not add a Received line. Received headers specifying IP addresses marked _M_X or _M_X_D_C_C in the --ww _w_h_i_t_e_c_l_n_t file are skipped. --hh _h_o_m_e_d_i_r overrides the default DCC home directory, which is often /var/dcc. --mm _m_a_p specifies a name or path of the memory mapped parameter file instead of the default _m_a_p in the DCC home directory. It should be created with the nneeww mmaapp operation of the cdcc(8) command. --ww _w_h_i_t_e_c_l_n_t specifies an optional file containing SMTP client IP addresses and SMTP headers of mail that do not need X-DCC headers and whose check- sums should not be reported to the DCC server. It can also contain checksums of spam. If the pathname is not absolute, it is relative to the DCC home directory. Thus, individual users with private whitelists usually specify them with absolute paths. Common whitelists shared by users must be in the DCC home directory or one of its subdirectories and owned by the set-UID user of ddccccpprroocc. It is useful to _i_n_c_l_u_d_e a common or system-wide whitelist in private lists. The format of the ddccccpprroocc whiteclnt file is the same as the _w_h_i_t_e_l_i_s_t file required by dbclean(8) and dccm(8). Unlike dccm, the ddccccpprroocc _w_h_i_t_e_c_l_n_t file is optional. When --ww is not used, settings equivalent to these are used: _o_p_t_i_o_n _l_o_g_-_n_o_r_m_a_l _o_p_t_i_o_n _d_c_c_-_o_n _o_p_t_i_o_n _D_C_C_-_r_e_p_s_-_o_n _o_p_t_i_o_n _D_N_S_B_L_-_o_n When --ww is used, the defaults mentioned in dcc(8) are used. Those defaults differ and turn off DCC Reputations, and DNS blacklist (DNSBL )checking. Because the contents of the _w_h_i_t_e_c_l_n_t file are used frequently, a companion file is automatically created and maintained. It has the same pathname but with an added suffix of _._d_c_c_w. It contains a mem- ory mapped hash table of the main file. A local whitelist entry ("OK) or two or more semi-whitelistings ("OK2") for one of the message's checksums prevents all of the mes- sage's checksums from being reported to the DCC server and the addi- tion of a _X_-_D_C_C header line by ddccccpprroocc. Because it is run by or on behalf of a single user, ddccccpprroocc ignores _e_n_v___T_o entries in the _w_h_i_t_e_c_l_n_t file. Users who don't want to use ddccccpprroocc shouldn't. --TT _t_m_p_d_i_r changes the default directory for temporary files from the system default. The system default is often _/_t_m_p. --aa _I_P_-_a_d_d_r_e_s_s specifies the IP address (not the host name) of the immediately pre- vious SMTP client. It is often not available. --aa _0_._0_._0_._0 is ignored. --aa. The --aa option should be used instead of --RR if the local SMTP server adds a Received line with some other format or does not add a Received line. --ff _e_n_v___f_r_o_m specifies the RFC 821 envelope "Mail From" value with which the mes- sage arrived. It is often not available. If --ff is not present, the contents of the first Return-Path: or UNIX style From_ header is used. The _e_n_v___f_r_o_m string is often but need not be bracketed with "<>". --tt _t_a_r_g_e_t_s specifies the number of addressees of the message if other than 1. The string _m_a_n_y instead of a number asserts that there were too many addressees and that the message is unsolicited bulk email. --xx _e_x_i_t_c_o_d_e specifies the code or status with which ddccccpprroocc exits if the --cc thresholds are reached or the --ww _w_h_i_t_e_c_l_n_t file blacklists the mes- sage, unless the message is whitelisted. The default value is EX_NOUSER. EX_NOUSER is 67 on many systems. Use 0 to always exit successfully. --cc _t_y_p_e_,[_l_o_g_-_t_h_o_l_d_,]_r_e_j_-_t_h_o_l_d sets logging and "spam" thresholds for checksum _t_y_p_e. Each logged message placed in a separate file in the directory specified with --ll. The checksum types are _I_P, _e_n_v___F_r_o_m, _F_r_o_m, _M_e_s_s_a_g_e_-_I_D, _s_u_b_s_t_i_t_u_t_e, _R_e_c_e_i_v_e_d, _B_o_d_y, _F_u_z_1, _F_u_z_2, _r_e_p_-_t_o_t_a_l, and _r_e_p. The first six, _I_P through _s_u_b_s_t_i_t_u_t_e, have no effect except when a local DCC server configured with --KK is used. The _s_u_b_s_t_i_t_u_t_e thresholds apply to the first substitute heading encountered in the mail mes- sage. The string _A_L_L sets thresholds for all types, but is unlikely to be useful except for setting logging thresholds. The string _C_M_N specifies the commonly used checksums _B_o_d_y, _F_u_z_1, and _F_u_z_2. _R_e_j_-_t_h_o_l_d and _l_o_g_-_t_h_o_l_d must be numbers, the string _N_E_V_E_R, or the string _M_A_N_Y indicating millions of targets. Counts from the DCC server as large as the threshold for any single type are taken as sufficient evidence that the message should be logged or rejected. _L_o_g_-_t_h_o_l_d is the threshold at which messages are logged. It can be handy to log messages at a lower threshold to find solicited bulk mail sources such as mailing lists. Messages that reach at least one of their rejection thresholds or that have complicated combina- tions of white- and blacklisting are logged regardless of logging thresholds. _R_e_j_-_t_h_o_l_d is the threshold at which messages are considered "bulk," and so should cause the X-DCC header line to contain the string "bulk" or "bulk rep" and ddccccpprroocc to exit with the value set by --xx. DCC reputation thresholds in the commercial version of the DCC are controlled by thresholds on checksum types _r_e_p and _r_e_p_-_t_o_t_a_l. Mes- sages from an IP address that the DCC database says has sent more than _r_e_p_-_t_o_t_a_l _l_o_g_-_t_h_o_l_d messages are logged. A DCC reputation is computed for messages received from IP addresses that have sent more than _r_e_p_-_t_o_t_a_l _r_e_j_-_t_h_o_l_d messages. The DCC reputation of an IP address is the percentage of its messages that have been detected as bulk, or having at least 10 recipients. The defaults are equivalent to --cc _r_e_p_,_n_e_v_e_r and --cc _r_e_p_-_t_o_t_a_l_,_n_e_v_e_r_,_1_0. The checksums of locally white-listed messages are not checked with the DCC server and so only the number of targets of the current instance of a white-listed message are compared against the thresh- olds. The default is --cc _A_L_L_,_N_E_V_E_R, so that nothing is discarded or logged. A common choice is --cc _C_M_N_,_2_5_,_5_0 to reject or discard mail with com- mon bodies except as overridden by the whitelist of the DCC server and --gg and --ww. --gg [_n_o_t_-]_t_y_p_e indicates that white-listed, _O_K or _O_K_2, counts from the DCC server for a type of checksum are to be believed. They should be ignored if prefixed with _n_o_t_-. _T_y_p_e is one of the same set of strings as for --cc. Only _I_P, _e_n_v___F_r_o_m, and _F_r_o_m are likely choices. By default all three are honored, and hence the need for _n_o_t_-. --SS _h_d_r adds to the list of substitute or locally chosen headers that are checked with the --ww _w_h_i_t_e_c_l_n_t file and sent to the DCC server. The checksum of the last header of type _h_d_r found in the message is checked. As many as 6 different substitute headers can be speci- fied, but only the checksum of the first of the 6 will be sent to the DCC server. --ii _i_n_f_i_l_e specifies an input file for the entire message instead of standard input. If not absolute, the pathname is interpreted relative to the directory in which ddccccpprroocc was started. --oo _o_u_t_f_i_l_e specifies an output file for the entire message including headers instead of standard output. If not absolute, the pathname is inter- preted relative to the directory in which ddccccpprroocc was started. --ll _l_o_g_d_i_r specifies a directory for copies of messages whose checksum target counts exceed --cc thresholds. The format of each file is affected by --EE. If _l_o_g_d_i_r starts with _D_?, log files are put into subdirectories of the form _l_o_g_d_i_r_/_J_J_J where _J_J_J is the current julian day. _H_?_l_o_g_d_i_r puts logs files into subdirectories of the form _l_o_g_d_i_r_/_J_J_J_/_H_H where _H_H is the current hour. _M_?_l_o_g_d_i_r puts log files into subdirectories of the form _l_o_g_d_i_r_/_J_J_J_/_H_H_/_M_M where _M_M is the current minute. See the FILES section below concerning the contents of the files. See also the _o_p_t_i_o_n _l_o_g_-_s_u_b_d_i_r_e_c_t_o_r_y_-_{_d_a_y_,_h_o_u_r_,_m_i_n_u_t_e_} lines in _w_h_i_t_e_c_l_n_t files described in dcc(8). The directory is relative to the DCC home directory if it is not absolute --BB _d_n_s_b_l_-_o_p_t_i_o_n enables DNS blacklist checks of the SMTP client IP address, SMTP envelope Mail_From sender domain name, and of host names in URLs in the message body. Body URL blacklisting has too many false posi- tives to use on abuse mailboxes. It is less effective than greylisting with dccm(8) or dccifd(8) but can be useful in situa- tions where greylisting cannot be used. _D_n_s_b_l_-_o_p_t_i_o_n is either of the forms _s_e_t_:_o_p_t_i_o_n or _d_o_m_a_i_n[_,_I_P_a_d_d_r[_,_b_l_t_y_p_e]]. _D_o_m_a_i_n is a DNS blacklist domain such as example.com that will be searched. _I_P_a_d_d_r is the string "any" or the IP address in the DNS blacklist that indicates that the mail message is spam. 127.0.0.2 is assumed if _I_P_a_d_d_r is absent. IPv6 addresses can be specified with the usual colon (:) notation. Names can be used instead of numeric addresses. The type of DNS blacklist is specified by _b_l_t_y_p_e as _n_a_m_e, _I_P_v_4, or _I_P_v_6. Given an envelope sender domain name or a domain name in a URL of spam.domain.org and a blacklist of type _n_a_m_e, spam.domain.org.example.com will be tried. Blacklist types of _I_P_v_4 and _I_P_v_6 require that the domain name in a URL be resolved into an IPv4 or IPv6 address. The address is then written as a reversed string of decimal octets to check the DNS blacklist, as in 2.0.0.127.example.com, More than one blacklist can be specified. They are searched in order. All searching is stopped at the first positive result. Positive results are ignored after being logged unless an _o_p_t_i_o_n _D_N_S_B_L_-_o_n line appears in the global or per-user _w_h_i_t_e_c_l_n_t file. --BB _s_e_t_:_d_e_b_u_g_=_X sets the DNS blacklist logging level --BB _s_e_t_:_m_s_g_-_s_e_c_s_=_S limits ddccccpprroocc to _S seconds total for checking all DNS black- lists. The default is 25. --BB _s_e_t_:_U_R_L_-_s_e_c_s_=_S limits ddccccpprroocc to at most _S seconds resolving and checking any single URL. The default is 11. Some spam contains dozens of URLs and that some "spamvertised" URLs contain host names that need minutes to resolve. Busy mail systems cannot afford to spend minutes checking each incoming mail message. In order to use typical single-threaded DNS resolver libraries, dccm(8) and dccifd(8) use fleets of helper processes. --BB _s_e_t_:_n_o_-_e_n_v_e_l_o_p_e says that SMTP client IP addresses and sender Mail_From domain names should not be checked in the following blacklists. _s_e_t_:_e_n_v_e_l_o_p_e restores the default for subsequently named black- lists. --BB _s_e_t_:_n_o_-_b_o_d_y says that URLs in the message body should not be checked in the in the following blacklists. _s_e_t_:_b_o_d_y restores the default for later blacklists. --BB _s_e_t_:_n_o_-_M_X says MX servers of sender Mail_From domain names and host names in URLs should not be checked in the following blacklists. _s_e_t_:_M_X restores the default. --BB _s_e_t_:_n_o_-_N_S says NS servers of sender Mail_From domain names and host names in URLs should not be checked in the following blacklists. _s_e_t_:_N_S restores the default. --LL _l_t_y_p_e_,_f_a_c_i_l_i_t_y_._l_e_v_e_l specifies how messages should be logged. _L_t_y_p_e must be _e_r_r_o_r or _i_n_f_o to indicate which of the two types of messages are being con- trolled. _L_e_v_e_l must be a syslog(3) level among _E_M_E_R_G, _A_L_E_R_T, _C_R_I_T, _E_R_R, _W_A_R_N_I_N_G, _N_O_T_I_C_E, _I_N_F_O, and _D_E_B_U_G. _F_a_c_i_l_i_t_y must be among _A_U_T_H, _A_U_T_H_P_R_I_V, _C_R_O_N, _D_A_E_M_O_N, _F_T_P, _K_E_R_N, _L_P_R, _M_A_I_L, _N_E_W_S, _U_S_E_R, _U_U_C_P, and _L_O_C_A_L_0 through _L_O_C_A_L_7. The default is equivalent to --LL _i_n_f_o_,_M_A_I_L_._N_O_T_I_C_E --LL _e_r_r_o_r_,_M_A_I_L_._E_R_R Something like this turns off the log messages: --LL _n_o_t_i_c_e_,_M_A_I_L_._d_e_b_u_g --LL _e_r_r_o_r_,_M_A_I_L_._D_E_B_U_G ddccccpprroocc exits with 0 on success and with the --xx value if the --cc thresh- olds are reached or the --ww _w_h_i_t_e_c_l_n_t file blacklists the message. If at all possible, the input mail message is output to standard output or the --oo _o_u_t_f_i_l_e despite errors. If possible, error messages are put into the system log instead of being mixed with the output mail message. The exit status is zero for errors so that the mail message will not be rejected. If ddccccpprroocc is run more than 500 times in fewer than 5000 seconds, ddccccpprroocc tries to start Dccifd(8). The attempt is made at most once per hour. Dccifd is significantly more efficient than ddccccpprroocc. With luck, mecha- nisms such as SpamAssassin will notice when dccifd is running and switch to dccifd. FFIILLEESS /var/dcc DCC home directory in which other files are found. map memory mapped file in the DCC home directory of information concerning DCC servers. whiteclnt contains the client whitelist in the format described in dcc(8). whiteclnt.dccw is a memory mapped hash table corresponding to the _w_h_i_t_e_c_l_n_t file. tmpdir contains temporary files created and deleted as ddccccpprroocc pro- cesses the message. logdir is an optional directory specified with --ll and containing marked mail. Each file in the directory contains one message, at least one of whose checksums reached one of its --cc thresh- olds. The entire body of the SMTP message including its header is followed by the checksums for the message. EEXXAAMMPPLLEESS The following procmailrc(5) rule adds an X-DCC header to passing mail :0 f | /usr/local/bin/dccproc -ERw whiteclnt This procmailrc(5) recipe rejects mail with total counts of 10 or larger for the commonly used checksums: :0 fW | /usr/local/bin/dccproc -ERw whiteclnt -ccmn,10 :0 e { EXITCODE=67 :0 /dev/null } SSEEEE AALLSSOO cdcc(8), dcc(8), dbclean(8), dccd(8), dblist(8), dccifd(8), dccm(8), dccsight(8), mail(1), procmail(1). HHIISSTTOORRYY Implementation of ddccccpprroocc was started at Rhyolite Software in 2000. This describes version 1.3.50. BBUUGGSS ddccccpprroocc uses --cc where dccm(8) uses --tt. February 5, 2007