unpaper 0.2 - written by Jens Gulden 2005, 2006 (build 2006-05-09 19:22:15) Licensed under the GNU General Public License (GPL). This software comes with no warranty. ------------------------------------------------------------------------------ Overview -------- unpaper is a post-processing tool for scanned sheets of paper, especially for book pages that have been scanned from previously created photocopies. The main purpose is to make scanned book pages better readable on screen after conversion to PDF. Additionally, unpaper might be useful to enhance the quality of scanned pages before performing optical character recognition (OCR). unpaper tries to clean scanned images by removing dark edges that appeared through scanning or copying on areas outside the actual page content (e.g. dark areas between the left-hand-side and the right-hand-side of a double- sided book-page scan). The program also tries to detect disaligned centering and rotation of pages and will automatically straighten each page by rotating it to the correct angle. This process is called "deskewing". Note that the automatic processing will sometimes fail. It is always a good idea to manually control the results of unpaper and adjust the parameter settings according to the requirements of the input. Each processing step can also be disabled individually for each sheet. Input and output files can be in either .pbm , .pgm or .ppm format, thus generally in .pnm format, as also used by the Linux scanning tools scanimage and scanadf. Conversion to PDF can e.g. be achieved with the Linux tools pgm2tiff, tiffcp and tiff2pdf. Installation ------------ On i386-Linux systems, the pre-compiled binary executable included in the distribution archive can be used. Simply copy the file 'unpaper' to /usr/bin or another directory in your executable search path. Compilation ----------- unpaper has been tested to be compilable on several hardware platforms and operating systems, including Linux/Intel, Macintosh/G4, Macintosh/68k, Solaris/Sparc and Windows/Intel. Linux/Intel The binary executable included in the distribution archive has been compiled using gcc version 4.1: gcc -D TIMESTAMP="" -lm -O3 -funroll-all-loops -fomit-frame-pointer -ftree-vectorize -o unpaper unpaper.c Compilation hints for other environments have been provided by users. Note that the following compiler invocations might work only with specific versions of the compilers or specific environments: Macintosh gcc -O4 -fast -mcpu=7450 -pipe -fomit-frame-pointer -ftree-vectorize -o unpaper unpaper.c (People with G5 should simply omit the -mcpu=7450 option. Thanks to Olaf Marzocchi.) In case you use cc as compiler, it may be necessary to increase the stack size by adding -Wl,-stack_size -Wl,1000000 (tested on Macintosh/68k). Windows gcc.exe unpaper.c -o unpaper.exe -I "C:\programme\Dev-Cpp\include" -L "C:\programme\Dev-Cpp\lib" (Thanks to Uwe Dierolf.) Usage ----- Usage: unpaper [options] Filenames may contain a formatting placeholder starting with '%' to insert a page counter for multi-page processing. E.g.: 'scan%03d.pbm' to process files scan001.pbm, scan002.pbm, scan003.pbm etc. -l --layout single Set default layout options for a sheet: |single-rotated 'single': One page per sheet, oriented |double upwards without rotation. |double-rotated 'single-rotated': One page per sheet, rotated anti-clockwise. 'double': Two pages per sheet, landscape orientation (one page on the left half, one page on the right half). 'double-rotated': Two pages per sheet, rotated anti-clockwise (i.e. the top-sides of the pages are heading leftwards on the unrotated sheet). Using this option automatically sets the --mask-scan-point and maybe --pre/post- rotate options. -start --start-sheet Number of first sheet to process in multi- sheet mode. (default: 1) -end --end-sheet Number of last sheet to process in multi- sheet mode. -1 indicates processing until no more input file with the corresponding page number is available (default: -1) -# --sheet Optionally specifies which sheets to {,[-]} process in the range between start-sheet and end sheet. -x --exclude Excludes sheets from processing in the {,[-]} range between start-sheet and end-sheet. --pre-rotate -90|90 Rotates the whole image clockwise (90) or or anti-clockwise (-90) before any other processing. --post-rotate -90|90 Rotates the whole image clockwise (90) or or anti-clockwise (-90) after any other processing. -M --pre-mirror Mirror the image, after possible pre- [v[ertical]][,][h[orizontal]] rotation. Either 'v' (for vertical mirroring), 'h' (for horizontal mirroring) or 'v,h' (for both) can be specified. --post-mirror Mirror the image, after any other [v[ertical]][,][h[orizontal]] processing except possible post- rotation. --pre-wipe Manually wipe out an area before further ,,, processing. Any pixel in a wiped area will be set to white. Multiple areas to be wiped may be specified by multiple occurrences of this options. --post-wipe Manually wipe out an area after ,,, processing. Any pixel in a wiped area will be set to white. Multiple areas to be wiped may be specified by multiple occurrences of this options. --pre-border Clear the border-area of the sheet before ,,, further processing. Any pixel in the border area will be set to white. --post-border Clear the border-area after processing. ,,, Any pixel in the border area will be set to white. --pre-mask ,,, Specify masks to apply before any other processing. Any pixel outside a mask will be set to white, unless another mask includes this pixel. Only pixels inside a mask will remain. Multiple masks may be specified. No deskewing will be applied to the masks specified by --pre-mask. -s --size , Change the sheet size before other pro- | cessing is applied. Content on the sheet gets zoomed to fit to the appropriate size, but the aspect ratio is preserved. Instead, if the sheets's aspect ratio changes, the zoomed content gets centered on the sheet. Size-name can also be a standard name as 'a4', 'letter', etc. Possible size names are: a5 a4 a3 letter legal. All size names can also be applied in rotated landscape orientation, use 'a4-landscape', 'letter-landscape' etc. --post-size ,| Change the sheet size preserving the content's aspect ratio after other processing steps are applied. --stretch ,| Change the sheet size before other processing is applied. Content on the sheet gets stretched to the specified size, possibly changing the aspect ratio. --post-stretch , Change the sheet size after other | processing is applied. Content on the sheet gets stretched to the specified size, possibly changing the aspect ratio. -z --zoom Change the sheet size according to the given factor before other processing is done. --post-zoom Change the sheet size according to the given factor after processing is done. -bn --blackfilter-scan-direction Directions in which to search for solidly [v[ertical]][,][h[orizontal]] black areas. Either 'v' (for vertical scanning), 'h' (for horizontal scanning) of 'v,h' (for both) can be specified. (default: 'v,h') -bs --blackfilter-scan-size Width of virtual bar used for mask |, detection. Two values may be specified to individually set horizontal and vertical size. (default: 20,20) -bd --blackfilter-scan-depth Size of virtual bar used for black area | detection. (default: 500,500) -bp --blackfilter-scan-step Steps to move virtual bar for black area | detection. (default: 5,5) -bt --blackfilter-scan-threshold Ratio of dark pixels above which a black area gets detected. (default: 0.95). -bx --blackfilter-scan-exclude Area on which the blackfilter should not ,,, operate. This can be useful to prevent the blackfilter from working on inner page content. May be specified multiple times to set more than one area. -bi --blackfilter-intensity Intensity with which to delete black areas. Larger values will leave less noise-pixels around former black areas, but may delete page content. (default: 20) -ni --noisefilter-intensity Intensity with which to delete individual pixels or tiny clusters of pixels. Any cluster which only contains n dark pixels together will be deleted. (default: 4) -ls --blurfilter-size Size of blurfilter area to search for |, 'lonely' clusters of pixels. (default: 100,100) -lp --blurfilter-step Size of 'blurring' steps in each |, direction. (default: 50,50) -li --blurfilter-intensity Relative intensity with which to delete tiny clusters of pixels. Any blurred area which contains at most the ratio of dark pixels will be cleared. (default: 0.01) -gs --grayfilter-size Size of grayfilter mask to search for |, 'gray-only' areas of pixels. (default: 50,50) -gp --grayfilter-step Size of steps moving the grayfilter mask |, in each direction. (default: 20,20) -gt --grayfilter-threshold Relative intensity of grayness which is accepted before clearing the grayfilter mask in cases where no black pixel is found in the mask. (default: 0.5) -p --mask-scan-point , Manually set starting point for mask- detection. Multiple --mask-scan-point options may be specified to detect multiple masks. -m --mask ,,, Manually add a mask, in addition to masks automatically detected around the --mask- scan-point coordinates (unless --no-mask- scan is specified). Any pixel outside a mask will be set to white, unless another mask covers this pixel. -mn --mask-scan-direction Directions in which to search for mask [v[ertical]][,][h[orizontal]] borders, starting from --mask-scan-point coordinates. Either 'v' (for vertical scanning), 'h' (for horizontal scanning) of 'v,h' (for both) can be specified. (default: 'h' ('v' may cut text- paragraphs on single-page sheets)) -ms --mask-scan-size | Width of the virtual bar used for mask detection. Two values may be specified to individually set horizontal and vertical size. (default: 50,50) -md --mask-scan-depth | Height of the virtual bar used for mask detection. (default: -1,-1, using the total width or height of the sheet) -mp --mask-scan-step | Steps to move the virtual bar for mask detection. (default: 5,5) -mt --mask-scan-threshold | Ratio of dark pixels below which an edge gets detected, relative to max. blackness when counting from the start coordinate heading towards one edge. (default: 0.1) -mm --mask-scan-minimum , Minimum allowed size of an auto-detected mask. Masks detected below this size will be ignored and set to the size specified by mask-scan-maximum. (default: 100,100) -mM --mask-scan-maximum , Maximum allowed size of an auto-detected mask. Masks detected above this size will be shrunk to the maximum value, each direction individually. (default: sheet size, or page size derived from --layout option. -mc --mask-color Color value with which to wipe out pixels not covered by any mask. Maybe useful for testing in order to visualize the effect of masking. (Note that an RGB-value is expected: R*65536 + G*256 + B.) -dn --deskew-scan-direction Edges from which to scan for rotation. [left],[top],[right],[bottom] Each edge of a mask can be used to detect the mask's rotation. If multiple edges are specified, the average value will be used, unless the statistical deviation exceeds --deskew-scan-deviation. Use 'left' for scanning from the left edge, 'top' for scanning from the top edge, 'right' for scanning from the right edge, 'bottom' for scanning from the bottom. Multiple directions can be seperated by commas. (default: 'left,right') -ds --deskew-scan-size Size of virtual line for rotation detection. (default: 1500) -dd --deskew-scan-depth Amount of dark pixels to accumulate until scanning is stopped, relative to scan-bar size. (default: 0.5) -dr --deskew-scan-range Range in which to search for rotation, from -degrees to +degrees rotation. (default: 5.0) -dp --deskew-scan-step Steps between single rotation-angle detections. Lower numbers lead to better results but slow down processing. (default: 0.1) -dv --deskew-scan-deviation Maximum statistical deviation allowed among the results from detected edges. No rotation if exceeded. (default: 1.0) -W --wipe Manually wipe out an area. Any pixel in ,,, a wiped area will be set to white. Multiple --wipe areas may be specified. This is applied after deskewing and before automatic border-scan. -mw --middle-wipe If --layout is set to 'double', this |, may specify the size of a middle area to wipe out between the two pages on the sheet. This may be useful if the blackfilter fails to remove some black areas (e.g. resulting from photo-copying in the middle between two pages). -B --border Manually add a border. Any pixel in the ,,, border area will be set to white. This is applied after deskewing and before automatic border-scan. -Bn --border-scan-direction Directions in which to search for outer [v[ertical]][,][h[orizontal]] border. Either 'v' (for vertical scanning), 'h' (for horizontal scanning) of 'v,h' (for both) can be specified. (default: 'v') -Bs --border-scan-size | Width of virtual bar used for border detection. Two values may be specified to individually set horizontal and vertical size. (default: 5,5) -Bp --border-scan-step | Steps to move virtual bar for border detection. (default: 5,5) -Bt --border-scan-threshold Absolute number of dark pixels covered by the border-scan mask above which a border is detected. (default: 5) -Ba --border-align Direction where to shift the detected [left],[top],[right],[bottom] border-area. Use --border-margin to specify horizontal and vertical distances to be kept from the sheet-edge. (default: none) -Bm --border-margin Distance to keep from the sheet edge when , aligning a border area. May use measurement suffices such as cm, in. -w --white-threshold Brightness ratio above which a pixel is considered white. (default: 0.9) -b --black-threshold Brightness ratio below which a pixel is considered black (non-gray). This is used by the gray-filter. This value is also used when converting a grayscale image to black-and-white mode (default: 0.33) -ip --input-pages 1|2 If '2' is specified, read two input images instead of one and internally combine them to a double-layouted sheet before further processing. Before internally combining, --pre- rotation is optionally applied individually to both input images as the very first processing steps. -op --output-pages 1|2 If '2' is specified, write two output images instead of one, as a result of splitting a double-layouted sheet after processing. After splitting the sheet, --post-rotation is optionally applied individually to both output images as the very last processing step. -S --sheet-size , Force a fix sheet size. Usually, the | sheet size is determined by the input image size (if input-pages=1), or by the double size of the first page in a two-page input set (if input-pages=2). If the input image is smaller than the size specified here, it will appear centered and surrounded with a white border on the sheet. If the input image is bigger, it will be centered and the edges will be cropped. This option may also be helpful to get regular sized output images if the input image sizes differ. Standard size-names like 'a4-landscape', 'letter', etc. may be used (see --size). (default: as in input file) --no-blackfilter Disables black area scan. Individual sheet {,[-]} indices can be specified. --no-noisefilter Disables noisefilter. Individual sheet {,[-]} indices can be specified. --no-blurfilter Disables blurfilter. Individual sheet {,[-]} indices can be specified. --no-mask-scan Disables mask-detection. Masks explicitly {,[-]} set by --mask will still have effect. In- dividual sheet indices can be specified. --no-mask-center Disables auto-centering of each mask. {,[-]} Auto-centering is performed by default if the --layout option has been set. In- dividual sheet indices can be specified. --no-deskew Disables deskewing. Individual sheet {,[-]} indices can be specified. --no-wipe Disables explicit wipe-areas. {,[-]} This means the effect of parameter --wipe can be disabled individually per sheet. --no-border Disables explicitly set borders. {,[-]} This means the effect of parameter --border can be disabled individually per sheet. --no-border-scan Disables border-scanning from the {,[-]} edges of the sheet. Individual sheet indices can be specified. --no-border-align Disables aligning of the area detected by {,[-]} border-scanning (see --border-align). In- dividual sheet indices can be specified. -n --no-processing Do not perform any processing on a sheet {,[-]} except pre/post rotating and mirroring, and file-depth conversions on saving. This option has the same effect as setting all --no-xxx options together. Individual sheet indices can be specified. --no-qpixels Disable qpixel-mode for deskewing (do not internally use a 4x bigger image when rotating). --no-multi-pages Disable multi-page processing even if the input filename contains a '%' (usually indicating the start of a placeholder for the page counter). --dpi Dots per inch used for conversion of measured size values, like e.g.'21cm, 27.9cm'. Note that this parameter should occur before specifying any size value with measurement suffix. (default: 300) -t --type pbm|pgm Output file type. (default: as input) -d --depth Output pixel depth. (default: as input) -T --test-only Do not write any output. May be useful in combination with --verbose to get informa- tion about the input. -in --input-file-sequence Sequence of input filename patterns which is repeatedly traversed while resolving input filenames. Specifying a single entry is equivalent to the first filename argument after the options-list. -out --output-file-sequence Sequence of output filename patterns which is repeatedly traversed while resolving output filenames. Specifying a single entry is equivalent to the second filename argument after the options-list. -si --start-input Set the first page number to substitute for '%d' in input filenames. Every time the input file sequence is repeated, this number gets increased by 1. (default: (startsheet-1)*inputpages+1) -so --start-output Set the first page number to substitute for '%d' in output filenames. Every time the output file sequence is repeated, this number gets increased by 1. (default: (startsheet-1)*outputpages+1) --insert-blank {,[-]} Use blank input instead of an input file from the input file sequence at the specified index-positions. The input file sequence will be interrupted temporarily and will continue with the next input file afterwards. This can be useful to insert blank content into a sequence of input images. --replace-blank {,[-]} Like --insert-blank, but the input images at the specified index positions get replaced with blank content and thus will be ignored. --overwrite Allow overwriting existing files. Otherwise the program terminates with an error if an output-file to be written already exists. -q --quiet Quiet mode, no output at all. -v --verbose Verbose output, more info messages. -vv Even more verbose output, show parameter settings before processing. --time Output processing time consumed. -V --version Output version and build information. History ------- 2006-05-09, version 0.2 - Version number fallback to 0.2 (instead of counting up to 1.2). This better reflects the current development status and appears more suitable in an open-source context. - User documentation with several explaining diagrams added, see file 'doc/index.html'. - Color support added. Files of type PPM can be read, processed and written now. By default, unpaper uses the same image format for the output file as has been read from an input file, i.e. if your input is PPM, the output will be PPM automatically, if the input is PGM, the output will be PGM etc. Use parameter --type pbm|pgm|ppm to force an output file format independently from the input file format. - Better image rotation implementation. Auto-rotation of pages is now done much faster. - The --cache option became obsolete (due to new image rotation implementation). - The compiler-switch -DNOSINCOS became obsolete (due to new image rotation implementation). It is no longer needed for compilation on some platforms. - Changed to gcc version 4.1 for creating the Linux binary included in the distribution archive. gcc 4.1's executable appears to be about 20% faster than the one created with gcc 3.3.6. - Added '-fomit-frame-pointer' and '-ftree-vectorize' to compiler options. (Thanks to Olaf Marzocchi for the hint about -ftree-vectorize.) - Fix: the first pixel of an image is now correctly read (was read wrong in previous versions). - Minor fixes. 2005-08-27, version 1.1 - Single-page input sheets can now be joined to make 2-page output sheets. See options --input-pages [1|2] and --output-pages [1|2]. - Multiple input and output filename patterns can now be specified using the options --input-file-sequence and --output-file-sequence. - Blank pages or sheets can be inserted into the sequence of input sheets using --insert-blank and/or --replace-blanks, - Zooming and stretching are now supported, see --stretch, --zoom and --post-stretch, --post-zoom. - The sheet size can be set while keeping the content's aspect ratio (and centering it with best-fitting on the sheet) using --size and --post-size. For changing the sheet size without performing any stretching/zooming of content use --sheet-size. - Faster rotation when deskewing images. The rotating algorithm has been optimized to run up to 4 times faster than before. Additional speedup may be achieved by caching intermediate results of trigonometric calculations when processing multiple page. This caching uses a lot of memory, it can be enabled using the --cache switch. - Rotation detection can now be configured to start at individual edges of the area to process. The parameter --deskew-scan-direction no longer takes v[ertical],h[orizontal] as values, but now l[eft],t[op],r[ight], b[ottom]. The default is 'l,r', which matches the previous version's default 'horizontal'. - The behaviour of the 'double' template, used with the --layout parameter by specifying "--layout double", has been changed slightly. The input images are no longer assumed to be rotated by 90 degrees, as they would be when scanned from paper with a usual ADF-scanner. That means, "--layout double" no longer implies "--pre-rotate 90" and "--post-rotate 90". To get the layout-template behaving as in earlier versions, use "--layout double-rotated", this behaves like the old "--layout double". - --border-align --border-margin allow to align page content heading towards one edge, but keeping a specified distance margin. - Using the new parameter --blackfilter-scan-exclude [l,t,r,b], areas can be specified on which the blackfilter will not work. This is useful to exclude the blackfilter from working on inner page content, e.g. to protect images with larger dark parts from deletion. Note that the --layout option may preset some blackfilter-scan-exclude masks. - The speed of the noisefilter has been optimized a bit. - The default --mask-scan-step value has been changed from 10 to 5, which should sometimes lead to better mask-area detection. - The default --deskew-scan-depth value has been changed from 0.66 to 0.5, should sometimes lead to better rotation detection. - The default --black-treshold value has been changed from 0.5 to 0.33, which should give a better looking, more saturated result when converting grayscale images to black-and-white mode. - Existing output files will no longer be overwritten by default. If an output file to be written already exists, the program terminates with an error instead. Overwriting can be allowed using the --overwrite switch. - Fix: Setting the parameter --pre-rotate or --post-rotate to 0 will no longer issue a useless warning message. - Tested on MacOS X, additional compilation instructions for MacOS have been added to the documentation. - Tested on SPARC Solaris. - Compiler switch -DNOSINCOS disables use of the sincos() function, which preserves compatibility with math libraries that do not provide this function. 2005-03-14, version 1.0.1 - Syntax cleanups, better ANSI-C compatibility to solve some compiler issues. Using the -pedantic compiler switch, only warnings with intended origin are still issued. - Minor changes to the usage-info. 2005-03-01, version 1.0 - Initial release.