UNIX has long been known for its software development environment; the wealth of first-rate and innovative tools is one reason behind UNIX’s commercial success. Lately, however, good tools have gotten somewhat out of reach. Not that UNIX vendors haven’t been doing their job, but nearly all of them have “unbundled” their development tools from the operating system itself, so that if you want to do any programming, you need to pay another $600 to $1000. That’s not much for a large corporation, but it’s a lot for a student or a home user. This book introduces a set of software development tools that are parallel to the time-honored UNIX tools. They provide the same (or more) functionality and a high level of compatibility with the tools UNIX users have grown up with. These are the compilers and related development tools, often called the GNU tools, from the Free Software Foundation (FSF). Unlike the UNIX tools—but displaying the same spirit as the early UNIX systems that were installed around college campuses—these tools are “free”: they are available from many places without charge, they include source code, and you’re encouraged to give these tools away to your friends. Instead of paying your UNIX vendor for a “development system,” you can get the source code for the GNU tools from an FTP or UUCP archive. Unfortunately, the operating system sometimes requires features that GNU does not support. Currently, for instance, if you have the SGI Irix system, you need to buy SGI's development system in order to get some critical libraries. That may tie up your phone for a few hours, but aside from the phone costs, it’s completely free. For a few tens of dollars, you can buy this book/CD combination and get a complete, pre-compiled development environment; similar pre-compiled packages are available from other vendors. distribution of software redistribution of software CD distribution free softwareCD distribution of There are a lot of advantages to the CD distribution—of which the most important is that the software has already been built and tested on several of the most commonly used computers and operating systems. If nothing else, that saves you a lot of time: although we’d all like to think that we can grab a software package from the Net, edit a few configuration files, type make, and forget about it, we know that real life is rarely this simple. The CD distribution also solves a nagging “chicken and egg” problem. If you get source code for free software, you need a compiler (and related utilities) to build executables. If you have your vendor’s development system, then you can get started, but if you don’t, there’s no point in loading up your disk with a few megabytes of free source code. By providing pre-built software for several common architectures, the CD gets you out of this bind. This book is a fairly traditional user’s guide to the GNU software development environment. We’ll talk about what these tools are and how to use them: how they fit together. We won’t go into esoterica; rather, we’ll try to make you familiar and comfortable with the most common uses of these tools. So, while we’ll spend some time discussing  gcc -O (compile C code, with optimization), we’ll barely mention  gcc -mno-half-pic. But first, we’ll discuss what free software is, why it exists, and what it means for software developers. UNIX When it comes to the sophistication and flexibility of its tools, UNIX can compete with any programming environment on the market. Many tools that now see widespread use throughout the computer field started on UNIX. Just look at the C language. But there’s a hump to get over when you’re beginning to use UNIX tools. One reason they’re so fast and powerful is that you have to learn very brief commands, or even (in the case of Emacs) keystrokes. Many people try to avoid UNIX because few of its tools support graphics or point-and-click interfaces. But after a short period of getting used to the system, you will find that commands and keystrokes are faster than pointing and clicking. Even more important, you can combine and automate commands. Furthermore, don’t forget the cost advantages: you can use the tools on a cheap terminal connected to a serial line. graphical interfaces So if you know what you’re doing, the tools in this book help you get your job done a lot faster than a graphical user interface. As a programmer, what you need is a powerful interface to text—and the tools certainly offer that. Anyway, some movement in the direction of graphical interfaces can be seen. Emacs now supports menus and the mouse, for instance. integration Integration is another strength of the free UNIX tools in this book. For instance, in the Emacs editor, you can issue a compile command, while making changes to a program. This compile command runs the make utility in order to build your program. If the source code is stored in the RCS source control system, make can automatically retrieve it. Throughout the discussions of tools, you will see ways to make them collaborate for your convenience. commands As we’ve said, free software tools are command-driven. Some of the commands can get quite powerful, too. For instance, in the gdb debugger, the print command can display any kind of variable from an integer scalar to an array of strings to a complicated data structure with members of different data types. key bindings So the first stage of your learning is to find a set of commands that do most of the things you want. The next stage is to combine them. For instance, Emacs lets you combine a number of keystrokes into a package and execute it with a couple of keystrokes; this is called a macro. You can also put commands into a start-up file, and Emacs will execute them every time it begins. gdb offers a similar initialization file for debugging sessions. If you’re a student learning to program who happens to be on a UNIX system, or a programmer from another environment who has a project to do on UNIX, this book will make you comfortable with UNIX and its tools. The book won’t teach you C or C++, but it will show you how to build, run, and debug programs. If you’re a more experienced UNIX user who wants a powerful set of tools, this book and CD combination will give you a one-stop programming kit. You might skip Chapter 2, but even a UNIX-savvy programmer will find useful tips in the rest of the book. When you begin to use a new operating system, you often feel like you want to learn everything at once. This book gives you the alternative of a saner step-by-step study plan. First, after a whirlwind introduction to UNIX, you learn the text editor. Once you can write your programs, you learn about the compiler and some useful C and C++ library calls. Then come the more optional tools, such as the debugger and source control system. Of course, we can’t say everything about every tool in one book. But you won’t need to consult other documentation too often; this book should satisfy 95 percent of your needs. We’ll list other available documentation for each tool as we come to it. You can install the documentation online from the CD, or order it in hard-copy form from the Free Software Foundation (to be discussed next). free software softwarefree Software developers have been trading their pet programs ever since computers were invented. Particularly in the university community, programmers have always felt free to share their tools with one another. And most computer vendors have, at one time or another, actively encouraged their users to share their software: for instance, DEC’s user group (DECUS) maintains a free library of member-contributed software. Of course, when networking became widespread (in the early 1980s for the university community and early 1990s for other users), it became much easier to share software: when you wrote something you thought useful, it was easy to send it to your friends through email or to post it in some FTP archive or (in the DOS world) on an electronic bulletin board, where others could pick it up. You could get the software “for free”; you could modify it and add your own features; and you could give your new versions to your friends. So there has always been a lot of free software floating around. The sorts of programs that have been freely available haven’t typically been large, substantial programs with significant commercial value—but that’s not to say that the software hasn’t been useful. We can’t imagine someone selling a program like sps (which is a fancier version of the UNIX ps command), but you can’t claim that sps isn’t useful, either. In the early '80s, major pieces of free software with significant commercial value started appearing, a trend which has really blossomed in the late '80s and early '90s. Entire operating systems (like Linux, a UNIX-compatible operating system that is currently most popular on Intel X86 hardware), compilers (like gcc and g++), interpreted programming languages (like Perl and Tcl), editors (like Emacs), and other major tools are now freely available. We can divide free software into three different classes. There are gray areas between each of these groups, but on the whole, most software falls into one category or the other. Public domain software public domain software Some software is truly and explicitly put into the public domain. This means that anyone can use the software for free and do anything they want to it. There are no restrictions of any kind. Freely distributable software freely distributable software ownershipsoftware softwareownership of distribution of software redistribution of software A lot of free software, particularly software coming from the university community, is “owned” by the person or organization that developed it. The portions of BSD UNIX that were developed at Berkeley fall into this category, as does the X Window System, along with many, many smaller programs. In this context, “ownership” means that people redistributing the software as part of another project must credit the original owner; it (probably) also means that, while you can modify the software to your heart’s content, you can’t redistribute your changes under the same name as the original without permission. That is, if you modify BSD UNIX, you can’t tell your friends that you’re giving them a copy of BSD UNIX without the owner’s (University of California’s) permission. General Public License (GPL) software license GPL (General Public License) General Public LicenseGPL copyleftGPL Free Software Foundation (FSF) This means software that’s covered by some version of the Free Software Foundation’s General Public License (also called a “copyleft”), which is reprinted in Appendix D. The GPL is designed to keep free software free. That is, it is crafted to guarantee that anyone receiving the software, no matter how they got it and how it has been modified, can continue to redistribute it under the same terms. We’ll go into this in more detail in a minute. By the way, some people think that GPL stands for “GNU Public License.” It certainly originated with the people who started GNU, but it’s used on a lot of other software (such as RCS and the Linux operating system). So “General” is the correct term. In this book, the term “free software” generally means “software licensed under the GPL, or some relatively similar terms”—though many people use the term to include all three categories. It’s worth taking a few moments to understand why the GPL is necessary. The first two kinds of free software guarantee that the original program is, for the time being, free: i.e., you can get the code and modify it. However, they don’t prevent one very common scenario. They don’t prevent a commercial venture (say, Joe’s Computers Inc.) from taking the free software from the Net, improving it significantly, and applying a restrictive license to it—perhaps preventing you from seeing the source code and requiring you to pay a large annual licensing fee for the right to use the software. If you think that scenario sounds far-fetched, think about the networking code in most UNIX systems. Most UNIX network implementations are based on work done for BSD UNIX. That code is “freely distributable software”; you can get the original BSD network code over the net and do whatever you’d like to it. But would your UNIX vendor give you the source code for their improved version of this same software? Probably not, unless you were willing to pay a lot of money to them—and possibly not under any circumstances. Over time, this mechanism has caused a lot of software to evaporate from the public arena. The original code is usually still available, but the good code—the code you’d really like to have—is a commercial product, and it is not free at all. Free Software Foundation (FSF) The Free Software Foundation was created in the mid 1980s to foster the development of software that would always be free. Their General Public License is designed to prevent free software from evaporating, as corporations cull the best contributions, improve them, and remove them from circulation. It specifically states that anyone using the software has the right to redistribute it, with the source code, and that modifications to the software must also be redistributable. In practice, this means that you cannot apply a restrictive license to your modified version of free code. It certainly does not mean that you are forced to distribute your modifications, even if you don’t want to. You’re free to keep your modifications to yourself—which, in many cases, is probably a good idea. But if you do decide to distribute your improved version, you must distribute it under the same terms as the original software. In particular, the GPL states that software must always be distributed either with the source code, or with a promise to provide the source code upon asking. LGPL (Limited GPL) The GPL (or a slightly limited version of it, called the LGPL) is applied to all software created by the FSF. It has also been applied to a lot of software that is not created or distributed by the FSF. The software discussed in this book, and the software included on the CD-ROM, is all covered by the General Public License; most of it was developed by the Free Software Foundation, or in association with them. cost of free software It’s a common misconception that “free software” means “software that doesn’t cost anything.” That’s really not the case. There is nothing to prevent someone from selling you a disk containing free software, or even charging you for FTPing the software from their network archive. In fact, the General Public License explicitly states that selling the software is allowable—but that you cannot restrict others from reselling what you have sold to them. The CD-ROM that accompanies this book isn’t free, for example; you also can buy copies of the software from the FSF itself, Cygnus Support, Cyclic Software, and other companies. The word “free” is used in a more political sense: it really means “liberated from licensing restrictions.” That’s a crucial part of the Free Software Foundation’s vision: its goal is to create a complete set of software (an operating system, plus all necessary utilities) that can be circulated without restriction. quality of free software bugs in free software Free software is one area in which the adage “You get what you pay for” doesn’t apply. Sure, lots of public domain software and shareware is bug-laden and poorly maintained. However, the software distributed by the FSF is, on the whole, among the most well-maintained and up-to-date software available. As a rule, it is usually better than the best commercial equivalents. The gcc and g++ compilers, which are discussed in Chapter 4, are widely acclaimed and generally produce better code than most vendors’ compilers. When bugs are discovered, as they are in any software, they are fixed very quickly; the FSF’s response to bug reports should be the envy of any commercial software vendor. securityfree software and viruses Trojan horses Another concern that’s often voiced about free software is security. The impression exists that free software is full of Trojan horses, viruses, and other noxious creatures that will disable your system as soon as you start using it. This may be true of some public domain software, but such behavior is (in the UNIX world, at least) fairly rare, though not unknown. Even in the DOS and Mac worlds, where viruses abound, it has been claimed that “free software” is actually safer than commercial software. We wouldn’t stake our systems on that claim. You owe it to yourself to be cautious about true public domain software and shareware. However, the software that’s distributed by the Free Software Foundation is completely trustworthy. Like any software, it may have bugs and security holes—no vendor, no matter what they charge, can guarantee that their software is “bug-free.” But you can rest assured that there are no intentionally introduced hazards to plague you. technical support support, technical Finally, some users are concerned that free software is “not supported.” As we’ve said, if commercial software vendors were as responsive to trouble reports as the FSF, our world would be a better place. However, that doesn’t eliminate the need for traditional customer support services, to provide everything from “handholding” to bug fixes. Such support services are available for a fee, from Cygnus Support and other companies. Cygnus provides a level of technical support that’s the equal of that provided with any commercial product. There are, we’ll admit, a few people in the free software community who think that “support isn’t necessary: you have the source code, so you can fix it yourself.” However, you don’t have to listen to them if you don’t want to. distribution of software commercial software Several misunderstandings about the relationship between free and commercial software have discouraged their use together. The most common—though not particularly relevant to this book—is that software covered by the GPL can’t be distributed with commercial products. This is just untrue. The General Public License requires that: The free software be distributed with source code or an offer to provide source code No additional restrictions be attached to the use or redistribution of the free software Neither of these provisions means that you can’t put free software on a distribution tape alongside your commercial product. gcc (GNU C compiler) The next most common misunderstanding is that code that’s “touched” by the GNU tools somehow becomes “free software” and can’t be sold commercially--that is, that the General Public License requires output from the gcc compiler to be placed under the same license. Again, this just isn’t true. While the Free Software Foundation hopes that gcc users will feel a sense of obligation to make their software freely available in one form or another, their license does not compel you to do so. Only if you incorporate source code from free software do you have to make your code freely available. There are a couple of border areas that have contributed to this misperception, though. Because you cannot apply a typical software license to code covered by the GPL, you also cannot incorporate “free” code within a commercially licensed product. This means that libraries present a special problem. Incorporating code from a “free” software library (like libg++ or the GNU version of libc) would create a program that had to be licensed under the GPL—since, by distributing this program, you would also be distributing the libraries. libraries There are several solutions to this problem. Most obviously, there’s no requirement that you use GNU libraries when you compile; you can use commercial C or C++ libraries. If you want to use “free” libraries, however, the Free Software Foundation recognized that restrictions on the use of libraries was preventing people from using their software; free software that no one felt free to use was not helping their cause. To this end, the FSF created a special version of the GPL for libraries (the LGPL); the LGPL allows you to distribute software that includes the GNU libraries without requiring you to make your software freely redistributable or requiring you to ship the source code for your libraries. This solution is not without its problems—and those problems lead to further solutions: LGPL (Limited GPL) bison utility byacc package Many programs require a parser to handle an input stream, a very difficult piece of software to write correctly. Programmers usually generate this part of their code through a "compiler-compiler" like yacc. Old versions of the bison parser generator would generate parsers that were under the GPL, thus preventing their use in non-GPL'd programs. This is not true of recent versions of bison, however. Nor has this ever been a problem with byacc, the parser generator included with this book. The LGPL still places too many requirements on the libraries that it covers. While you aren’t required to apply the GPL to software that includes LGPL libraries, you are required to distribute your software in a way that allows the user to substitute updated libraries for the original ones. Depending on what you’re doing, this can be a minor annoyance, or completely impractical: for example, if your software controls a microwave oven, you can bet that the users (a) have no way of relinking your program with a later library release, and (b) don’t care. Cygnus Support provides their own version of the C libraries; these have a less restrictive license that only requires that you acknowledge use of their software. The CD supplied with this book includes these versions. GNU project UNIXGNU project Without a doubt, the GNU project is the world’s premier free software factory. Its gcc and g++ compilers set the standard for compilers on UNIX. They also made the development of Linux possible. Richard Stallman’s Emacs editor not only boasts a fanatic following, but has been copied on other platforms. We’re not going to delve deeply into the purpose of the GNU project here (for an introduction, go to any site storing GNU software and get the file called INTERVIEW ), but it goes beyond a set of tools; its goal is the creation of an entire UNIX-like operating system. The project was started by Richard M. Stallman, the developer of Emacs and a famous proponent of free software, around 1985. The Free Software Foundation coordinates and directs the GNU project. Cygnus Support, Inc. Cygnus Support, Inc. and Cyclic Software put together the packages of free software on the CD-ROM included with this book. They were founded by free-software developers to promote the use of free software by large organizations that perhaps wouldn’t consider it under normal circumstances. The main role of Cygnus is to provide a level of support at least as reliable as that offered by software manufacturers for commercial software. In addition to solving customer problems, Cygnus creates some free software themselves (such as the GNATS bug-tracking system and the DejaGNU test platform), fixes bugs, and writes cross-compilers. They offer customer service for the most mission-critical GNU software, such as the compilers and related program-development tools. Cygnus maintains close contact with the FSF, although the two are not affiliated. While no longer the only commercial company focused on free software, Cygnus is the largest and best known. A formal description of the company appears in Appendix A. This book discusses the Free Software development environment, specifically: Editor: the Emacs editor Compilers: gcc and g++ for C and C++, respectively Libraries: libc and libg++. The libraries included on the CD are the Cygnus versions that are not subject to the LGPL. Assembler and linker: the gas assembler and ld linker Binary utilities: size, nm, gprof, and other tools that work with object files and executables. gprof does not run on all architectures. Debugger: gdb, the state-of-the art debugging tool Software build program: make, for automating the compilation process Source control program: the RCS package, for source file revision control flex package byacc package The CD-ROM includes source code for all these packages. The CD also includes flex and byacc, which aren’t discussed in the book. The best source of information on these packages is O'Reilly's Nutshell Handbook lex & yacc. We’ve included them because they’re needed to recompile several of the other tools. The CD includes executables for SunOS 4.1.X (on SPARC processors only), Solaris 2.X (on SPARC processors only), HP-UX (on the HP 9000/700), AIX (on the IBM RS/6000), Irix (on the SGI Iris/Indigo), and Digital UNIX (on the Alpha). Linux executables are not on the disk, because Linux users already have binaries for most of the tools as part of their Linux distributions. Although the CD contains executables only for the platforms listed above, it contains the source code for all platforms and cross-compilation environments that the Free Software Foundation knows about. Cygnus does not support all the tools or all the architectures; support is discussed in . Thanks to the thoroughness of the Free Software Foundation, plus the loyalty shown by thousands of users who have ported the software to innumerable systems, an automated configuration procedure exists that can handle a huge variety of platforms. Check the configuration files provided with the software; you may be able to build the executables yourself. , explains the basic procedure. A more extensive guide to building software on different platforms is available in the O’Reilly book Porting UNIX Software, by Greg Lehey. Even if you have one of the platforms for which binaries are provided, you may find a reason to rebuild from sources. For instance, you may want to change the default pathnames stored in Emacs. Or you may hear of new versions of these tools (they are being upgraded all the time) and decide you want them. Since these platforms are so common, you should have no trouble rebuilding your software. Windows NT Microsoft Windows NT VAX/VMS It’s worth noting that the GNU tools are not only used under UNIX. Congruent Corporation has ported the GNU tools to Microsoft’s Windows NT operating system, and many of the tools run under VAX/VMS. For some systems, the GNU tools are the only development environment: for example, they’re the only tools available for Linux, 386BSD (another freely available operating system), and BSDI (a commercial operating system based on BSD UNIX). binaries, installing installing binaries read.me file (on CD) Instructions for installing the binaries are on the CD-ROM, in a file named read.me. hard drive, copying to These tools are not meant to run from your CD. We have compressed them so that we could fit more onto the CD. But you would probably choose to move the executables to a directory on your hard disk anyway. After all, you bought your CD player to hold a variety of disks, not just this one. Your editor, compiler, and debugger are pretty constant companions, and you won’t want to unmount them just so you can load up the latest hot graphics CD. updatingsoftware tools softwareupdating support, technical technical support New features are being added to free software all the time; sometimes a radically different product results. For instance, Version 19 of Emacs (which is on the accompanying CD) introduced the pull-down menus that are so popular on graphics terminals. The most recent version of gdb supplies a graphical interface. So, while you can be happy with the software on the CD for a long time, you will eventually want to get a new version of some tool. Cyclic Software One option is to get support from Cygnus (to cover the compiler tools) or Cyclic (to cover RCS and CVS). Neither company provides Emacs. If they can supply the binaries for your system, this is the simplest action. (They support a lot of platforms that aren’t on this book’s CD.) Their addresses are: Cygnus Support 1937 Landings Drive Mountain View, CA 94043 USA WWW: http://www.cygnus.com Email: info@cygnus.com Telephone:+1-800-CYGNUS-1 or +1-415-903-1400 FAX: +1-415-903-0122 Cyclic Software 1701 16th St. NW #652 Washington, DC 20009 USA WWW: http://www.cyclic.com/ Email: info@cyclic.com Telephone:+1-202-265-6119 You can also buy collections of products on tape from the Free Software Foundation. Only sources are provided. Their address is: Free Software Foundation, Inc. 59 Temple Place, Suite 330 Boston, MA 02111-1307 USA WWW: http://www.gnu.ai.mit.edu Email: gnu@prep.ai.mit.edu Telephone: +1-617-876-3296 FAX: +1-617-492-9057 FAX (in Japan): 0031-13-2473 (KDD) When buying from Cygnus, Cyclic, or the FSF, you are helping to support free software. FTP, obtaining software via softwareobtaining via FTP But one of the attractions of free software is that it is free of charge—if you can take the trouble to obtain it over a network. A number of Internet sites have the sources for all the software in this book on FTP sites. One of the most popular is ftp.uu.net. Go to the directory /systems/gnu on that site. The home site for the FSF is prep.ai.mit.edu, where the software is under /pub/gnu, but that site is often overloaded and sometimes has to deny access to anonymous logins. If you want to ftp free software regularly, go to the /pub/gnu/GNUinfo directory and get the file FTP for information on sites around the world that carry free software; you can then choose a site closer to you that is less well known. For people without Internet access, free software sources are also available on some BBS sites. free softwareculture of culture of free software resources GNU As you have probably guessed, free software is based largely on volunteer efforts: although many people have contributed to the wealth of software that’s available, the number of people who make their living (all or in part) from free software is very small, indeed. Free Software Foundation (FSF) The most important organization devoted to the proliferation of free software is the Free Software Foundation. They have made lots of software available on their own, and they’re a good source for information about other programs. One of the FSF’s goals is to develop a complete UNIX-like operating system; they have already developed many of the utilities for this operating system, including versions of tar, more, vi, and sed (plus the software described in this book). The best way to find out what software is available from the Free Software Foundation is to ftp to prep.ai.mit.edu or one of the systems mirroring it, and look in the directory /pub/gnu. newsgroups, GNU information via USENET, GNU information via It’s also a good idea to look in the gnu Usenet newsgroups; this is where you’ll find free software discussed most heavily. Particularly important newsgroups are: gnu.announce For announcements of newly available free software, new releases, etc. gnu.emacs.help For questions about the Emacs editor gnu.g++.help For questions about the g++ compiler gnu.gcc.help For questions about the gcc compiler gnu.misc.discuss For general discussion and questions The last group, gnu.misc.discuss, is where to find out about the “feel” of the free software community; there’s a lot of philosophical and legal discussion, in addition to questions about what software is (or should be) available (and if it doesn’t exist, why not). This group is very sensitive to any mention of a commercial product, so be forewarned. Another good source of information is the GNU’s Bulletin, the Free Software Foundation’s newsletter. This is published in January and June; you can get a copy by sending your name and address to: Free Software Foundation, Inc. 675 Massachusetts Avenue Cambridge, MA 02139 USA They ask you to send sufficient postage to return their reply, but this is not required. (The postage is $0.78 for someone in the U.S., and enough International Reply Coupons for a package of 100 grams to reach overseas correspondents.) You can also send electronic mail to gnu@prep.ai.mit.edu ; that’s a good way to ask questions. The Free Software Foundation is always in need of money, talent, and equipment. If you use the software described in this book, you should realize that they’re giving away a package with a commercial value of several thousand dollars. You are encouraged to make a tax-deductible donation to the Foundation so they’ll be able to continue providing this service. The donation allows them to do all the things that a software development environment has to do—rent space, pay staff, buy equipment, and so on. League for Programming Freedom (LPF) LPF (League for Programming Freedom) We’ll end the chapter by mentioning one other important organization. Although there is no formal connection between the Free Software Foundation and the League for Programming Freedom (LPF), most people who are interested in free software agree with the goals and aims of the LPF. The LPF is an advocacy group whose charter is to “protect the freedom to write software ... which is threatened by ‘look and feel’ interface copyright lawsuits and by software patents.” Software patents, in particular, are increasingly being awarded to companies to cover common processes that many programmers use. Many commercial software organizations support the LPF. Its purpose is quite different from the FSF. The LPF simply wants to make sure that the developers of all kinds of software—free and commercial—can live without fear of being sued because their software is vaguely “like” somebody else's. If you are interested in the LPF, send mail to lpf@uunet.uu.net.