NEXTSTEP Compiling And Linking Q&A

Creation Date: August 14, 1998
Keywords: NeXTSTEP

Disclaimer

This document pertains to the NeXTSTEP operating system, which is no longer a supported product of Apple Computer.  This information is provided only as a convenience to our customers who have not yet upgraded their systems, and may not apply to OPENSTEP, WebObjects, or any other product of Apple Enterprise Software.  Some questions in this Q&A document may not apply to version 3.3 or to any given specific version of NeXTSTEP.

Q:  I thought that an #if 0 block will ignore everything inside it.  But it doesn't.

A:  The ANSI standard says that #if statements only ignore syntactically correct code.  So if there are syntax errors, you get an error.  Two particularly nasty ones are unterminated comments, and unterminated string constants.  See examples below.

/tmp/foo.c contains:

	#ifdef NEVER 
	/* this is the start of 
	a comment that never ends
	#endif

/lib/cpp /tmp/foo.c generates:

	/tmp/foo.c:1: unterminated '#if' conditional

Or, /tmp/foo.c contains:

	#ifdef NEVER
	some words with a single quote ' OK
	#endif

it generates:

	/tmp/foo.c:2: unterminated string constant
	/tmp/foo.c:1: unterminated #if conditional

Note: In 3.0 or 3.1, the error generated reads:

	/tmp/foo.c:2: unterminated character constant

Q:  I have libraries compiled for NeXT 3.0 machines that I need to use on NEXTSTEP for Intel Processors.  What should I do to make them work?

Q:  I wrote an application under 3.1 on an Intel NEXTSTEP platform that uses a library I wrote under 3.0, but it won't link properlyÐwhy?

Q:  How do I make my 3.0 libraries accessible in a multi-architecture form to 3.1 applications?

A:  Libraries that were compiled under NEXTSTEP 3.0 and earlier should work fine under 3.1Ðfor NeXT hardware.  Some extra work is required to make them work properly under NEXTSTEP 3.1 for Intel Processors.  Because the m68k architecture and the i386 architectures are radically different, it is necessary for NEXTSTEP to have access to a binary for both architectures in order to provide an executable that runs on both machines.  Libraries, being binary files containing machine instructions, are no different from any application in this respect.  Building multi-architecture applications is easy, since the Project Builder application takes care of the details of building Multi-Architecture Binary files (MAB files) when requested.  But Project Builder doesn't manage libraries, so an application that you have compiled may fail to link properly because of the lack of a library matching the architecture type of the machine you are compiling for.

3.1 provides a new utility, libtool, for generating multi-architecture (or "fat") libraries.  libtool is intended to replace both the ar and ranlib utilities.  Traditionally, the construction of a library is managed by a Makefile, which usually does roughly the following:

1)	Compiles the .c and .m (source) files into .o (object) files, using the compiler (cc).
2)	Archives the resulting .o files into a .a (library) file, using the archiver (ar).
3)	Generates a table of contents file in the archive, using the ranlib utility.

A simple sequence to generate a linkable library starting from just two C files might be:

	/bin/cc -c -o first.o first.c
	/bin/cc -c -o second.o second.c
	/bin/ar ruv libsilly.a first.o second.o
	/bin/ranlib libsilly.a

Under 3.1, however, steps 2 and 3 above are combined into one libtool call, so that the equivalent sequence would be:

	/bin/cc -c -o first.o first.c
	/bin/cc -c -o second.o second.c
	/bin/libtool -o libsilly.a first.o second.o

Notice that the call to ar, with the r, u, and v switches on, has been replaced with a call to libtool, and that calling ranlib is no longer necessary.  The r, u, and v switches allowed ar to update only the objects that were newer than the ones in the archive; the libtool call doesn't have this capability, but is much faster than ar, and in addition, it is capable of archiving fat objects, which ar cannot do.

So where do multiple-architecture objects come in?  So-called "fat" objects are generated when the compiler is told to generate objects of more than one architecture type, using the -arch flag.  If you look at the compile text field in Project Builder, you should notice that it uses this flag whenever it is asked to generate something for both the m68k and i386 architectures.  The change needed to the above sequence is:

	/bin/cc -arch m68k -arch i386 -c -o first.o first.c
	/bin/cc -arch m68k -arch i386 -c -o second.o second.c
	/bin/libtool -o libsilly.a first.o second.o

And that's all there is to it.  The gains to library-building using libtool are the ability to make fat libraries, and the elimination of the need to use ranlib.  The loss is the ability to update only those files in the archive that have timestamps older than the input filesÐin general, the amount of time lost in moving from ar and ranlib to libtool should be minimal, if any.

For the Makefile minded, here's a simple example of what needs to be changed.  The first file is the original, which compiles into a single-architecture library, and the second file will generate the multi-architecture library:

	-------------------------------
CC		= /bin/cc
RANLIB	= /bin/ranlib
AR		= /bin/ar
ARFLAGS	= ruv

CFLAGS	= -g -O2 -arch m68k -arch i386

SRCS		= first.c second.c

OBJS = $(SRCS:.c=.o)

.c.o: ; $(CC) $(CFLAGS) -c -o $@ $<

all: libsilly.a

libsilly.a: $(OBJS)
	$(AR) $(ARFLAGS) $@ $(OBJS)
	$(RANLIB) $@

clean: ; /bin/rm -f *.o *.a *~
-----------------------------------
CC		= /bin/cc
LIBTOOL	= /bin/libtool

CFLAGS	= -g -O2 -arch m68k -arch i386

SRCS		= first.c second.c

OBJS = $(SRCS:.c=.o)

.c.o: ; $(CC) $(CFLAGS) -c -o $@ $<

all: libsilly.a

libsilly.a: $(OBJS)
	$(LIBTOOL) -o $@ $(OBJS)

clean: ; /bin/rm -f *.o *.a *~

--------------------------

Q:  How can I get make to recompile my program when I've only edited one of the .h files? I've changed the class by editing the interface file, but it thinks the program is up to date.  

Q: Is there an easy way to recursively trace all the header files that my source file depends on?

A:  You can, of course, fool make by typing

	touch foo.m

in a shell, before compiling, but that's not a very elegant solution.  What you need is a Makefile.dependencies file.  

In NEXTSTEP Release 2, this can be easily generated by typing

	make depend
 
in a shell, assuming your Makefile was generated by InterfaceBuilder. 

In NEXTSTEP Release 3.0, ProjectBuilder provides an "Args" textfield in the Builder inspector.  Just type

	depend

in a shell, to generate a Makefile.dependencies file. 

In NEXTSTEP Release 3.1, the user interface of ProjectBuilder has been further simplified.  You just select "depend" as your Target for the Build operation. 
 
Makefile.dependencies contains all the dependencies, listing the files included by your .m files.  A typical line of this file looks something like:

	MyClass.o : MyClass.m MyClass.h 

From then on, when you do a make in this directory, whether from InterfaceBuilder in NEXTSTEP Release 2,  or from ProjectBuilder in NEXTSTEP Release 3, or a shell, it checks this file before deciding the program is up to date.  Makefile.dependencies can also be edited by hand, if for any reason make depend doesn't do what you need.  If you add new files to your project, you should delete Makefile.dependencies and regenerate it before recompiling.  Note that ProjectBuilder in NEXTSTEP Release 3 is smart enough to not recompile your files in case only your interfaces file (.nib) has been modified.  It just copied the nib.

In answer to the second question, cc has a -M flag that can be useful if you want to see a listing of all the header files your source file depends on, including the AppKit and operating system files.  See also the man page.

Q:  I'm building a bundle project, but I want the bundle to have a different extension than the default .bundle so I can treat it as a document for my own application or for an application like Preferences or BackSpace.

A:  When bundles were originally released it was thought that they would only be used to dynamically load infrequently used parts of an application.  However, it's now very common and even encouraged to make your application extensible by having it accept bundle documents at runtime.  Applications like Preferences, BackSpace and IconBuilder all do this. Since these bundles are essentially unique file formats (you wouldn't, for instance, try to load a BackSpace module into Preferences) they need unique extensions.

In Release 3.0 and 3.1 you can change the extension of a bundle from its default of .bundle by adding the following lines to the Makefile.preamble and Makefile.postamble respectively:

Addition to Makefile.preamble

Add the following to Makefile.preamble to set a new extension and install directory (replace XXXX with the new extension and AppName with your application's name):

	BUNDLE_EXT = .XXXX
	INSTALLDIR = $(HOME)/Library/AppName
	OTHER_PRODUCT_DEPENDS = $(NAME)$(BUNDLE_EXT)
	OTHER_GARBAGE = $(NAME)$(BUNDLE_EXT)

Addition to Makefile.postamble

Add the following to Makefile.postamble to build the bundle with the new extension:

Remove the old version of the named bundle before installation:

	before_install::
		rm -rf $(DSTROOT)$(INSTALLDIR)/$(NAME)$(BUNDLE_EXT)

Move the bundle to its new extension after installation:

	after_install::
		mv $(DSTROOT)$(INSTALLDIR)/$(NAME).bundle    \
			$(DSTROOT)$(INSTALLDIR)/$(NAME)$(BUNDLE_EXT)

Link the bundle to its new extension for incremental building/testing:

	$(NAME)$(BUNDLE_EXT):
		-ln -s  $(NAME).bundle $(NAME)$(BUNDLE_EXT)

In Release 3.2 this problem will be fixed by the addition of the BUNDLE_EXTENSION macro. You can simply set this macro in the Makefile.preamble to get the new extension.

Q:  When a link is in progress, it seems to take over my machine.  It's sluggish to activate another application and do any other work.  Is there a cure for this? 

A:  From a scheduling point of view, there's not much you can do to convince the kernel to give fewer resources to the linker. The linker is designed as the kernel's Dream Program by asking for certain size blocks with a certain regular pattern.

However, the linker is I/O bound.  It's moving a lot of bits back and forth so anything you can do to make the travel time faster is a win:  faster hard disks, local access to files rather reading files all over NFS, etc.  Constructing the symbol table is a large resource hog; linking without symbols is much faster than with.  However, often you need the symbols for debugging while in development.  You can use the ª-xº option to ld(1) to remove symbols from the sections of your code that you're not debugging currently, and then compile new sections with symbols so that only the parts you are working on have symbol information.  This is assuming you are concentrating your debugging efforts on only the changed parts. 

To do this, you need to do the following for each of your .o files:
	
	localhost> ld -r -x foo.o
	localhost> mv a.out foo.o

Then when you modify your sources, make(1) just recompiles the object file with full symbols if you have the ª-g ºflag on.

NeXT's software engineers use systems with the smallest supported memory configurations to ensure they are developing software that is usable on the base configuration.  We have implemented compile servers to minimize the impact of large compiles on these local machines.  The compile servers maintain the sources locally and all compiles take place there.

Q:  I've recently upgraded Release 2 and I've recompiled my applications, including all my libraries.  Now I can't link my application because of undefined symbols.  This worked in Release 1.  Is this a bug?

Q:  I'm porting my C code from another UNIX environment.  This code compiled and linked fine in the old environment, but fails to link on the NeXT.  What's going on?
	
A:  The GNU C compiler used by the NeXT was chosen, in part, because it complies with the ANSI C standard.  Under Release 2 and later, the loader is more strictly ANSI compliant than under Release 1.  The older BSD environment used by many UNIX systems tolerates lazy coding practices that ANSI does not.

To verify that this is your problem use the nm(1) command on the undefined symbols in your program (see the man page for more info): 

	localhost> nm -o myLib.a | grep mySymbol
	myLib.a:foo.o: 000000004 C _mySymbol

Useful tip:  To print the table of contents for a library, use otool(1):

	localhost> otool -Sv myLib.a

Note the 'C' in front of the symbol '_mySymbol';  this indicates that it is a common symbol.  The problem revolves around libraries that declare variables, but do not bother to initialize them.  A common example is errno.  We have all seen code that will declare errno in different files like:

	int errno;
	
We all know that errno is defined somewhere, so we leave it to the linker to figure out which .o actually owns the variable to link it in.  The linker assumes that the object file that not only declares the variable but also initializes it (called a Data Symbol), owns it and links in that module.  If a module merely declares the variable, then this is called a Common Symbol.

Now comes the difference between the way ANSI looks at this versus generic BSD environments.  What if no module actually initializes the variable in question.  Then every module has this symbol declared as a Common Symbol.  What is the linker supposed to do at this point?  Make a guess and pick the first module that references this Common Symbol and link it and any accompanying code that comes with it?  Chances are you may never actually call the code in that module, but since this module declared knowledge of this Common Symbol, the loader has no other choice but to link it in.  Your final executable file will have this extra code along with any other modules that this unwanted code may reference.

In an effort to make executables smaller and more efficient, to reduce paging and link and loading times, our ld complies with ANSI and ANSI considers this to be an error.  This is why when you were linking your code, your variables could not be found.  

The solution:  The right way to fix this would be to initialize your variables in the library module that owns the data.  If this is not feasible, then ranlib(1) knows how to revert to the older BSD-like behavior  (see the -c option in the man page):

	localhost> ranlib -c myLib.a

Q:  Does it make sense to use the -c and -s options for ranlib simultaneously?  As in:

	localhost> ranlib -s -c myLib.a

A:  Yes, but sometimes (lots of times) it does not work.  What you are trying to do here is to speed up the link editor.  This is accomplished by having ranlib build the table of contents in sorted order; this in turn is used when the link editor searches for an object file in a library that defines an undefined symbol.  Since a sorted list ONLY works when there is exactly ONE object file in the library that defines each symbol this may fail when the -c flag is turned on because there maybe may the same common symbol in many of the object files in that library.  In this case ranlib prints a message and an unsorted table of contents (like the -a option) is produced.  Then, every time the link editor is used on that library it issues a warning about the table of contents not being sorted and that slower link editing will result.  There are some uncommon uses of libraries that must use this unsorted symbol table to get the functionality required.

The key thing to remember in all of this is that the link editor (ld) does NOT search the object file's symbol tables of the library members to determine what symbols are defined.  It ONLY searches the table of contents produced by ranlib.