'(Lisp Development)

From time to time, the question of how one does Lisp development comes up on comp.lang.lisp. As I am not an expert at Lisp development, I think I am in a perfect position to take a stab at answering the question. The answer is by no means complete as it only applies to my own personal experience. I also don't tell you how to do anything here. This is the view from Mt. Everest.

OK, I don't know for a fact that most Lispers use Emacs, but I strongly suspect that it's true. There are two major flavors of Emacs in use that I know of. There is GNU Emacs and XEmacs. I currently use GNU Emacs.

SLIME is a newish Emacs mode for hacking Common Lisp. SLIME turns Emacs into an IDE that provides similar behavior for some half dozen Common Lisp implementations for both Emacs and XEmacs running on Linux/Unix, OS X, and even Windows. SLIME is my current choice for a Lisp development environment.

Debian's Linux x86 build of Emacs running in Apple's Terminal application via ssh:

My Carbon build of Emacs running SLIME:

Here is a larger image of Emacs in action (different contents).

I don't normally run with the window split. I just split the windows to show off SLIME's REPL along with some content. The terminal version is showing some of the HTML of this page instead of a Lisp program.

I guess the most common way to work on a Lisp project is to simply start Emacs and use the M-x slime command. That's what I did for my fractal program. Another way is to connect to a running Lisp process with M-x slime-connect. This is what I do with the Carbon application that I am working on. The latter method is also useful for long running Lisp processes.

Just as in C, C++, Java, Perl, Python, etc, Lisp code is kept in files. All the normal editing operations are performed on files. In this respect, hacking in Lisp is like hacking in any other language that you are used to. What's different is that what you are hacking is a running Lisp program. When you edit a function definition or add a new one, you compile it into a running program. There is no compile, link, run, debug cycle as you know it from C or Java.

Ponder that for a minute.

When you fix a bug in a C function, you have to recompile, relink, and reload your program before you can test the fix. You don't do that in Lisp. You make the fix and then go straight to testing it. This process can be even faster than fixing a bug in a scripting language like Perl.

As you develop a Lisp project, you will end up defining functions, macros, classes, etc in different files. How is this all organized? In C you have makefiles. In Lisp several methods have been devised over the years. The one I use has gained a lot of favor in the free Lisp community and is called ASDF. ASDF is to Lisp what Make is to C.

Well yes, I suppose they do. Maybe it's better to think retro. Looks can be deceiving. In competent hands, these tools are actually quite productive. The learning curve is not necessarily any steeper than it would be with fancier GUI tools. I've actually gotten more done with Emacs and Lisp than with Xcode and C. Also what I've shown here is free software. As a rule, the free stuff tends not to be as polished as proprietary stuff. Then again, there are LispWorks users who use Emacs + SLIME. So give it a chance. You can't really test out the waters without swimming in the deep end anyway.

If you still need convincing, then you need to see...

If you can't say it with words or pictures, then perhaps moving pictures with audio words will do it for you. I must confess this movie is awesome! (150 MB of awesome).

Please check to see if you still have a pulse. I sure was inspired. I was inspired enough to check out Tramp. I was very happy to find that Carbon Emacs from GNU's CVS repository already includes Tramp. Since then, I've been editing my web site with Carbon Emacs over an ssh tunnel instead of logging in with a terminal and firing up a terminal version of emacs.

I've gone even further. Naturally my web server has SBCL installed. I decided I too would like to be able to do remote Lisp programming from Carbon Emacs. Rather than setup a ~/.sbclrc file, I've gone and just made a shell script for starting SBCL and setting it up with Swank. As the Perl folks like to say, TIMTOWTDI. I've tested it and I can connect from Carbon Emacs just fine. In the process I finally took the trouble to set up some ssh keys with ssh-keygen so that I don't have to keep sending my password over the Internet (even though it is encrypted). Here is how I start SBCL and leave it running (unless I make the mistake of typing ,s in the REPL buffer which closes all Lisps SLIME is talking to).

What this does for me is run SBCL as a detachtty process. Detachtty is a clever utility for when you don't want all the complexity of screen. What? You don't know what those are? Hmmm. Perhaps you are not a Unix person. The point is, SBCL is still running when I log off. It runs until I tell it to stop or there's a power failure. With SLIME, I can connect to the running SBCL using ssh tunneling just as you saw in the movie (if you bothered to watch it).

I haven't got everything quite setup the way I would like it yet though. Marco mentioned that it is necessary to do file name translation because Emacs is running on a different machine from Lisp. I also don't like setting up port forwarding in a separate terminal. I would prefer to have that integrated with Emacs so that I can just type a single command to connect to my remote Lisp and have Emacs know to translate the file names that the remote Lisp needs but not try to translate the file names of local Lisp files I may be working on at the same time on a local Lisp instance.

Developing software in Lisp is a highly interactive experience. It is also an immersive (that really should be a word) experience. Like any other hacker, a Lisp programmer becomes enmeshed with the development environment. The interactive nature of the environment does not interrupt the flow of thought like a compile, link, and load cycle does. Lisp also lets you quickly bury the minutia and build up to the domain level.

To be sure, Lisp is no magic bullet. It won't solve all your programming ills. Lisp does not remove the need for intelligence or thought from the programmer. What Lisp does do is free your mind from the shackles of non-abstraction.