Goingware logo

Debugging Software on the Macintosh

An introduction to debugging Classic Macintosh software with the MacsBug assembly code debugger.

Terry Teague

Contributed by Terry Teague

February 3, 1995

Introduction

Note: This article applies to Mac OS 9 and earlier versions, or to debugging Classic applications under Mac OS X. -- Mike

Programmers and testers can use a variety of tools on the Macintosh to debug and test software. This document is intended to give some basic instruction for beta testers of Macintosh software products on how to use such tools to provide meaningful bug reports for the developers of those products. This document looks best when viewed with SimpleText.

Under normal operating conditions if software contains errors in programming, and causes a "crash", the user is presented with a System Error or "bomb" alert, telling them what went wrong (usually in a cryptic fashion), and in some cases gives the user an option on how to proceed - often the user either has no choice and has to Restart (this is normally the case in a non-multitasking environment), or the application terminates gracefully and the user can continue working. In extreme cases, where the machine "hangs" or the software is stuck in a loop, the user usually has no indication of any problem, and generally has to do a hard reset of the machine, losing all work (alternatively with System 7.0 or later, pressing the Command-Option-ESC keys will often get you out of trouble - but use it as a last resort, and always Restart normally after that).

With a debugging program ("debugger") installed, when a crash occurs, the debugger takes over control, and presents to the user information about what went wrong, then waits for user input to decide what to do next. There are two basic types of debuggers - "low-level" and "source (or high) level" - the former relies less on the operating system and can continue to work after a serious crash has corrupted important system memory areas, but it is more difficult to use; the latter uses more of the operating system, is much easier to use, and allows debugging at the source code level, provided source code is available. Examples of debuggers are : MacsBug (Apple Computer, low-level), TMON (Icom Simulations, low-level), The Debugger (Jasik Designs, low and high level), MW Debug (MetroWerks, low and high level), SourceBug, VoodooMonkey, and Macintosh Debugger (Apple Computer, high-level).

Other useful tools to have are : ResEdit (Apple Computer) a resource editing tool, Programmer's Key (freely distributable) for use with debuggers (see below for more info), Discipline (Apple Computer) for spotting common programming errors, DoubleTrouble (Apple Computer) for spotting common programming errors, DisposeResource (Apple Computer) for spotting common programming errors, Leaks dcmd (Apple Computer) for finding memory "leaks", EvenBetterBusError (Apple Computer) for spotting common programming errors, Swatch (freely distributable) for observing memory usage, ZoneRanger (freely distributable or available from MetroWorks) for observing memory usage. Most of these products are freely available, or available for a nominal fee. There are some commercial products available that do the same job or better, but if you already have these, you probably don't need this document.

Also recommended are SCSI Probe (freely available), and some kind of extensions manager (such as Extensions Manager which is freely available, or as part of System 7.5 software). A utility for getting a profile of your environment (i.e. list extensions, System version, hardware configuration) is also highly recommended - there are several that come bundled as part of a commercial package. e.g. TAA is part of CE Software's products, Now Profile is part of Now Software's products, System Info is part of Norton Utilities v3.0 or later; Apple Personal Diagnostics (Apple Computer) provides extensive profile information. There are other FreeWare or ShareWare products such as TechTool, TattleTech, MacCheck. For taking screen shots, you can use the built-in System software (Command-Shift-3 etc), but there are various FreeWare, ShareWare and commercial alternatives that are available - Flash-It! (ShareWare) is recommended. In the case of frequent crashes, a utility to check to health of your hard disk directories, such as Disk First Aid, should be used on a regular basis.

This document is intended mainly for debugging on 680x0 based machines; although many of the techniques can be used on Power Macintosh machines, there are some different requirements for debugging native PowerPC software, which is beyond the scope of this document. This document will only deal with the use of the MacsBug debugger. All references to using System software features, will assume System 7.0 or later (referred to as System 7.x; System 6.0.8 or earlier is referred to as System 6.0.x); however this does NOT mean you shouldn't test the product with other versions of the system software!

Getting all the pieces

Much of the software described above is copyrighted by Apple Computer, and is not intended to be freely distributed. It is generally available to Macintosh developers on CD-ROM and various online services, as part of their subscription. Some of the software is available on Apple authorized online services, such as America Online, AppleLink, CompuServe, eWorld, GEnie, and on various Apple operated anonymous FTP sites. This document does not intend to list specific places from which the software can be obtained.

However for reference, the following information may be useful : MacsBug was last released offically at v6.2.2 (Jul 91); since then however there have been many pre-release versions (not all made publically available), that have been created to support new hardware and new System software, particularly machines using the 68LC040 CPU, PowerBooks, Power Macintosh, and System 7.x. The latest version available at the time of writing is v6.5d9 (Jul 94). If you are using a modern machine, it is recommended that you get the latest version of MacsBug. Most recent versions of some of the other tools are at the time of writing : Programmer's Key v1.4.2 (Dec 91), DisposeResource v1.0 (Jun 92), DoubleTrouble v1.0 (Jan 92), Leaks (Oct 90), EvenBetterBusError (Apr 91), Discipline v2.0.2 (Dec 91), ResEdit v2.1.3 (Jul 94), Swatch v1.7 or v1.8 (both pre-release), ZoneRanger v1.2 (Oct 94), SCSI Probe v3.5 (Dec 92), Extensions Manager v2.0.1 (Jul 93) or v3.0 (part of System 7.5 - Aug 94), Now Profile v5.0.1 (Dec 94), System Info v3.1 (Sep 94), MacCheck v1.0.5 (Dec 93), Flash-It! v3.0.2 (Apr 93).

Installation

  • Place the MacsBug file into the System folder. If using MacsBug versions prior to v6.5d6, you should also place the "Debugger Prefs" file in the System folder (newer versions of MacsBug now include the resources that were in the standard Debugger Prefs file).

  • Some machines have the following feature built-in, but it is recommended that you install the Programmer's Key extension - this allows you to use the keyboard to access features of the debugger, rather than using the "programmer's switch" that is available (often with difficulty) on most (but not all) machines. Place the extension into a closed System folder to have the extension automatically placed in the Extensions folder.

  • The other optional debugging tools may be installed according to their documentation - in some cases this means placing an control panel/extension into a closed System folder to have the control panel/extension automatically placed in the Control Panels/Extensions folders, in other cases you need to use ResEdit to install some resources in the Debugger Prefs file which resides within the System folder.

  • Restart the machine. If MacsBug is installed correctly, a message will appear on the "Welcome to Macintosh" screen, saying so.

  • To test that MacsBug is working correctly, either simultaneously press the Command key and Power button (desktop machines with ADB keyboard only), or simultaneously press the Command key and tilde (~) key (most other machines, such as PowerBook, Macintosh Plus), or press the programmer's switch (interrupt button - often marked as a downward arrow symbol within a circle - make sure you hit the right button, otherwise you will restart your machine!). If all is well, you will be landed in MacsBug (this is often referred to as "drop into MacsBug"), some information will be displayed, and MacsBug will await your input. Press the G key, then press the Return key to return to where you were previously. If the machine hangs, or a program appears to be in a loop, you can often use programmer's key or switch and MacsBug to get you out of such a situation.

Basics of MacsBug

Whenever you are in MacsBug, you can enter "help" or "?" (no quotes) to get help on any aspect of using MacsBug.

Some basic commands to be used are (they can be typed in any mixture of case, and multiple commands per line can be used if separated by semicolons) :

  • HOW = How - displays the reason why you dropped into MacsBug. NMI is displayed whenever you use the programmer's key or switch to drop into MacsBug. "User Break" is displayed whenever a programmer has deliberately inserted debugging statements into the code (maybe for unexpected situations). The reason is always displayed when you drop into MacsBug, but it is often useful to use this command later when recording information to a log file.

  • G = Go - returns you to the currently executing software at the place it was just prior to dropping into MacsBug (i.e. continues execution) - whether this is successful or not depends on how bad the crash was etc.

  • ES = ExitToShell - returns you to the "shell application" - in a multi-tasking environment, this is usually Finder. Some cleanup of the currently executing sofware is usually done (but NOT always - be particularly careful of communications software or extensions etc) before the software is terminated and you are returned to the shell application. If you were in Finder at the time of the crash, doing ES will usually result in Finder being re-launched. This is usually a relatively safe way of exiting from a crash - other software that is running at the crash is often not affected. However it is recommended that you save your work, and Restart as soon as you can, after a crash.

  • RS = Restart - unmounts all mounted volumes, and does a Restart. Use this command, if you are unable to Restart normally using Finder, or the crash is particularly bad.

  • DX = Debugging Toggle - if a programmer has deliberately inserted debugging statements into the code (User Break), and you keep being dropped into MacsBug frequently enough that it is annoying to use the software, and there doesn't appear to be a real problem, then you can turn off ALL User Breaks by using the DX command (and you will not be dropped into MacsBug anymore due to User Breaks), until next time you reboot or use the DX command (either toggle or specify ON).

  • SC = Stack Crawl - shows (usually) the sequence of code that was executed prior to a crash - it can be useful to determine how the crash happened. Sometimes a SC command will give you no useful information or an error message - if this happens, an alternative command SC7 (stack crawl using the stack's A7 register) can produce information (not always useful).

  • IP = Instruction list by half-Page - shows the currently executing code as 680x0 assembly code, in context around the current program counter (PC) - the PC is marked by an asterisk (*) in the listing. Often looking at the instructions around the crash can determine how the crash happened.

  • TD = Total Display - shows all the 680x0 machine registers. Often looking at common registers (e.g. any of the address registers such as A0; Macintosh ToolBox result codes in D0) can determine how the crash happened.

  • HC = Heap Check - checks the validity of a memory "heap". If invalid (corrupted by errant software), a message will displayed, indicating this. Sometimes displaying memory around the area of the corruption can determine how the crash happened.

  • HT = Heap Total - give statistics on the current memory heap. This can be useful to determine if a low-memory condition occurred.

  • LOG filename = Log to a file - when a filename is specified, a text file will be created, and any subsequent MacsBug output will be sent to both the screen and the file. When you wish to complete logging, type the Log command with no additional parameters, and the current log will be closed. A word of warning - sometimes memory areas used by the Macintosh operating system and specifically the File Manager can get corrupted, or the File Manager can be "busy" when you drop into MacsBug - trying to save a log is almost guaranteed to hang the machine in these cases. If you are debugging code that patches the File Manager (such as transparent compression software), or you have other reasons to believe saving a log may be dangerous, it is recommended that critical information be gathered and written down on paper first. Also if on a PowerBook, you will probably have problems if a hard drive you are trying to save the log to, has "spun down".

  • DM startaddress numberofbytes = Display Memory - allows you to display memory. Normally memory is displayed in hexadecimal and ASCII, but MacsBug has a feature called "templates" (see below) that allows you to display memory in more meaningful ways.

  • MCD = Macro Definitions - MacsBug supports the feature of macros - shortcuts for typing difficult to remember or complex commands and/or values (e.g. mnenomics for low memory locations used by the operating system). MacsBug comes with a number of pre-defined macros - use the MCD command to list them. To use a macro, just type it anywhere as part of a command - e.g. a useful macro to use (if available) is WindList - this will show information (in an easy to read format) on all open (not necessarily visible) windows for the currently executing software; thereafter pressing Return will step through the list of windows automatically until the end is reached. You can create your own macros (to create permanent macros use ResEdit).

  • TMP = Templates - MacsBug supports the feature of templates - a way of displaying memory in more human-readable fashion. Many important operating system data structures can be displayed using templates (e.g. window records). MacsBug comes with a number of pre-defined templates - use the TMP command to list them. To use a template with the DM command, after specifying the start address, add the template name to the command line - e.g. DM 09D6^ WindowRecord. You can create your own templates using ResEdit.

  • ?DCMDS = Help on (optional) Debugger Commands - Macsbug supports the feature of debugger extensions, called "dcmd's" - a way of extending the functionality of MacsBug. MacsBug comes with a number of pre-defined dcmd's, and many other optional dcmd's are available from Apple Computer and third-parties - use the ?DCMDS command to list them, and the syntax of how to use them. To use a dcmd, first it must have been installed into MacsBug or the Debugger Prefs file, and secondly, type the command as you would for any other command - e.g. vol (will list all online volumes), or file (will list all open files).

Given the above information, you are now prepared to gather debugging information whenever a crash occurs. Newer versions of MacsBug come standard with a macro called StdLog, which when invoked will create a log file, record useful information for you, and close the log. MacsBug also offers another feature that can be useful in some circumstances - the ability for a macro to be executed automatically whenever you drop into MacsBug. To use this feature, you need to create a macro called Everytime. This macro can do anything, including doing a Go to automatically return to the currently executing software. Of course you must create this macro carefully, otherwise you can get yourself into difficult situations.

As a minimum, whenever a crash occurs, you should enter the following commands :

LOG logfilename HOW SC (or SC7) IP TD HC HT LOG G (or ES)

Advanced MacsBug

If you are a bit more adventurous, there are many other MacsBug commands you can use to aid in debugging. Some of the following commands may also involve some of the other utilities that were mentioned earlier in the document.

  • HS = Heap Scramble - whenever a Memory Manager routine is called, MacsBug will "scramble" the current memory "heap", by moving blocks of memory around. This feature is designed to find any programming errors caused by assumptions made about memory "handles". It will slow your machine down considerably - the speed is related to the amount of memory being scrambled, and you can often influence this by changing the application memory partition size using Get Info in Finder. The slow speed can have a useful side-effect, that of being able to observe user interface problems such as incorrect redraws etc. To test low-memory conditions, set the application memory partition size to as low as you can make it.

  • ATHCA = A-Trap Heap Check from Application - can be used in conjunction with HS to check the validity of the memory heap on every (or the specified list) Macintosh ToolBox or Operating System call (A-Trap) that is made from the application - this helps to narrow down when memory corruption occurs.

  • HX = Heap Exchange - allows you to look at other memory heaps, such as the System Heap.

  • HD 'CODE' = Heap Dump 'CODE' resources - shows all CODE segments/resources (that make up an executing program) that are currently in memory (and whether they are locked, purgeable etc). Some software will use other kinds of code (e.g. XCMD), and sometimes special loader programs will make it difficult to find such code in memory (e.g. HyperCard when it loads XCMDs). Of course the Heap Dump command can be used to find any kind of resource (e.g. ALRT, DLOG, DITL). In conjunction with the "file" (or if available, "rd") dcmd, you can tell exactly what files these resources come from.

  • SB 12D FF = Set Byte at 12D to FF - this causes the operating system to drop you into MacsBug each time a CODE segment is loaded from disk. This can be useful to track down programming errors that occur as a result of assumptions being made about how code is executed, or can be used to track performance problems. Be careful not to "step into" the code that is executing in ROM for the SegLoad A-Trap at this point, or strange things will happen (it is best to just poke around memory at critical points to see what code is being loaded, and then let execution occur normally). Also other problems can occur if the SegLoad A-Trap has been patched by the software. To turn off this feature do a SB 12D 00. It is possible to turn this feature on before an application has launched, to track the initialization phases, but initialization is heavily dependent on the development environment used, and explanation of this process is beyond the scope of this document.

  • SS 0 = Step Spy at location 0 - this feature is designed to find any programming errors caused by assumptions made about memory "handles", by seeing if any code writes to memory location 0 (which is a no-no). It will slow your machine down very much. It is also not recommended if you are on a PowerBook, or using EvenBetterBusError, since they deliberately write to location 0 frequently. SS can also be used on any range of memory addresses - it can be used on the stack to see if particular routines are corrupting the stack (the symptom of this is code that jumps to random places in memory). The advantage of SS is that it stops at the specific instruction that causes the corruption, rather than possibly thousands of instructions later as would be the case with some other MacsBug commands to detect corruption.

  • T = Trace (or Step Out) - allows you to execute code one instruction at a time, except subroutine calls and A-Traps are treated as a single instruction.

  • S = Step (or Step In) - allows you to execute code one instruction at a time, including stepping into subroutine calls and A-Traps.

  • DM CurMap = Dump Memory from CurMap (A5A) - if you don't have the optional "rd" dcmd installed, this command can be useful in conjunction with the "file" dcmd, to determine the current resource file (top of resource map chain). This can help track down problems with resources being loaded from the wrong file.

If you are debugging software that involves communications or the File and Device Managers, you will often run into a situation where the machine appears to be hung. If you drop into MacsBug in these situations and do an IP command, you will often see code that looks like the following :

_vSyncWait +0000 4080BB8C MOVE.W $0010(A0),D0 +0004 4080BB90 BGT.S _vSyncWait ; 4080BB8C

You may also see the actual routine name "SyncWait" or "_vSyncWait" somewhere in the listing (the above is in ROM). This tight little loop is waiting for some I/O to finish, and it will never complete (for some reason). Entering the following command can be very useful to find out what I/O operation was attempted at the time :

DM A0 IOParamBlockRec

And sometimes when you are desparate, setting the I/O result field to a zero or negative value with the following command, can allow execution to continue (perhaps with unpredictable results) :

SB A0+10 00

There are many other techniques for debugging, but discussion of such techniques is best left to the documentation that accompanies the specific debugging tools.

© 1995 T. R. Teague. All rights reserved.

Apple, the Apple logo, AppleShare, LaserWriter, Macintosh, PowerBook, and Macintosh Quadra are trademarks of Apple Computer, Inc., registered in the U.S.A. and other countries. AudioVision, Balloon Help, Macintosh PC Exchange, Power Macintosh, and QuickDraw are trademarks of Apple Computer, Inc. All other product names are trademarks or registered trademarks of their respective holders. Mention of non-Apple products is for informational purposes and constitutes neither an endorsement nor a recommendation. Apple assumes no responsibility with regard to the selection, performance, or use of these products.

Permission is hereby granted to use this document in any manner seen fit, providing the above copyright notice is retained.

Vote for Us at the Programming Pages

Voting for GoingWare at The Programming Pages will encourage more people to read these articles.

Call GoingWare: +1 (831) 401-3790, info@goingware.com