MacBench 4.0 is a subsystem-level benchmark that measures the performance of a Mac OS system's graphics, disk, processor, FPU, video, and CD-ROM subsystems.

MacBench normalizes all scores relative to the base machine, a Power Macintosh 6100/60. The base machine receives a score of 100 MacBench 4.0 units on the main tests. For all MacBench tests, bigger numbers mean better performance .

MacBench's new Graphics technology reproduces the graphics operations performed by leading Mac OS business and publishing applications, affording you a better-than-ever measure of your PC's graphics subsystem. MacBench runs the following main tests that provide an overview of a system's graphics, disk, processor, FPU, and CD-ROM subsystems performance:

Processor Floating Point Unit Disk Graphics CD-ROM

Along with its market-centered tests, MacBench also includes Publishing Disk and Publishing Graphics tests (the latter in both low and high resolution formats) that reproduce the specialized activity of more demanding applications.

A new addition to MacBench are full-motion video tests that measure a Mac OS system's video subsystem performance. You can even have MacBench play a movie file of your choice.

MacBench 4.0 in a nutshell

MacBench is a subsystem-level benchmark for all Mac OS systems -- Apple Macintosh and Power Macintosh computers, and Mac OS-compatible systems. MacBench measures the performance of a Mac OS system's processor, floating point, disk, graphics, CD-ROM, and video subsystems.

For MacBench 4.0, we recorded the graphics, disk, and CD-ROM activity performed by top-selling Mac OS products, including database, business graphics and desktop publishing, spreadsheet, and word processing applications. MacBench's tests reflect how well your Mac OS system should run popular Macintosh applications.

Note MacBench 4.0 doesn't run actual applications during its tests.

Measuring subsystems, not components

MacBench 4.0 measures the performance of subsystems, not the performance of individual components. For example, the hard disk is part of the disk subsystem and does not affect the results for graphics tests. So, instead of talking about individual components, it makes more sense to talk about a Mac OS system's subsystems and what composes those subsystems.

Although MacBench's tests attempt to measure the performance of individual subsystems, all these tests must run on the processor. Thus, the processor subsystem necessarily affects results for every test. In our testing, we've also noted that the disk subsystem tends to be the bottleneck for most business systems.

As we see it, a Mac OS system has the following basic subsystems: processor (including the floating point unit), disk, graphics, and CD-ROM. The next sections discuss each individual subsystem, enumerating the components in each subsystem, and describes factors that may affect test results.

The processor subsystem

We consider the components of the processor subsystem to include: CPU FPU RAM Any L1 or L2 cache Bus architecture Any CPU accelerator card

Factors about the processor subsystem that can affect MacBench's results include: The CPU model and clock speed. As a rule of thumb, the faster the CPU, the better. Bus speed. How memory is interleaved.

Does the CPU process data in 16-, 32-bit, or 64-bit chunks and do its peripherals talk to the CPU in 16-, 32-bit, or 64-bit chunks. The more information the processor can send, the faster the whole computer will seem to run.

Within a processor family, processor test scores won't be too different from each other. For example, all 68040 processors running at the same speed will score about the same on the Processor Test.

Of course, faster processors outperform slower ones, and more powerful processors score better than lower-powered processors. FPU scores also tend to be stable across families of Mac OS systems, but a hardware FPU outscores a software FPU every time.

Note For the purpose of MacBench's tests, the disk, graphics, and CD-ROM subsystems also include the processor subsystem (CPU, memory, cache, and bus) because the processor subsystem executes the tests and, thus, affects results.

The disk subsystem

The disk subsystem includes your Mac OS system's: Hard disk type SCSI interface SCSI software Disk cache software Disk compression software Bus used to carry information to and from the processor subsystem Whether you have enough room on the hard disk and whether the hard disk is fragmented are also important factors in your systems disk performance.

Factors about the disk subsystem that can affect a Mac OS system's performance include: Presence of a hardware disk cache and its size. Presence of a software disk cache and its size. Amount of available free disk space. Whether the software disk cache is caching writes. Disk subsystem performance generally improves when caching writes. Disk compression, which may slow performance. Operating system software. The amount of free disk space on the hard disk. Whether the hard disk is fragmented.

We recommend you defragment a Mac OS system's hard disk each time you run any MacBench disk test. You can use any software defragmenting utility to defragment your system's hard disk. A defragmenting utility gathers all the separate parts of a fragmented file and sequentially orders them.

The graphics and video subsystem

Your Mac OS system's graphics and video subsystem includes: Monitor Graphics card Any QuickDraw- or graphics-accelerator cards Display driver Bus used to carry information to and from the processor subsystem

You should remember two important facts about your Mac OS system's graphics subsystem.

CPU performance directly affects graphics subsystem performance, because the CPU handles many QuickDraw instructions.

The bus, which shuttles graphics information between the graphics adapter and the CPU, can directly affect graphics subsystem performance.

Factors about the graphics subsystem that can affect MacBench results include: Graphics card Amount of graphics RAM (video RAM) Display driver software Number of colors the monitor's displaying Monitor's resolution; we usually test monitors set at a 640-by-480 resolution Presence of any QuickDraw or other graphics accelerator cards

The CD-ROM subsystem

Your Mac OS system's CD-ROM subsystem includes: CD-ROM drive Adapter to which the drive is connected Software drivers it requires to operate Any disk caches Bus used to carry information from the controller to and from the processor subsystem Factors about the CD-ROM subsystem that can affect MacBench results include: Presence of a hardware CD-ROM cache and its size. Presence of a software CD-ROM cache and its size. Operating system software. Clean the CD-ROM.

MacBench's base machine

A software-based benchmark works best with a single computer as a reference point. The benchmark can then compare all scores gathered from other computers to this reference point. For MacBench, that computer (which we call the "base machine") is a Power Macintosh 6100/60, which is equipped with a PowerPC 601 processor running at 60 MHz, 16MB of RAM, a 258MB hard disk, an AV graphics card, and running System 7.5.5.

All of MacBench's scores are relative to the Power Macintosh 6100/60; if the Power Macintosh 6100/60 scores 100 on a test and your Mac OS system scores 200, then your system is twice as fast at that test than the Power Macintosh 6100/60.

About the market-centered and publishing tests

Like previous versions of the benchmark, MacBench 4.0 offers "market-centered" tests based on business applications. The goal of these tests is to measure the level of performance typical users will achieve while performing common operations with leading business applications.

MacBench also provides tests based on high-end publishing applications. The publishing applications, and the tasks they perform, place a wide range of performance stress on Mac OS systems.

MacBench's Disk, Publishing Disk, Graphics, and Publishing Graphics tests play back the disk and graphics subsystem operations we recorded for the market-centered and publishing applications.

Market-centered tests

MacBench uses "market-centered" tests in the, for example, Disk and Graphics playback tests. Market-centered tests aim to measure the level of performance typical business users will achieve while performing common operations with leading Mac OS-based business applications.

The market-centered results will be useful to the vast majority of users because most users spend a great deal of computing time working with the same basic types of programs--such as word processing, database, or spreadsheet applications--during the course of their day. Such users will gain great value from the Disk and Graphics tests. (The weights we use to calculate the scores reflect confidential market-share information from Computer Intelligence.)

We picked these applications because of their high current and projected market shares by unit of the Mac OS system software market. This market-centered approach lets MacBench reflect the center of the software market and provides you with a more realistic view how your Mac OS system can run today's software.

Publishing tests

While the market-centered tests reflect the bulk of the software market, the publishing tests will be most helpful to users of more demanding, specialized software -- specifically publishing software. The publishing applications and the work the tasks they perform in the Publishing Disk and Publishing Graphics tests represent particularly demanding work.

High-end publishing applications tend to be performance sensitive and are more likely than business applications to show the differences, for example, between 32MB of RAM and 64MB of RAM. (In Business applications, those RAM gains have lower returns.) Because these applications are more demanding, their users often care more about performance in general and the performance of their particular applications.

For the Publishing Disk and Publishing Graphics tests, MacBench uses two applications: Adobe Photoshop and QuarkXPress . Although these applications are also present in the market-centered tests, the publishing tests place more demands on the applications, in the form of larger file sizes and more intensive tasks.

About ZD benchmarks

Application-based. This test runs real applications through a series of scripted activities and uses the time a computer requires to complete those activities to produce a performance score. The score is a performance metric for an entire system. Winstone 97 is the primary example of an application-based benchmark.

Ziff-Davis uses proprietary profiling tools, proprietary playback engines, and a rigorous methodology to accurately test a computers subsystem.

Playback. ZD's proprietary playback technology acts rather like a subsystem-level VCR, playing back the previously recorded subsystem-level activities the profiled applications made while executing the scripts. Using ZD-proprietary tools, ZDBOp developers recorded logs of the graphics, disk, and CD-ROM subsystem activity of the applications. These logs contained every graphics (or disk or CD-ROM) function the applications performed. When a playback test starts, the benchmark uses these logs, which come on the Mac OS Benchmark CD-ROM, to literally play back those stored calls. MacBench 4.0's Disk, Graphics, and CD-ROM playback tests fall into this category.

Synthetic. A synthetic test is the best choice when application-based or playback tests are impractical or technically impossible to implement. For this kind of test, we profile application activity in a subsystem and run a statistical approximation of the operations that subsystem performed. MacBench's Processor and FPU tests are examples of synthetic tests.

Inspection. Inspection tests are non-profiled tests that execute a specific operation in a specific subsystem. You might want to run these tests when you conduct more specific investigations of a subsystem's application-based, playback, or synthetic test scores. MacBench includes inspection tests for the disk, graphics, and CD-ROM subsystems. An example of such a test is the Random Read 64K disk test.

Note: Be aware that it's very easy to take an inspection test's result out of context. The application-based, playback, and synthetic tests provide more rounded and accurate views of a system's or subsystem's performance than a single component test.

What's New in this 4.0 Release

New! Graphics technology

Previous versions of MacBench featured startlingly accurate disk and CD-ROM playback technology. Based on the same principles, MacBench now adds full-scale graphics playback as well. MacBench is the first benchmark in the Mac OS realm to offer this technology in the desktop environment.

MacBench doesn't include the time required for reading the log files in its results.

MacBench Graphics and Publishing Graphics tests use ZD-proprietary playback technology to capture most of the graphics operations a Mac OS application performs and then replay those operations. So, when a Graphics or Publishing Graphics test is running, it appears as if the application itself is running (except for slight pauses every few seconds while the test reads more data from the test log and for the rapid pace of the replay). Of course, the application itself isn't running; instead, MacBench is replaying all of the graphics subsystem operations the application made.

To create the graphics playback tests, we designed a tool that captures the graphics subsystem calls each Mac OS application made during our profiling. We then created a graphics replay engine to replay the raw log files containing the graphics operations.

New! Video tests

MacBench 4.0 now offers full-motion video tests you can use to test video subsystem performance. MacBench provides four short movies in a variety of formats and sizes. If you like, you can also play a movie of your choice and control how it's played. Use MacBench 4.0's redesigned Test Settings dialog box to adjust these controls. For more information, see "" on page .

MacBench offers two video tests: Frames Played and Maximum Frame Rate. For each test, you can play MacBench's four built-in movies plus one user-defined clip. If you're playing the built-in movies, you can play them from either the MacBench CD-ROM or a hard disk. However, the user-defined movie can be on any hard drive or CD-ROM drive accessible to MacBench.

The Frames Played test plays the movie at its correct speed (with sound, if sound is included) and reports the percentage of frames the system displayed. The Maximum Frame Rate test plays the movie as fast as possible with the sound off. The test returns the average frame rate for that movie, reported as frames per second.

Applications we profiled

MacBench 4.0 contains 11 profiled applications. We profiled MacBench 4.0's disk, graphics, and CD-ROM playback tests using the following applications. The list also indicates whether the applications ran in native or emulation mode.

Adobe Illustrator 6 (native) Adobe Photoshop 3.5 (native) Adobe PageMaker 6 (native) ClarisWorks 3 (native) FileMaker Pro 2.1v4 (emulation) Microsoft Excel 5a (native) Microsoft PowerPoint 4 (native) Microsoft Works for Macintosh 4 (native) Microsoft Word 6.1a (native) QuarkXPress 3.32 (native) WordPerfect 3.5 (native)

For the Publishing Graphics tests, we profiled QuarkXPress and Photoshop. However, the Publishing script we used to profile these applications was different and more representative of high-end users' work than the applications' Graphics script.

We also profiled the following CD-ROM applications for the MacBench CD-ROM tests:

Macintosh System 7.5 Golden Master Corel Professional Photos CD-ROM: World's Best Photos Disney's Animated Storybook, Toy Story Intuit Quicken Deluxe CD-ROM Version 6 Jellyvision, Inc.'s You Don't Know Jack

MacBench minimum system requirements

If the Mac OS system doesn't meet MacBench's minimum requirements, MacBench won't be able to run its tests properly. The following list describes the minimum requirements to install and run MacBench.

A 68030 processor or better System 7.5 or better 12MB or more of physical RAM A CD-ROM drive (required for the CD-ROM and standard video tests) Apple CD-ROM driver extension or equivalent. You need this driver if you're running MacBench off a CD-ROM or want to run the CD-ROM or video tests. A 13-inch or better monitor size

Minimum display resolution of 640x480 (for the Hi-Res Publishing Graphics test, the resolution must be at least 1152x780)

The following amounts of free disk space: for the standard disk inspection tests, 32MB; for the Disk test, 115MB; and for the Publishing Disk, 126MB If you're playing the built-in movies from disk, then you'll also need enough free disk space to hold those movies. The sizes are: CinePak 1 3.4MB CinePak 2 10.5MB Motion JPEG-A 29.7MB Uncompressed 58.1MB

Running time for Main Tests suite

The time MacBench takes to run the Main Tests suite varies based on your Mac OS system's speed and configuration. The following table shows a few examples of how long some machines in our Research Center took to run this suite. Faster machines took less time, while slower machines took more time.

This computer took about this much time:

Quadra 630 71 minutes Power Macintosh 6100/60 55 minutes 35 seconds Power Macintosh 8500/180 18 minutes

Standard Testing Procedures

Keep these guidelines in mind

MacBench 4.0, like any other software, requires a Mac OS system to meet certain conditions. If the system doesn't meet these conditions, the test may not finish successfully.

Before you run any of MacBench 4.0's tests, read the following list of caveats.

1)When MacBench is running, you should not move the mouse, click the mouse button, or press any keys on the keyboard. Clicking a mouse button or pressing keys on the keyboard can interfere with MacBench tests and hurt your systems results.

2)Before running a series of tests, always reboot the Mac OS system. Keep the memory as clean and open as possible by reducing the number of extensions, fonts, and control panels you don't need.

3)On a computer whose hardware and software configuration remains constant, MacBench's scores shouldn't vary by more than 3% from test run to test run. If you're seeing more than a 3% variance, double-check that you're following the above guidelines. If the abnormal variance persists, contact MacBench Technical Support. For information on technical support, turn to "" on page .

The Tests

Processor and Floating Point Tests

About the Processor test

Processor subsystem components include the CPU, FPU, RAM, the bus architecture, and any CPU accelerator card, among other factors.

MacBench's processor tests measure a Mac OS system's processor subsystem performance by running instruction playbacks that carefully emulate the processor activity of real-world applications.

The Processor test focuses solely on a Mac OS system's processor subsystem. The test performs no I/O operations or other functions that might involve a system's graphics or disk subsystems.

The workload for the Processor test is a carefully weighted combination of many different modules. Each module does one or more types of real work, such as:

Compressing data Calculating dates Analyzing words Formatting text

The combined overall workload simulates the processor behavior of leading Mac OS applications.

How we created the Processor test

To ensure the accuracy of the Processor test's simulated workload, we profiled a variety of leading Mac OS applications. The application profiles contained a large set of statistics about how those applications interacted with the processor. Those statistics include such important facts as the types of instructions the applications execute, the frequency of each instruction type, and how often the processor was able to find the data it needed in its RAM cache.

The Processor test reflects the results of the profiling process for Mac OS applications. During the development process, we fine-tuned the test's behavior to correlate highly with the profiled applications' behavior.

About the Floating Point test

The Floating Point test measures how fast your Mac OS system's floating-point unit (FPU) handles floating point calculations. MacBench's Floating Point test, based on our observations and experience, reflects the kinds of tasks the Mac OS's FPU typically performs. This test:

Calculates areas enclosed by polygons that have varying sizes, shapes, and number of sides. Calculates a Poisson Distribution. Performs a Fourier Transform. Solves several linear equations.

About the Processor and FPU scores

For both tests, MacBench records how many operations the test system performs for the duration of the test. Then, MacBench compares this result to how many operations the base machine performed in the same amount of time.

For both tests, the base machine -- a Power Macintosh 6100/60 -- reports a score of 100. With these scores, bigger numbers mean better performance .

Both the Processor and Floating Point tests produce relative scores: Their results are normalized, unitless numbers meaningful only when you compare them to other MacBench 4.0 processor and floating point results. They are good indicators of the relative power of your Mac OS system's processor subsystem.

Calculating the Processor and FPU scores

MacBench uses the same procedure to calculate the Processor and Floating Point test scores.

After running a test, MacBench computes an unweighted, absolute score for the test. Then, MacBench scales that number to make its meaning more intuitive. Thus, scores for the Processor and Floating Point tests are unitless, relative numbers--they are meaningful only when you compare them to other MacBench 4.0 processor scores.

Disk Tests

Disk playback tests

The disk subsystem includes the hard disk, disk controller, disk device drivers, any disk caches (hardware or software) you have installed, and the bus used to carry information from the disk controller to and from the processor subsystem.

The disk playback tests provide an overall comparative measure of the disk subsystem's performance as compared to the base machine's.

MacBench's Disk and Publishing Disk tests execute the same series of disk operations the profiled applications performed. The disk playback tests employ many different files, folders, and file operations in a blend based on a recording of the disk usage by the profiled applications.

For all disk playback results, bigger numbers are better .

What the disk playback tests do

Here's what happens when you run a disk playback test:

When you select a disk playback test, you may see a warning that the test will take some time to run. (You can turn off this warning in the Preferences dialog box.)

MacBench creates several megabytes worth of test files. This setup process causes all disk tests to take longer on Mac OS systems with slower processors or under-powered hard drives.

The Disk test requires 115MB of free disk space and the Publishing Disk test requires 126MB.

Note The data the disk playback tests use is about as compressible as the data the profiled applications used, so you can get meaningful results from this test even when you're using disk compression products on your Mac OS system.

Each disk playback test is a single large test that performs the following operations on the test files:

Performs read, write, append, file lookup, creation, and deletion operations, and varies the parameters for each operation. The profiled commands include any synchronous and asynchronous File Manager calls we saw in the profiling.

Intersperses the operations to measure how your system's disk subsystem handles complex interactions.

MacBench times how long the test system runs the disk playback test and compares that time to the base machine's. From this timing, MacBench computes the kilobytes (1,024 bytes) per second rate at which the test system read or wrote to the test files. MacBench then uses a weighted harmonic mean to calculate the final playback test score, which shows how your Mac OS system performed in relation to the base machine, a Power Macintosh 6100/60.

How we created the disk playback tests

We gathered information on what types of disk operations Mac OS-based applications perform by profiling 11 top-selling Mac OS-applications for the Disk test.

For the Publishing Disk test, we profiled the disk activity of only two applications: QuarkXPress and Photoshop. The Publishing Disk is based on our profiling of power users employing these applications on high-end publishing tasks, such as copying and moving large image files.

Each disk playback test is a nearly exact replay of the profiled applications' disk activity.

To ensure the accuracy of our profiling, we used commercially available and in-house tools to record application I/O operations, but without interfering with or modifying the original application. Using these tools, we recorded the File Manager commands for each application and combined those recorded logs into the Disk and Publishing Disk tests.

How MacBench computes the disk playback scores

During a disk playback test, MacBench plays back the logs we recorded for each of the 11 profiled applications. The benchmark times how long the test machine plays back each application log. MacBench calculates the transfer rate for each application.

MacBench then uses a weighted harmonic mean of the transfer rates (using the application's market-share weights) to produce the disk playback result. MacBench then normalizes the test system's score to the base machine's score.

Disk inspection tests

Be aware that it's very easy to take an inspection tests result out of context. The playback tests provide a more rounded and accurate view of a subsystems performance than a single component test.

Each MacBench disk inspection test creates a 32MB file. MacBench then accesses the file using fixed-length records in either a sequential or random fashion. For each test, MacBench reads or writes data to or from the hard disk and records the time it takes the disk subsystem to perform the operation.

You can use the inspection tests to help determine different Mac OS systems' Disk scores. For example, if you run the All Disk Tests suite on several Mac OS systems, you can see which operations execute faster on each of those systems.

Graphics Tests

Graphics playback tests

Your Mac OS system's graphics subsystem includes the monitor, video card, any QuickDraw- or graphics-accelerator cards, video driver, and the bus used to carry information to and from the processor subsystem

MacBench 4.0's Graphics tests use an exciting new playback technology that allows us to capture nearly every graphics operation a Mac OS application performs and then replay those operations. While we've always been able to record the disk calls applications made, we're now able to extend that technology to the graphics side.

So, when a Graphics or Publishing Graphics test is running, it appears as if the application itself is running, except extremely quickly. You may see slight pauses every few seconds while the test reads and decompresses data from the log files into RAM. MacBench's playback tests reflect the graphics subsystem performance as the subsystem carries out actual, real-world graphics activity.

What the graphics playback tests do

The graphics playback tests provide an overall comparative measure of the graphics subsystems performance as compared to the base machines. Here's what happens when you run a graphics playback test:

Each graphics playback test is a single large test that reproduces the QuickDraw and Mac OS graphics API commands issued by our profiled applications. MacBench reads log files containing the recorded graphics calls made by each application and reproduces those calls.

As MacBench reads and reproduces the graphics calls, you'll see each application's interface, menus, dialog boxes, and so on, flashing onto the screen, appearing as they did during out profiling.

At the end of the test, MacBench calculates the number of thousands (that's 1,000, not 1,024) of pixels per second the test system drew during the test. MacBench normalizes the final number to produce the graphics playback score.

How we created the graphics playback tests

To create the Graphics and Publishing Graphics tests, we designed a tool that captures the graphics calls each Mac OS application made when we ran the application's script. We then created a graphics replay engine to replay the log files containing the recorded graphics operations.

We gathered information on what types of QuickDraw operations Mac OS-based applications perform by profiling 11 top-selling Mac OS-apps. The Graphics reflects the proportional use of the QuickDraw commands our profiled applications used.

The Publishing Graphics test reflects the profiling of two applications: Photoshop and QuarkXPress. The script we used to profiled these applications was different and more representative of high-end users work than the applications Graphics script.

To ensure the accuracy of our profiling, we used commercially available and in-house tools to record application I/O operations. These tools did not interfere with or modify the original application. Using these tools, we recorded the QuickDraw commands for each of the 11 applications and used the profile logs to create the Graphics.

Publishing Graphics tests and display resolution issues

We recorded the Publishing Graphics tests at two resolutions: 640x480x8 (low-resolution) and 1152x870x8 (high-resolution). MacBench allows you to select from the Tests menu the resolution you want to test.

You cannot run a high-resolution script in a low-resolution environment. If the display is not at the proper resolution, MacBench aborts the high-resolution test with an error message.

The reason for the different resolutions is that the applications display differently at higher resolutions. For example, at lower resolutions, applications may display rows of gray bars instead of text characters. But at higher resolutions, the applications may display the text characters, thereby using different QuickDraw commands and placing extra demands on the graphics subsystem. Because the applications send different graphics commands to the OS based on different display resolutions, MacBench graphics tests reflect that.

Graphics playback scores

The graphics playback tests return a score of 100 for the base machine and normalizes your system's score accordingly. With this result, bigger numbers mean better performance .

You can use the Graphics result to determine how to optimize the Mac OS system's graphics subsystem to best suit your needs. For example, the Graphics results can help you determine how effectively QuickDraw accelerator cards operate in your working environment.

How MacBench computes the graphics playback scores

The time MacBench uses to read in the graphics playback logs isn't included in the timed portion of the test.

During a graphics playback test, MacBench plays back the logs we recorded for each profiled application. The benchmark times how long the test machine plays back each application log. MacBench divides that rate by the amount of time it took the MacBench base machine to complete the same test. In this way, MacBench computes a relative speed for each application in the playback test.

MacBench then uses a weighted harmonic mean to combine the applications' relative speeds to produce a relative speed for an entire category.

Finally, MacBench combines the scores for all the categories using a weighted harmonic mean to produce a single, unitless score for the selected graphics playback test.

CD-ROM Tests

We profiled the following CD-ROM applications for the CD-ROM test: Macintosh System 7.5 Golden Master Corel Professional Photos CD-ROM: World's Best Photos Disney's Animated Storybook, Toy Story Intuit Quicken Deluxe CD-ROM Version 6 Jellyvision, Inc.'s You Don't Know Jack

Note When you run any CD-ROM test, the CD-ROM that includes MacBench must be in the CD-ROM drive so MacBench can find the files it needs to run the test.

CD-ROM test

The CD-ROM subsystem includes a CD-ROM drive, an adapter to which the drive is connected, software drivers, any disk caches, and the bus, which carries information from the controller to and from the processor subsystem

The CD-ROM test provides an overall comparative measure of the CD-ROM subsystem's performance as compared to the base machine's.

The CD-ROM test replays the CD-ROM subsystem activity we recorded for the profiled CD-ROM applications.

When you run the CD-ROM, the CD-ROM that includes MacBench must be in the CD-ROM drive so MacBench can find the files it needs to run the test.

The CD-ROM is a single large test that performs the following operations on the test files:

Read and file lookup operations, with the parameters varying for each operation. The profiled commands include any synchronous and asynchronous File Manager calls we saw in the profiling.

Intersperses the operations to measure how your system's CD-ROM subsystem handles complex interactions.

MacBench times how long the test system runs the CD-ROM and compares that time to the base machine's. From this timing, MacBench computes the kilobytes (1,024 bytes) per second rate at which the test system read the test files. MacBench then uses a weighted harmonic mean to calculate the final CD-ROM score, which shows how your Mac OS system performed in relation to the base machine, a Power Macintosh 6100/60.

How we created the test

We gathered information on what types of CD-ROM operations Mac OS-based applications perform by profiling five top-selling CD-ROM applications. The CD-ROM is an almost exact playback of these operations.

To ensure the accuracy of our profiling, we used commercially available and in-house tools to record application I/O operations, but without interfering with or modifying the original application. Using these tools, we recorded the File Manager commands for each of the five applications and combined them into one CD-ROM.

CD-ROM test scores

The CD-ROM returns a score of 100 for the base machine and normalizes your system's score accordingly. With this result, bigger numbers mean better performance .

Because the test mimics the kind of work top-selling CD-ROM-based applications do, you can use the results as a guide to the kind of throughput you can expect to see when working with the Mac OS system's CD-ROM subsystem.

How MacBench computes the CD-ROM score

When you run the CD-ROM, MacBench runs a single, large test that replays the profile logs we recorded for our five profiled CD-ROM applications. During the test, MacBench times the test system's CD-ROM subsystem operations.

When the CD-ROM completes, MacBench takes the number of kilobytes per second the test system read and uses a weighted harmonic mean to calculate the CD-ROM score. (MacBench bases the weights on each application's total market share.) MacBench then normalizes the test system's score to the base machine's score.

For the CD-ROM, MacBench produces a score that is a unitless, relative number--the number is meaningful only when you compare it to other MacBench 4.0 CD-ROM scores.

Video tests

About the video tests

MacBench's video tests provide concrete measures of a Mac OS system's ability to play full-motion video. Video playback functions stress multiple areas of the system, including the system's processor and graphics subsystems, as well as the CD-ROM or disk subsystems, depending on the location of the movies you're playing.

MacBench provides four movies (footage which WinBench 97's video tests also use) of a carousel . The movies play back the video at various speeds and data rates. MacBench also lets you play a movie of your choice and control how it's played. Use MacBench 4.0's Test Settings dialog box to adjust these controls.

MacBench is designed to test single-video, single-audio movies. If you run a custom video test using a movie that includes multiple video (say, two movies in the same frame), MacBench will return results but the results will have very little meaning.

About the video test results

Power Mac systems return better performance results if virtual memory is turned off .

MacBench offers two video tests: Frames Played and Maximum Frame Rate. For each test, you can play the four MacBench-provided movies plus one user-defined clip.

The Frames Played test plays the movie at its correct speed (with sound, if sound is included) and reports the percentage of frames the system displayed.

The Maximum Frame Rate test plays the movie as fast as possible with the sound off. The result the test returns is the average frame rate for that movie, reported as the number of frames per second.

What Do All These Numbers Mean?

Use the following pointers as a starting place for interpreting your results.

IMPORTANT: Bigger numbers and longer results bars mean better performance than smaller numbers and shorter results bars.

MacBench's Main Tests suite -- consisting of the Processor, Floating Point, Disk, Graphics, and CD-ROM tests (if you have a CD-ROM drive) -- gives you the overall numbers that tell you about the performance of your computer's subsystems as compared to the MacBench base machine's.

The scores returned by the Main Tests suite are relative; they're only meaningful when you compare them to the scores for the Power Macintosh 6100/60, MacBench's base machine. Inspection tests, however, use specific units, such as kilobytes (1,024 bytes) per second for the disk tests or kilo pixels (1,000 pixels) per second for the graphics tests.

You have to judge MacBench's scores in light of your setup and how you use your system every day.

Think subsystems and not components. Getting a hard drive 10 times faster than your old one won't increase your Disk score 10 times. This is because other disk subsystem components--such as the disk cache (and even other subsystems) -- may still slow throughput. Nothing exists in a vacuum; the performance of every component and subsystem in a computer depends on the performance of other components and subsystems.

If you have two Mac OS systems with unequal scores, and you feel the scores should be similar, use MacBench's System Information window or each system's results files to compare every aspect of their configuration and setup. Minimize the differences between the two machines as much as you can. Conduct tests that focus on changing only one component at a time.

Don't compare MacBench 4.0 scores with those from previous versions of MacBench. MacBench 4.0 is a new product built using new technologies and its scores supersede all previous versions.

Know what you're looking for. What aspect of your Mac OS system's performance matters most to you? How you use your Mac OS system every day should influence your MacBench testing.

Remember, a Mac OS system's configuration will affect its MacBench results. Comparing results for two machines with different configurations is at best a potentially confusing experience.

Comparing results

When you compare benchmark results, you can make two main types of meaningful comparisons:

You can compare two similar machines to see which is faster.

You can change a component or setting within a Mac OS system to see how changing that component affects the system's performance.

The following sections explain key issues you should consider in both these cases to make sure your comparison is valid.

Comparing two or more machines

You can use results for comparison in a couple of ways:

You can compare systems that have the same configurations to see which has the best performance.

You can compare systems that have different configurations to see how the differences affect performance.

When you're comparing two similar Mac OS systems, you need to be aware of the hardware and software installed on each. Even seemingly minor differences in setup and configuration can really skew your interpretation of benchmark results.

To make sure the two systems whose results you want to compare have the same basic configurations, you should examine their results files. The results files contain the hardware and software configurations for each system.

When you're comparing results for similar Mac OS systems, you should also make sure you've tested the systems in the same way. The easiest way to get the most accurate and repeatable results is to use a standard testing procedure (see "" on page ). If you follow this procedure every time for each Mac OS system you test, you'll be able to compare their results meaningfully.

Changing components or settings

If you want to compare benchmark results for a Mac OS system whose components you've changed, keep the following in mind:

Make sure the only change to the system is the component you're trying to change.

Be sure to run the same test in the same way before and after you make the component change. For example, if you run the first test with no background applications loaded, be sure to run the second test the same way.

Verify that nothing in the system's configuration changed other than the component you intentionally changed. (You can view the system's configuration information using the System Information window or the results file.)

Why results differ

If you run the same MacBench test on your system repeatedly, you'll get the same score almost all the time. The margin of error in the MacBench processor and graphics scores is about 3 percent. The margin of error is slightly higher for the disk tests, but remains under 5 percent. If you modify any of your Mac OS system's subsystems, you probably will see a difference in results.

What Can Affect Results

Your mileage may vary

Although some of the information in this chapter is tried-and-true, you'll have to evaluate every piece in light of your own needs. Keep in mind the following questions as you read through the chapter: How hard do you really stress your Mac OS system? What applications do you use regularly? How do you use those applications? As you work with these applications daily, what subsystem do they stress the most?

Mac OS system "performance" is a highly subjective issue that can vary from one user to the next and from one system to the next. No book or manual can tell you what your computing needs are and whether your system's current setup meets those needs. You need to decide that for yourself.

General performance tips

Rebuild the desktop file at least monthly. To do this, press Command-option as your Mac OS system boots up. Reboot your Mac OS system each time before you run MacBench. Stop any background tasks, such as screen savers or network applications, before you run MacBench. If your Mac OS system has a 68040 CPU, enable the processor cache. Close all other open applications before you run MacBench. Avoid moving the mouse or pressing keys while MacBench runs its tests.

Processor subsystem tips

Enable processor caches. Remove any control panels, extensions, fonts, sounds, etc. you don't need. All of these programs drag down your system. Use a font-management utility to load only the fonts you need when you need them. Reduce memory overhead by disabling processes such as automatic network connection utilities and application start-up utilities. If you're using your Mac OS system for processor-intensive work, get a CPU accelerator to cope with the heavier processing demands.

Disk subsystem tips

The following paragraphs list tips for your Mac OS system's disk subsystem.

For best performance, the hard disk partition where you run MacBench should contain as much contiguous free disk space as possible. Defragment the hard disk before you begin testing. Increase or decrease the cache size as needed to fine-tune your system's performance. First, adjust the disk cache setting in the Memory control panel so your system uses less RAM for the cache, giving the system more memory of its own to use. (This can boost MacBench r