Testing different Firefox channels

      Comments Off on Testing different Firefox channels

This time I have decided to do a comparison test between the versions of Firefox. I took the latest stable release (FF 6.0) and compared it to all the releases in other channels. That includes the following browsers:

  • Firefox 6.0 final
  • Firefox 7.0 beta
  • Aurora 8.0a2 (2011-08-22)
  • Nightly 9.0a1 (2011-08-23)

I have tested all the browser with the exact copy of the same profile with same add-ons and the same opened tabs and loaded pages. I have 44 add-ons in each profile and at the time of testing I had 2 Panorama groups, 7 pinned tabs and 61 opened tabs. However, because I use BarTab Lite add-on, only three three were loaded when testing: Gmail, Kraken Test and SunSpider Test. I have run the tests one after another and not at the same time. Besides the test results I have also noted the memory footprint of a firefox.exe process reported by Windows Task Manager.

Here are the results:

Firefox 6.0 final
Memory use before the tests: 221.620 K
Memory use after Kraken: 250.852 K
Memory use after Kraken and Sunspider: 274.536 K
Max memory use when testing: 532.264 K
Kraken results: 10485.7ms +/- 0.5%
Sunspider results: 386.3ms +/- 6.3%

Firefox 7.0 beta
Memory use before the tests: 227.088 K
Memory use after Kraken: 245.352 K
Memory use after Kraken and Sunspider: 286.920 K
Max memory use when testing: 535.116 K
Kraken results: 10548.9ms +/- 1.3%
Sunspider results: 359.2ms +/- 4.0%

Aurora 8.0a2 (2011-08-22)
Memory use before the tests: 242.020 K
Memory use after Kraken: 281.604 K
Memory use after Kraken and Sunspider: 297.432 K
Max memory use when testing: 551.820 K
Kraken results: 10592.8ms +/- 1.7%
Sunspider results: 374.5ms +/- 4.9%

Nightly 9.0a1 (2011-08-23)
Memory use before the tests: 250.188 K
Memory use after Kraken: 277.288 k
Memory use after Kraken and Sunspider: 291.076 k
Max memory use when testing: 553.500 K
Kraken results: 10762.3ms +/- 2.2%
Sunspider results: 379.9ms +/- 7.3%

I didn’t use the browsers for other tasks when running tests. Fore memory use data I waited about 20 seconds for the numbers to become stable.
The reason for the test was the main feature of FF7 – better memory handling. My overall impression on using Beta and Aurora for everyday tasks have shown me that FF beta 7.0 uses much less memory when confronted with many opened tabs and feeels to respond better than FF6 final.

These results, however, don’t directly show that. In fact, the results don’t show any improvements either at memory handling or speed. However, the feel and responsivness when several tabs are open are more important to me.