Leak And Bloat Tests

This page describes how to perform tests that measure memory leaks and bloat for MailNews and its sub-components.

Aim

• To provide a continuous check within MailNews and its sub-components for the following items:
  1. Total memory leaks.
  2. Total memory usage (aka bloat).
• Provide a consistent number from build to build where no source code has changed.
• Provide reasonable coverage for the normal activities that a user performs.

Method

Current Method:

• Measure Leaks and Bloats, in a similar way to Firefox (using XPCOM_MEM_LEAK_LOG and --trace-malloc).
• The method of testing memory bloat and leaks will be the same sequence of events, using the same set of test data.
  1. Startup Main Mail Window
  2. Open Address Book and Message Composition windows
  3. Close Address Book and Message Composition windows
  4. Quit the application

Future improvements will be discussed on the discussion page of the Mozilla wiki.

Results

Printed on tinderbox output, these consist of:

• Mail RLk
  • Reference Count Leaks
• Mail Lk
  • Total Bytes malloc'ed and not free'd
• Mail MH
  • Maximum Heap size
• Mail A
  • Allocations - number of calls to malloc and friends.

There are currently no graphs for these results.

Manually Running Tests

Setting up

Build Set up

Build Thunderbird or SeaMonkey with your standard mozconfig file, but with the following options set:

• ac_add_options --enable-debug
• ac_add_options --enable-trace-malloc

Running the Tests

In your <objdir> run the following command:

make mailbloat

This will run the tests and produce some result files. See Debugging Memory Leaks for more information on what to do with this data.

TODO: add more information on processing output.

Code Locations

The files specific to Leak and Bloat testing are be stored in:

http://mxr.mozilla.org/comm-central/source/mailnews/test/performance


These files consist of:

• Overlays (used to provide the hooks for the javascript):
  • bloat/bloatAddrOverlay.xul
  • bloat/bloatComposeOverlay.xul
  • bloat/bloatMainOverlay.xul
• Javascript files (used to drive the tests):
  • bloat/bloatAddrOverlay.js
  • bloat/bloatComposeOverlay.js
  • bloat/bloatMainOverlay.js
• Preference Settings (used to provide a defined profile, see below):
  • common/mailnewsTestPrefs.js
• Python scripts (used to set up the profile and run the test):
  • bloat/setUpBloatTest.py
  • bloat/runtest.py
  • common/setUpCommonMailNews.py

Pre-defined profile

Initial Setup:

• One POP3 account (Mails TBD)
• One Identity
• One SMTP server defined (not used)

Future requirements/possibilities:

• One Address Book where PAB has ~1000 entries
• Large Message folders
• IMAP
• NNTP Server and subscribed newsgroup.

08/04/2008: prefs.js created via TB with the above settings, the first section is for preferences included in tinderbox, the second section is ones which don't currently get set.

user_pref("mail.account.account1.server", "server1");
user_pref("mail.account.account2.identities", "id1");
user_pref("mail.account.account2.server", "server2");
user_pref("mail.accountmanager.accounts", "account1,account2");
user_pref("mail.accountmanager.defaultaccount", "account2");
user_pref("mail.accountmanager.localfoldersserver", "server1");
user_pref("mail.identity.id1.fullName", "Tinderbox");
user_pref("mail.identity.id1.smtpServer", "smtp1");
user_pref("mail.identity.id1.useremail", "tinderbox@invalid.com");
user_pref("mail.identity.id1.valid", true);
user_pref("mail.root.none-rel", "[ProfD]Mail");
user_pref("mail.root.pop3-rel", "[ProfD]Mail");
user_pref("mail.server.server1.directory-rel", "[ProfD]Mail/Local Folders");
user_pref("mail.server.server1.hostname", "Local Folders");
user_pref("mail.server.server1.name", "Local Folders");
user_pref("mail.server.server1.type", "none");
user_pref("mail.server.server1.userName", "nobody");
user_pref("mail.server.server2.check_new_mail", false);
user_pref("mail.server.server2.directory-rel", "[ProfD]Mail/tinderbox");
user_pref("mail.server.server2.download_on_biff", true);
user_pref("mail.server.server2.hostname", "tinderbox");
user_pref("mail.server.server2.login_at_startup", false);
user_pref("mail.server.server2.name", "tinderbox@invalid.com");
user_pref("mail.server.server2.type", "pop3");
user_pref("mail.server.server2.userName", "tinderbox");
user_pref("mail.smtp.defaultserver", "smtp1");
user_pref("mail.smtpserver.smtp1.hostname", "tinderbox");
user_pref("mail.smtpserver.smtp1.username", "tinderbox");
user_pref("mail.smtpservers", "smtp1");
user_pref("mail.startup.enabledMailCheckOnce", true);
user_pref("mailnews.start_page_override.mstone", "1.9pre");
user_pref("mail.shell.checkDefaultClient", false);
// Ensure OS X and Outlook/OE books are disabled
user_pref("ldap_2.servers.osx.position", 0);
user_pref("ldap_2.servers.oe.position", 0);

Preferences in generated profile, but not set:

user_pref("mail.root.none", "/home/moztest/.thunderbird/t7i1txfw.minimum/Mail");
user_pref("mail.root.pop3", "/home/moztest/.thunderbird/t7i1txfw.minimum/Mail");
user_pref("mail.server.server1.directory", "/home/moztest/.thunderbird/t7i1txfw.minimum/Mail/Local Folders");
user_pref("mail.server.server2.directory", "/home/moztest/.thunderbird/t7i1txfw.minimum/Mail/tinderbox");
user_pref("mail.attachment.store.version", 1);
user_pref("mail.folder.views.version", 1);
user_pref("mail.spam.version", 1);
user_pref("mailnews.quotingPrefs.version", 1);
user_pref("mailnews.ui.threadpane.version", 6);

Changes to Leak and Bloat Tests