EditAttachWiki Help
r122 - 06 Dec 2007 - 16:02:12 - BryanStearnsYou are here: OSAF >  Projects Web  >  DevelopmentHome > PerformanceProject

Performance Project

BusyDevelopersGuideToChandlerPerformanceOptimization

How to run performance tests

Performace Evaluation Session

Primary 0.7 Use Cases

Use Case Target Links Notes
#8 New event (double click) with 3k ideal: < 0.1 s
acceptable: 1 s 		Bug#4119
Stamp with 3k ideal: < 0.1 s
acceptable: 1 s 		Bug:4589 Done 4/11
Jump calendar by one week with 3k 2nd time ideal: < 0.1 s
acceptable: 1 s 		Bug:4637
Overlay calendar with 3k ideal: < 0.1 s
acceptable: 3 s 		Bug:4714
Overlay calendar with 3k 2nd time ideal: < 0.1 s
acceptable: 1 s 		Bug:4714
Publish calendar with 3k ideal: < 1 s
acceptable: 3 s 		Bug:6024
Subscribe to calendar with 3k ideal: < 1 s
acceptable: 3 s 		Bug:6024
Triage in Dashboard with 3k ideal: < 0.1 s
acceptable: 1 s 		Bug:8785

Instructions on how to read Tinderbox performance pages.

NOTE: The target times are doubled on PPC Mac.

Automated test results

Primary 0.6 Use Cases

These are the top 9 use cases we will be tracking for 0.6. Targets are based on user research and measurements of other calendars' import/export features. See below for target hardware.

Use Case Target Windows
8359 		Mac
8359 		Linux
8359 		Links
#1: App startup (with existing, "empty" repository) ideal: < 5 s
acceptable: 10 s 		7.20s 10.8s 4.82s Bug#1849
#2: New event creation (file menu) ideal: < 0.1 s
acceptable: 1 s 		0.390s 0.700s 0.510s Bug#3986
#3: New event creation (in-place) ideal: < 0.1 s
acceptable: 1 s 		0.310s 0.610s 0.280s Bug#3987
#4: Creating a new calendar ideal: < 0.1 s
acceptable: 1 s 		0.120s 0.180s 0.350s Bug#3989
#5: Importing a 3000 event calendar ideal: < 10 s
acceptable: 30s 		58.6s 134s 66.8s Bug#3988
#6: App startup (with existing, 3000 event repository) ideal: < 5 s
acceptable: 10 s 		9.76s 16.0s 7.14s Bug#4117
#7: New event creation (file menu), 3000 ideal: < 0.1 s
acceptable: 1 s 		1.84s 4.06s 1.62s Bug#4118
#8: New event creation (in-place), 3000 ideal: < 0.1 s
acceptable: 1 s 		1.96s 4.22s 1.76s Bug#4119
#9: Creating a new calendar, 3000 ideal: < 0.1 s
acceptable: 1 s 		0.110s 0.080s 0.060s Bug#4120

Secondary use cases

Use Case Target Windows
8359 		Mac
8359 		Linux
8359 		Links
Switch view ideal: < 0.1 s
acceptable: 1 s 		0.500s 0.720s 0.750s Bug:1850
Stamping ideal: < 0.1 s
acceptable: 1 s 		0.440s 0.760s 0.540s Bug:3985
Switch View, 3000 ideal: < 0.1 s
acceptable: 1 s 		1.70s 2.58s 2.20s Bug:4144
Stamping, 3000 ideal: < 0.1 s
acceptable: 1 s 		0.980s 1.66s 1.36s Bug:4589
Scrolling calendar, 3000 acceptable: < 0.1 s 0.030s 0.090s 0.020s Bug:4215
Scrolling table, 3000 acceptable: < 0.1 s 0.190s 0.240s 0.100s Bug:4453
Jumping to the next week with 3k calendar ideal: < 0.1 s
acceptable: 1 s 		1.30s 2.58s 1.40s Bug:4637
Overlay calendar with 3k event calendar ideal: < 0.1 s
acceptable: 1 s 		4.40s 8.60s 3.24s Bug:4714
Resizing app with 3k calendar acceptable < 0.1 s       Bug:4189

See RuntimePerformance for more details.

Target Hardware

Our ballpark is the mid-range setup that a consumer would have bought in 2003.

  • Windows/Linux: Pentium(R) 4, ~2 GHz, 512 RAM
  • Mac: G4, ~1GHz, 512 RAM
  • Intel iMac, the first Intel processor based Macs available. The two default configurations are 512 MB RAM and 1 GB RAM - we've opted to go with 1 GB for now because we bought it like that. This does mean comparing the speeds is biased if we get memory issues.

The 4 machines we have decided to use, along with configuration details are listed below. We realize that these are a bit faster than our original target but feel they are close enough to meet our needs.

Machine OS Processor CPU Memory
Windows Win XP Professional Edition SP2 Pentium(R) 4 Single CPU, 2.8 GHz 512 MB RAM
PPC Mac Mini Mac OSX 10.4 Tiger G4 Single CPU, 1.25 GHz 512 MB DDR SDRAM
Linux Ubuntu Dapper Drake Pentium(R) 4 Single CPU, 2.8 GHz 512 MB RAM
Intel iMac Mac OSX 10.4 Tiger Intel Core Duo 1.83 GHz Intel Core Duo 1 GB 667MHz DDR2 SDRAM

All measurements should be done using release version of Chandler (optimized).

Memory Footprint

Mozilla Suite under semi-heavy usage with several open windows, tabs, mail, Chatzilla, running for several days can be under 150 MB on Windows. On startup it is under 20 MB. Firefox is about 15 MB.

Procedure: launch Chandler with --create, quit, relaunch without options and measure.

Platform Ideal Acceptable r11484
Windows <20 <50 61.2
PPC Mac OS X <20 <50 ???
Intel Mac OS X <20 <50 ???
Linux <20 <50 73 (resident)

Bugs: See bugs with footprint keyword.

The latest memory performance data can be found at:

Download Footprint

Platform Ideal Acceptable 0.5.01 0.5.03 0.7alpha2 0.7alpha4.dev-r11465 0.7.dev-r14619
Windows <10 <20 13.5 20.6 17.6 19.1 18.6
PPC Mac OS X <10 <20 30.2 30.2 52.1 41.6 44.0
Intel Mac OS X <10 <20       32.4 33.3
Linux <10 <20 25.9 21.0 23.4 25.4 26.6

See DiskFootprint for more details.

In 0.5.03 we added PyICU.

Meeting Notes

The weekly performance meeting occurs on Monday's at 11:00am, attended by Katie, Sheila and Aparna.

Edit | WYSIWYG | Attach | Printable | Raw View | Backlinks: Web, All Webs | History: r122 < r121 < r120 < r119 < r118 | More topic actions
 
Open Source Applications Foundation
Except where otherwise noted, this site and its content are licensed by OSAF under an Creative Commons License, Attribution Only 3.0.
See list of page contributors for attributions.