Hi everyone!During the last month I've been testing on at-spi2-atk to determine whether this commit (http://git.gnome.org/browse/at-spi2-atk/commit/?id=d0f7dd49eebedc8c3993a116411f5a8320965968) had marked a change in performance and how much.
Below I explain a bit what I have done: - I have installed jhbuild and compiled all the necessary packages.- I have created 2 branches to make tests on them. One just before the commit and another one just after. - I have researched what tools could be used to measure performance and among all that I have found I'm left with sysprof (to measure the cpu time) and valgrind (to measure memory consumption). - I have researched what applications would be the best to use in tests. Browsing the related bugs (http://git.gnome.org/browse/at-spi2-atk/commit/?id=d0f7dd49eebedc8c3993a116411f5a8320965968, https://bugzilla.gnome.org/show_bug.cgi?id=577098, https://bugzilla.gnome.org/show_bug.cgi?id=419383) and after a first attempt with several apps, I am left with nautilus and rhythmbox. - I have created a few bash scripts to automate and standardize the tests (creation, selection and compilation of the branches, tool selection, etc.). Scripts don't work completely automated and there is an option that I had to comment because it doesn't work fine (I'm sorry, but I'm not a bash expert :( ). But still, they save a lot of time and work.
- I have ran these scripts and saved the results in their directories.- With these results I have produced a report with LibreOffice Calc and several graphs accompanying it (in order to understand it better).
Note: attached to this email is a package with all the work.I would also comment that the results of the tests have been quite positive. It has gained in performance in both CPU time and memory consumption.
Attached to the spreadsheet and graphics, it is another plain text report commenting a bit the spreadsheet data (at first sight it can be a bit difficult to understand them). Even if you want more info than is reflected in the spreadsheet/text, you can open the tests output files with sysprof tool or a text editor and get it. I. e., sysprof output files have info structured by levels; you can click on an app and see the consumption by function and so on.
By the end, to comment that I'm still working on it. Right now I'm doing the same tests on the main branch and on the same but without the changes introduced by the commit, to see if the performance difference is better or worse with the recent changes. Also it is possible incorporate some additional tool. You can see the progress in the scripts (I ship the latest version I'm working on), but still take a while to have the results (tests with these tools are very slow).
That's all for now. If you have any questions, please do not hesitate to ask.
Regards,
Alejo Pacín.
Attachment:
performance_v2_1.tar.gz
Description: GNU Zip compressed data
At-spi2-atk Performance Report
At-spi2-atk performance before/after commit with id d0f7dd49eebedc8c3993a116411f5a8320965968
Note:
Nautilus was ran over /usr/bin (about 3000 items).
Rhythmbox was ran over a big library (about 2000 items).
Sysprof
Nautilus
% CPU Acc. Disable Acc. Enable Acc. Enable + Orca Diff. AE-AD (%) Diff. AEO-AD (%) Diff. AEO-AE (%)
Before 12.58 24.39 26.56 11.81 13.98 2.17
After 12.14 20.18 24.90 8.04 12.76 4.72
Diff -0.44 -4.21 -1.66 -3.77 -1.22 2.55
Rhythmbox
% CPU Acc. Disable Acc. Enable Acc. Enable + Orca Diff. AE-AD (%) Diff. AEO-AD (%) Diff. AEO-AE (%)
Before 2.23 39.83 37.51 37.60 35.28 -2.32
After 2.01 18.36 36.07 16.35 34.06 19.15
Diff -0.22 -21.47 -1.44 -21.25 -1.22 21.47
Valgrind
Nautilus
B Mem Acc. Disable Acc. Enable Acc. Enable + Orca Diff. AE-AD (B) Diff. AE-AD (%) Diff. AEO-AD (B) Diff. AEO-AD (%) Diff. AEO-AE (B) Diff. AEO-AE (%)
Before 89,221,533 221,826,210 238,803,972 132,604,677 148.62 149,582,439 167.65 16,977,762 7.11
After 115,010,219 166,093,059 237,429,630 51,082,840 44.42 122,419,411 106.44 71,336,571 30.05
Diff 25,788,686 -55,733,151 -1,374,342 -81,521,837 -104.21 -27,163,028 -61.21 54,358,809 22.94
Rhythmbox
B Mem Acc. Disable Acc. Enable Acc. Enable + Orca Diff. AE-AD (B) Diff. AE-AD (%) Diff. AEO-AD (B) Diff. AEO-AD (%) Diff. AEO-AE (B) Diff. AEO-AE (%)
Before 131,817,321 358,606,548 364,267,915 226,789,227 172.05 232,450,594 176.34 5,661,367 1.55
After 125,044,945 187,062,126 359,069,303 62,017,181 49.60 234,024,358 187.15 172,007,177 47.90
Diff -6,772,376 -171,544,422 -5,198,612 -164,772,046 -122.45 1,573,764 10.81 166,345,810 46.35
- Explanation:
The previous data are organized by tool/app/commit-time/accessibility.
Sysprof data are expressed in percentage of CPU time.
Rows:
Results before the commit.
Results after the commit.
Differences between the previous rows.
Columns:
Results wit accessibility disable.
Results wit accessibility enable.
Results wit accessibility enable and Orca running.
Differences between the 2nd and the 1st column.
Differences between the 3rd and the 1st column.
Differences between the 3nd and the 2nd column.
Valgrind data are expressed in bytes of consumed memory.
Rows:
The same as sysprof.
Colums:
Results wit accessibility disable.
Results wit accessibility enable.
Results wit accessibility enable and Orca running.
Differences between the 2nd and the 1st column.
The same as the previous column but expressed in percentage (2nd - 1st / 1st).
Differences between the 3rd and the 1st column.
The same as the previous column but expressed in percentage (3rd - 1st / 1st).
Differences between the 3nd and the 2nd column.
The same as the previous column but expressed in percentage (3rd - 2nd / 2nd).