Re: GObject memory statistics

[Tommi Komulainen <tommi komulainen nokia com>]

On Tue, 2006-07-04 at 03:37 -0400, Matthias Clasen wrote:
> Ben Maurer has recently asked where the heap 
> goes that GTK+ apps use. Here is a very quick
> patch to add some statistics to GObject. 

Hi,

Please find attached an older study we made on a minimum "average" maemo
application using the IT-2005 software. The application uses widgets and
functions common to most maemo applications, most notably:
      * HildonApp+HildonAppView
      * GtkMenu
      * GtkToolbar
      * HildonFindToolbar
      * HildonFileChooser
      * libosso (DBus)
      * GConfClient

The results should apply to stock gtk+ as well, the hildon widgets
aren't that memory hungry.

The memory consumption is calculated based on what was allocated but not
freed during widget creation, usage and showing.

In addition to theme and fonts related allocations, iconv was
surprisingly heavy. We got rid of iconv in many places in IT-2006,
though we didn't do similar study afterwards.

HTH.

-- 
Tommi Komulainen                            <tommi komulainen nokia com>

Where the memory goes in Gtk/Hildon application
===============================================

This is memory usage study for N770 IT-2005 application framework.

Here are results of how much memory is used by functions application
is calling, and where the memory actually goes (estimated allocation
overhead (8 bytes/alloc) is indicated with '+<value>'):

0. gtk_init():  163KB +19KB
  + gdk_display_open():
    - XOpenDisplay():  18KB
      (this is the X library socket buffer I think)
    - gdk_window_new()
      -> gdk_window_set_title()
      -> gdk_utf8_to_string_target()
      -> iconv_open():  108KB +14KB
         - pthread_once():  75KB +14KB  (~2000 separate allocations)
  + g_option_context_parse():  22KB +2KB
    - g_type_init_with_debug_flags():  12KB +1KB

1. osso_initialize():  24KB
  + dbus_connection_send_with_reply_and_block():  18KB
    - dbus_realloc():  18KB

2. Gconf initialization and setting/getting the *first* Gconf key:  12KB
  + gconf_engine_set():  12KB
    - dbus_connection_send_with_reply_and_block():  11KB
      - dbus_realloc():  18KB

3. Create *first* Gtk widget (hildon app+view): 227KB +38KB
  + gtk_settings_get_default():                182KB +30KB
    - gtk_rc_reparse_all_for_settings():       175KB +29KB (thousands objects)
      - sapwood: theme_create_rc_style():             28KB
        (this can overlap with functions below)
      - gtk_rc_style_new():                           24KB
        (this can overlap with functions below)
      - sapwood: theme_pixbuf_set_filename():         22KB +4KB (>500 objects)
      - sapwood: theme_pixbuf_new():                  10KB +4KB (>500 objects)
      - g_array_insert_vals():                        14KB
      - gtk_theme_engine_get -> g_type_module_use():   4KB
      - g_hash_table_replace():                        4KB
      - gtk_rc_find_pixmap_in_path():                  3KB

4. Create *first* menu for appview:  40+6KB
  The menu had 5 items of 4 different types

5. Create *first* normal and find toolbar: 86KB + 14KB
  + gtk_combobox_entry_new_text():          35KB +8KB
  + gtk_toolbar_new():                      13KB +3KB
  + hildon_find_toolbar_new():              11KB +2KB
  + gtk_tool_button_new_from_stock():       10KB +1KB
  + gtk_box_pack_end -> gtk_rc_get_style():  4KB
    (some of size is explained by reallocs)
  + gtk_separator_tool_item_new():           1KB

Note that re-creating menu and toolbars, which on first time took 146KB,
took on second time only 27KB.  The memory usage division was then
following:
  + gtk_widget_show_all(): 10KB
  + hildon_find_toolbar_new(): 4KB
    (contains combobox and a few buttons)
  + gtk_combo_box_entry_new_text(): 3KB
  + gtk_tool_button_new_from_stock(): 2KB
  + gtk_menu_new(): 1KB
  + gtk_menu_item_new_with_label(): 1KB
  + gtk_radio_menu_item_new_with_label(): 1KB
  + gtk_toolbar_new(): 1KB
  + gtk_toolbar_insert(): 1KB
  + gtk_box_pack_end(): 1KB

6. Show the widgets:  274KB +32KB
  + pango_layout_get_extents():  162KB +16KB
    - pango_shape -> _pango_engine_shape_shape(): 96KB +2KB
      - pango_font_get_glyph_extents ->
        XftFontLoadGlyphs -> FT_Load_Glyph():        10KB
      - pango_fc_font_lock_face -> XftOpenPattern(): 84KB +1KB
        - XftFontOpenIfo - FT_Open_Face():      42KB
	- XftInitLibrary -> FT_Init_Freetype(): 24KB
	  - FT_New_Library(): 16KB
	  - FT_Add_Module():   8KB
    - pango_itemize_with_base_dir():              68KB +14KB
      - pango_font_map_load_fontset():              54KB +12KB
        - FcConfigSubstitute -> FcInitLoadConfigAndFonts():     30KB +8KB
	  - FcConfigBuildFonts -> FcDirCacheReadDir():  20KB +2KB
	  - FcInitLoadConfig -> XML_ParseBuffer():      9KB +5KB
	- FcFontFontRenderPrepare -> FcPatternAddWithBinding(): 10KB +3KB
        - XftDefaultSubstitute -> XftDisplayInfoGet():          12KB
	  - XGetDefault -> _XrmInternalStringToQuark(): 10KB
  + gtk_widget_render_icon():  39KB +10KB
    - gtk_style_lookup_icon_set ->
    - gtk_icon_set_render_icon():               13KB +2KB
      - gtk_icon_theme_load_icon -> gtk_icon_theme_lookup_icon():  8KB +1KB
      - gtk_icon_theme_get_for_screen -> gtk_icon_theme_new():     5KB +1KB
      _gtk_icon_factory_ensure_default_icons(): 26KB +8KB
      - gdk_pixbuf_new_from_inline -> gdk_pixbuf_new_from_data(): 13KB +1KB
  + gtk_gc_get(): 7KB
    (could be because of memchunk realloc)
  + gdk_window_register_dnd(): 4KB
  + gdk_window_show -> gdk_synthetize_window_state ->
    gdk_display_put_event -> gdk_event_copy -> gdk_event_new(): 4KB
  + gtk_menu_shell_append(): 3KB

Note: there's also 96KB Gdk Scratchbuffer to transfer image data
to X server through shared memory.

7. Entering Gtk main loop and exiting it on timeout:  24KB +1KB
  + pango_layout_get_cursor_pos -> -> pango_font_get_metrics: 5KB
    - pango_shape -> -> FT_Load_Glyph(): 4KB
  + XInternAtom -> _XReply -> XEnq(): 5KB
    (maybe the reply buffer)
  + sapwood_pixbuf_get_geometry -> g_cache_insert ->
    sapwood_pixmap_get_for_file: 4KB
    - gdk_pixmap_foreign_new -> gdk_pixmap_foreign_new_for_display(): 3KB
  + gdk_draw_layout -> gdk_draw_layout_with_colors(): 2KB
    - gdk_pango_renderer_get_default -> gdk_pango_renderer_new(): 1KB
    - pango_renderer_draw_layout -> -> _gdk_x11_renderer_get(): 1KB
  + gtk_window_group_new(): 2KB
  + gdk_keymap_get_for_display(): 2KB
  + dbus_watch_handle -> dbus_realloc(): 2KB
  + _gtk_entry_get_borders(): 1KB
    (gets a style)

All the values are rounded to nearest KB.

By using g_blow_chunks(), only 4KB could be freed out of the 91KB allocated
with Glib MemChunks.


Some conclusions
----------------

I think we should really look what iconv() and _pthread_once() do,
or get completely rid of iconv().  Currently it's needed for:
- Converting window titles from utf8 to latin1
- Converting png comment fields (containing hotspot etc info)
  from latin1 to utf8
- Doing filesystem file name conversions from/to latin1


Update: In IT-2006 software the need for iconv() is removed.