Re: Opera Memory Fix

["Kevin Kubasik" <kevin kubasik net>]

On Nov 16, 2007 10:47 AM, D Bera <dbera web gmail com> wrote:
> > There is however a leak at FileSystemQueryable.cs 345, its not the
> > most horrid offender, but it can be one.
>
> You are referring about the comment about the leak there ? about the
> hashtable name_info_by_id ? If that one, then its ok. Its a known
> issue, hard to implement and not really a leak. Is there anything else
> ?
No, thats really it.
>
> > The real issue is the reflection/serialization stuff, some of this
> > might be mono issues, some might be on us, but there are a ton of
> > strings hanging around in the XML serialization class libraries. There
>
> They are probably from the tons of serializer that beagle uses.
> Serializers are expensive to build, so we create one of each (still
> there are many) and keep them for the whole time.

Yeah, those are killers, but we can't lose them. Although
hypothetically our migration to Dbus in the potential future could
help remove some of that, but thats not happening any time soon =/
>
> > are over 8,000 System.MonoType instances that pretty much hang around
> > (apperently they are not collectable, and abundantly used in
>
> Related to the above strings ? Or separate issue ?
>
No, they are a result of all our reflection code. Every call to
GetType(), GetTypes(), or typeof() returns a Type object (nothing new
or exciting) but if we touch that object, it will hang around. A
snippet from a conversation in #monodev

(9:44:59 AM) kkubasik: Also, I've been doing some profiling with
heap-buddy, im finding that there are almost 8000 System.MonoType
objects
(9:45:45 AM) kkubasik: that are hanging around, it's a big codebase
(beagle) but I was wondering if that was sane/normal
(9:45:50 AM) kkubasik: or if were missing something
(9:46:15 AM) jb: afaik they are not collectable
(9:47:10 AM) kkubasik: are there classes/methods we should avoid to
reduce the total number?
(9:47:50 AM) jb: well, everything in Reflection relies on them, so
they are quite unavoidable
(9:48:03 AM) kkubasik: ahhh, we do a ton of reflection :(
(9:48:04 AM) jb: lupus, am I right about the MonoType not being collected?
(9:49:07 AM) lupus: they are collected at appdomain shutdown
(9:49:20 AM) kkubasik: are there known problem spots with the
reflection code/are optimizations planned?
(9:49:30 AM) kkubasik: or is this just the overhead with reflection?
(9:49:42 AM) msieradzki [~marek dynamic-78-8-6-132 ssp dialog net pl]
entered the room.
(9:49:43 AM) jb: yep, that's what I thought
(9:49:51 AM) lupus: we discussed this issue in beagle I think a year
ago: they do GetTypes() all other the place for no good reason
(9:50:02 AM) robert: kkubasik: once you touch a type, it will remain in memory.
(9:50:07 AM) kkubasik: oooo
(9:50:20 AM) kkubasik: I think we trimmed some of the real hotspots on that
(9:50:32 AM) kkubasik: but ill see if there's anywhere else we can clean it out
(9:51:04 AM) lupus: well, 8000 Types means that you're likely loading
all of corlib, gtk-sharp and god knows what else:)
(9:51:24 AM) kkubasik: haha, yeah :)
(9:52:17 AM) lupus: is this in the UI program or the indexer?
(9:52:24 AM) kkubasik: the indexer
(9:52:58 AM) kkubasik: well, we definatly use GetType() in some places
where it can probably be avoided
(9:53:48 AM) kkubasik: another big memory eater is the
System.Xml.Serialization.ClassMap
(9:53:54 AM) kkubasik: but I'm assuming that's gotta be there
(9:54:46 AM) lupus: well, the first thing is to review all the calls
to GetTypes()
(9:55:23 AM) kkubasik: yeah, im going through that now, its just
beagle sits on almost 10,000 strings
(9:55:30 AM) kkubasik: and im trying to ween us of some of them
(9:55:35 AM) kkubasik: ;)
(9:56:11 AM) lupus: hope you don't intern them as well:)


I've been trying to figure out how we can consolidate some of our use
of type stuff (since we often just request new objects via GetType())
we might be able to consolidate some of the Messages stuff or our
Attribute Extraction stuff, but nothing is jumping out as the major
offender. =/

On a somewhat related note, I just build Mono 1.2.6 from subversion
and it is _significantly_ leaner when idling. While we still have the
same 'spike' or 'active indexing' memory usage, when were doing
nothing we sit at like 17 MB RSS which is nice :)

Anyways, I think that any real reductions will require major
restructure/refraction, and aren't fit for this release.

> Thanks for doing the rounds,
> - dBera
>
> --
>
> -----------------------------------------------------
> Debajyoti Bera @ http://dtecht.blogspot.com
> beagle / KDE fan
> Mandriva / Inspiron-1100 user
>



-- 
Cheers,
Kevin Kubasik
http://kubasik.net/blog