.NET (Core and Framework)

Luc Pattyn24-Aug-07 12:57

24-Aug-07 12:57

Hi,

there are good reasons why the gc is lazy, and should not be called explicitly:

1. it is an expensive operation, since it needs to check all the data in your app, figuring
out which objects still have references to them (are "alive") and which don't. There can
be a lot of data involved, it does not fit in the cache, and it trashes the cache
(i.e. whatever was rightfully in there now is gone).

2. while doing its job, the gc suspends all your threads, to make sure the stacks
and objects don't change while it is examining them; so your app is temporarily dead.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

this weeks tips:
- make Visual display line numbers: Tools/Options/TextEditor/...
- show exceptions with ToString() to see all information
- before you ask a question here, search CodeProject, then Google

Dave Kreskowiak24-Aug-07 6:17

Dave Kreskowiak

24-Aug-07 6:17

Xpnctoc wrote:
Using Task Manager to monitor my memory use,

Stop right there. You're looking in the wrong place. Managed code (.NET CLR) apps run inside a virtual machine, just like a Java app. What Task Manager is showing you is the RAM reserved for the entire virtual machine your app is running in, not just your app. So, even though your app isn't using that memory, it's still being held by the virtual machine for future use. That's why the memory doesn't get freed.

Xpnctoc wrote:
Is my Garbage Collector broken?

Nope. It's just being lazy. It will hang on to that memory, so long as Windows doesn't want it back. When Windows starts to get low on memory, it'll ask the virtual machine to release whatever it can back to Windows.

Xpnctoc wrote:
Should I somehow manually invoke garbage collection to clear some memory?

This won't solve the "problem". This will tell the GC to free up any managed memory that hasn't already been freed. But, this memory goes back into the Managed Heap (reserved for future use), NOT back to Windows.

Xpnctoc wrote:
Does anyone know if the GC works on a time interval or a certain percentage of consumed memory, or what?

The GC is extremely good at it's job and runs whenever it detects the need to. There is no set schedule and it tunes it's own performance based on how it sees your app running and what it expects your app to do in the future.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007

Xpnctoc24-Aug-07 8:59

24-Aug-07 8:59

Thanks for the insight, Dave. I have never heard that .NET was actually running on a virtual machine. I knew there was JIT and such involved, but that puts things into a little different perspective.

And like I told Luc, I think I'm coming to the realization that I'm just paranoid because of the extreme data memory requirement of the app I'm writing. But I'd even take it one step farther and suggest to myself that I'm also only working in the theoretical, because it would take a long time using the app to develop the need for so much memory.

By then we'll probably be talking terabytes of memory on standard computer models anyway Smile | :)

Luc Pattyn25-Aug-07 0:04

25-Aug-07 0:04

You may want to display (and periodically update) the value of Environment.WorkingSet
inside your app; that allows you to see memory usage by your process.

Luc Pattyn [Forum Guidelines] [My Articles]

Xpnctoc25-Aug-07 4:14

25-Aug-07 4:14

Thanks for the tip. I wasn't aware of that one. I'm sure that will help.

#realJSOP25-Aug-07 3:13

#realJSOP

25-Aug-07 3:13

A side note - don't allocate more than about 85k of memory to any single object unless you're prepare to handle cleanup manually. Anything bigger than that gets shoved onto the "large object heap", and the GC doesn't clean that up for you.

"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001

Xpnctoc25-Aug-07 4:14

25-Aug-07 4:14

Got that covered. All of my core logic I'm writing in unmanaged code. I found early on that I didn't like the overhead of some of the System.Collections.Generic classes -- especially the bulky disk usage in serialization compared to doing a straight binary write of unmanaged data structures. So I chose to write the app in C++/CLI, using .NET for the ease of creating Windows forms and controls while keeping my core logic and processing in native C++. So I'm fully prepared to do my unmanaged allocation and cleanup. Like I said, the behavior I'm calling into question occurs with things I have little or no control over, such as Menu controls and dialogs.

Luc Pattyn25-Aug-07 8:12

25-Aug-07 8:12

Hi John,

I don't fully agree, AFAIK objects on the large object heap are managed like all other
managed objects, with a single difference: the LOH does not get reshuffled, hence
there is a risk of fragmentation.

Example: allocate 1MB, then 100MB, repeat this N times until memory is almost full,
then throw away the references to all 100MB objects and try to get one really big
object of 300MB.

IMO the threshold (85KB?) is not documented, is not available through some API,
may vary over time, and may depend on your system's memory situation.
I feel the Framework should be more specific about this, it should guarantee a
reasonable minimum value for the threshold, so one can make sure fragmentation will
not become an issue by keeping objects smaller than that.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

Scott Dorman26-Aug-07 3:09

Scott Dorman

26-Aug-07 3:09

Luc Pattyn wrote:
the threshold (85KB?) is not documented, is not available through some API,
may vary over time, and may depend on your system's memory situation.

In general this is true, the 85KB limit isn't publicly documented. I did find it in Rotor[^] (SSCLI) in the gc.h header file for the garbage collector's native code:

/* misc defines */
#define LARGE_OBJECT_SIZE   85000

Scott.

—In just two days, tomorrow will be yesterday.

[Forum Guidelines] [Articles] [Blog]

Vaibhav Sharma26-Aug-07 16:54

Vaibhav Sharma

26-Aug-07 16:54

Hi,

I am amazed that after reading all the replies and posts on this article no one mentions of the Using...End Using code block.

I am sure you must be knowing it but just a little bit of detailing on the using statements:

'VB Version, C# no different, just lang. changes<br />
Using dlgOpen As New OpenFileDialog()<br />
    dlgOpen.Filter="*.*"    <br />
    'Any other properties<br />
    dlg.ShowDialog()<br />
End Using

Now what the above code fragment does is that the CLR will create the dlgOpen object and will be responsible for its disposal, not YOU.
This has serious benefits in terms of Performance & Memory Utilization since as soon as the code execution reaches End Using the dlgOpen will be Disposed off without requiring you to do any thing at all.

:::One Catch Though While using the Using statements:::
the dlgOpen will not be accessible outside the Using block since its scope is limited is to the Using...End Using block.

So i strongly recommend you to go ahead and use the Using...End Using blocks for all the DialogBoxes since most of them times they return only one value which can easily be stored on a local/global variable.

Awaiting reply eagerly.

Regards,
Vaibhav Sharma

Xpnctoc26-Aug-07 17:24

Embedded resources -- pros & cons

26-Aug-07 17:24

I appreciate the tip. I am aware of the "using" syntax, but in my case that won't work. As I mentioned in my response to John Simmons's post above, my application is written in C++/CLI, which has no such construct. There were multiple reasons for that language choice, some of which I mentioned and others that are irrelevant to this conversation. And even if I was in C#/VB, I'd still have the same problem with the Menu control. Can't use "using" for something like a menu that has to be present for the duration of execution.

Regardless, I think it was one of Luc Pattyn's responses that directed me to try examining memory usage through the Environment.WorkingSet (thank's again, Luc). This provides memory use much closer to what I would expect based on my interaction with the application. I still find it alarming that the .NET virtual machine continues to eat up additional memory when it's not needed, but there I just have to trust that the GC will do its job when push comes to shove.

Xpnctoc24-Aug-07 4:16

Re: Embedded resources -- pros & cons

24-Aug-07 4:16

Hello. I hope these are some simple questions:

1. Is there a way to read an embedded resource out of a managed DLL without having to load the whole assembly?

2. If, as is my understaning, the answer to #1 is "NO", what is the point of embedding resources instead of loading your resources from individual files at run-time? In other words, if I have a DLL that contains 128MB of images, and I only want to extract a 64KB image from it, what is the point of embedding when I'd have to take the 128MB hit on my memory use -- especially when an assembly can't be unloaded?

Thanks for your input.

Luc Pattyn24-Aug-07 4:54

Re: Embedded resources -- pros & cons

24-Aug-07 4:54

Hi,

AFAIK:
1. No
2a. You can make a single-file app that holds images, fonts, whatever.
2b. You can protect your data by putting it in as a resource (possibly using encryption).
2c. You can group things that belong together into a single DLL, have several of those
and load the one you need (e.g. language selection, regionalization)
2d. If you are that concerned about memory footprint, make smaller DLLs; at one resource
per DLL, it does not make much sense any more.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

Xpnctoc24-Aug-07 8:43

Re: Embedded resources -- pros & cons

24-Aug-07 8:43

Good point. I had actually thought of the resource grouping as opposed to having a bundle of extra files. And I'm even OK with having to load the entire DLL to get a resource. It's the inability to unload the DLL that I don't like.

Maybe I'm still clinging to the old-school mentality of maximum memory conservation. I'm only 30 yrs old, but that still puts me at old enough to have had it beaten into my head in school about programming for minimal memory use and maximum efficiency.

Regardless, if I have data resources that need to be copied into memory for runtime manipulation, and the DLL can't be unloaded, well then I have two full copies of those resources resident in memory for the duration of execution. That seems wasteful, and that's what I can't stand about it.

Luc Pattyn24-Aug-07 12:48