One aspect is extremely important:
Application Domain is useful when you need to unload loaded assembly. This is not allowed by .NET due to safety purpose: what is some code will be called after unloading?
If you try to create re-loadable plug-ins, it will create a string memory leak. If you want to load a different version of the plug-in during run-time, old version will stay in memory. The only way is this: load (or compile into memory which effectively performs compile-and-load) a reloadable assembly in a separate Application Domain and used it through this domain. As the the domains are memory-isolated as strongly as processes, there is no any danger in unloading the domains, so there is an unload method of the Application Domain, with all the assemblies loaded in it.
Please see my past solution describing skeletons of architecture using this feature:
Create WPF Application that uses Reloadable Plugins...[
^],
AppDomain refuses to load an assembly[
^],
code generating using CodeDom[
^].
—SA