Changing App.Config on the fly for unit testing

An alternative is to abstract the settings that you intend to use.

So instead of changing your app.config on the fly, simply change a mock or the instance values of the configuration object.

For example:

public interface IApplicationConfiguration
{
    string MaybeAFolderINeed { get; }
    string MyConnectionString { get; }
}

Then you could mock it and return the relevant values.

public class ClassRequiringConfig
{
    public ClassRequiringConfig(IApplicationConfiguration configuration) { ... }
    public ClassRequiringConfig()
           : this(new DefaultApplicationConfigurationImpl()) { ... }
}

Just a thought.