This code will edit your app.config:
var cfg = ConfigurationManager.OpenExeConfiguration(Assembly.GetExecutingAssembly().Location);
cfg.AppSettings.Settings.Add("test1", "test2");
cfg.Save();
Make sure you check tha correct config file when you look for changes. Not the one in your project, but the one in your bin folder ;)