When you release your program - which is what a setup program is all about, after all - you won't have a debug folder! If noithing else, you should be releasing the Release version which is generally smaller, faster and contains better optimised code and no debug information.
So, no debug folder - which means you would be trying to write to a folder under your application, which will have been installed in the "Program Files" (or "Program Files x86") folder. The trouble with that is that it really isn't a good idea to keep data files of any sort there - if nothing else, changes here are restricted to admin (win 7) or seen as virus activity.
Instead, use
string path = Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData);
and check if it contains a folder for your application. If it does, use it.
If it doesn't, create it and use it.
After that, it is a simple matter to make a setup program - you just have to add a Setup and Deployment Project to your solution and follow the instructions, pretty much.