This is complicated, because if the calculator is part of your application, then it will close when your application does - it's only when you are (for example) opening Windows Calculator as a process (via Process.Start) that it becomes a problem, because the calculator is a completely separate application, and is not related to your app in any way at that point.
You can (as Wes suggests) keep a list of Processes you have started, but frankly as a user I'd probably be rather annoyed if applications I was using closed because a different one did - and if I lost any data as a result, I'd be after your head! :laugh:
But it's simple enough to do:
http://msdn.microsoft.com/en-us/library/system.diagnostics.process.close(v=vs.110).aspx[
^] even gives an example.