What you're showing is not a confirmation from the code behind, but javascript. You're just writing it on the server, but it executes on the client.
Make a function that will be called on client button click (there is a property on Aspx Button control OnClientClick. In that function write something like:
return confirm("Are you sure you want to delete?");
That way, Yes will return true and you'll get code behind event. No will return false and the server trip (and event will not happen).
The advantage of this is that you can have the function written on the page or in a file somewhere and you don't need to register the script, just to set ClientClick property to "return functionName();"