You can use two javascript's functions.
One in your popup:
function MyPopFunction(user,domain){<br />
self.parent.DoSomething(user,domain);<br />
}<br />
Add to your submit button :
OnClientClick="javascript:MyPopFunction();<br />
And in your page:
function DoSomething(user,domain)<br />
{<br />
<br />
$("#<%=txtUser.clientid %>").val(user);<br />
$("#<%=txtDomain.clientid %>").val(domain);<br />
$find("<%=ModalPopupExtender1.clientid%>").hide(); <br />
}<br />