Well let's start with the most obvious problem, you opened it in a
text editor. The text editor assumes anything you want opened in it is text, and is therefore showing you the equivalent text of the executable.
EVERY file on your computer is in 1's and 0's. All the time. It's how computers work. But those 1's and 0's are meaningless on their own, they need to be interpreted by something. Your text editor probably assumes the data is in ASCII format, and tries to display equivalent ASCII characters (e.g. 01000001 is agreed upon in the ASCII standard to mean 'A', so upon encountering that byte 'A' is displayed). Even in a text editor, the character 0 is represented by a whole byte (most commonly, it may be bigger), not just a single 0 in binary format, the same goes for 1. (For example in ASCII 0 is 00110000, and 00000000 is interpreted as a null character. Not at all the same.)
You'd need a special editor to do what you want (and even then, hexadecimal representation is favored over binary, for reasons I won't go into here).
Next, an .exe isn't a simple list of machine code instructions. There are sections for data, and other things (you can generally open an executable in something like 7zip and see some of the internal parts). Even exe's from .Net are in binary, just not machine code.
As for why the file is visible...well how else would you distribute it? How would I give you an .exe if I couldn't interact with the file? What if I need to move it, or delete it? And just giving a name to redirect to it...well on some level the OS does this with every file already (you'd have to read about file systems, it's too complicated to go into here).
I'm not sure what your goal is with this, but I'm pretty sure you're going about it the wrong way. I recommend you read
Code[
^], it's very accessible even to complete beginners, and will give you a very good sense of how it all works.