RecieveBufferSize is not the same as the number of bytes in the buffer itself: it's likely to be considerably bigger.
So when you try to read RecieveBufferSize into your array:
Dim inStream(10024) As Byte
buffSize = clientSocket.ReceiveBufferSize
serverStream.Read(inStream, 0, buffSize)
It looks at the siize of the buffer and the number of bytes you asked it to read and throws an error because one will not fit into the other.
Use the size of your array, and it should go away.