The problem occurs because you do not add a closing quote to the field that begins with a double quote. So when StringBuilder converts it to a string it tries to escape the floating quote character.
I suspect you are looking at the variables in the debugger, rather than the actual output of your code.
However I suspect your code should be adding quotes after the fields as well as before them like:
if ((valueCounter - 1) > 0)
valueBuilder.Append("\"");
valueBuilder.Append(values[valueCounter - 1].ToString());
if ((valueCounter - 1) > 0)
valueBuilder.Append("\"");