The problem is that the computer is little-endian, which means that although the format is ARGB, the order the bytes are stored in memory is B - G - R - A (i.e., B is stored in a lower memory address than G, then R, and then A).
Your code is assuming the opposite ordering of bytes. To get the intended result, you should just invert the assignment order:
if (counter % 4 == 0)
rgbValues[counter] = color.B;
if (counter % 4 == 1)
rgbValues[counter] = color.G;
if (counter % 4 == 2)
rgbValues[counter] = color.R;
if (counter % 4 == 3)
{
rgbValues[counter] = color.A;
list_ct++;
}
This should solve the problem.