The problem is you are not adding the points themselves to the list but instead you are adding the whole temp list. The same list of points that you clear afterwards. Do not clear the temp list, create a new list instead:
void Form1_MouseUp(object sender, MouseEventArgs e)
{
startDrawing = false;
points.Add(tempPoints);
tempPoints = new List<Point>();
}