Try regular expression.
I think a function like this would work to give you a true/false if the value in the column is valid
It will only allows numbers.
private static bool IsNumeric(string cellData)
{
System.Text.RegularExpressions.Regex rx = new System.Text.RegularExpressions.Regex("^[0-9]*");
return rx.IsMatch(cellData);
}