Using QueryCellInfo is the proper place, but always calling Double.Parse(cellValue) and trying to catch an exception when it fails is not the quickest way to handle this. Instead use double.TryParse. This will test for a valid value without raising an exception and not raising an exception is generally significantly quicker than rasing one.
double d;
string text = grid[row,col].Text;
if(double.TryParse(text, System.Globalization.NumberStyles.Any, null, out d))
{
//d will have the double value
}