Here's something that should do the trick in a single query:
SELECT Upper(SubString(csUriQuery, 1, 1)) as Letter, COUNT(*) as Number
FROM WebLogs
WHERE ASCII(Upper(SubString(csUriQuery, 1, 1))) >= 65 AND ASCII(Upper(SubString(csUriQuery, 1, 1))) <= 90
GROUP BY Upper(SubString(csUriQuery, 1, 1))
ORDER BY Letter
If you're not worried about filtering out non-letter characters (i.e. 1, $, etc.) you should probably remove the WHERE clause.
You also don't really need the count in the select, except for verification.
Then, you could check if the letter is in the results, and you'll only hit the DB once.
If you would rather return a single string of characters that "exist" in the DB, You could probably modify it to return a single concatenated string of the letters, and use a Contains function in your code.
But this should get you where you want to go.
Happy coding.