Click here to Skip to main content
16,016,475 members
Please Sign up or sign in to vote.
2.00/5 (1 vote)
See more:
Good day all.

The below url has summary data for 2012 12 30
http://howickweather.info/summaries/day.php?d=20121230


I am trying to get the Mean temperature and Windrun using the below.
The problem is I only get

+ ''
+ 'Mean temperature'
+ '' + getVal("meanTemp", 1) + '°C '
+ 'Windrun'
+ '' + getVal("windrun", 1) + ' km '
+ ''
+ ''

not the actual values ( yet they show on the web page )

Any help greatly appreciated - I've spent hours trying to resolve. Thanks Ian.

Imports System.Net
Imports System.IO
Imports System.Xml
Imports System.Text
Imports System.Diagnostics


Public Class Form1
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Call main()
End Sub

Sub main()
Const URLString As String = "http://howickweather.info/summaries/day.php?d=20121230"

Dim SourceRequest As Net.HttpWebRequest = Net.HttpWebRequest.Create(URLString)
Dim SourceResponse As Net.HttpWebResponse = SourceRequest.GetResponse()
Dim SourceStream As New IO.StreamReader(SourceResponse.GetResponseStream)
Dim SourceCode As String = String.Empty

While SourceStream.EndOfStream = False
SourceCode &= vbNewLine & SourceStream.ReadLine
End While

SourceCode = SourceCode.Trim(vbNewLine)
RichTextBox1.Text = SourceCode

End Sub

End Class
Posted
Comments
Sergey Alexandrovich Kryukov 7-Feb-14 19:01pm    
How do you know that the expected value is found in certain line of the code? This is a reliable way, in principle. You need to parse HTML.
—SA

1 solution

Please see my comment to the question.

Using HttpWebRequest you obtain a document. If this is a HTML document, you will need to parse it. It would be good if this is a document well-formed as XML, then you could parse it using one of .NET XML parsers. Not all Web pages are like that, unfortunately, so you may need HTML parser which does not require well-formed XML compliance. Try this one: http://www.majestic12.co.uk/projects/html_parser.php[^].

This is, by the way, is called Web Scraping, http://en.wikipedia.org/wiki/Web_scraping[^].

—SA
 
Share this answer
 
Comments
parkerig 7-Feb-14 22:28pm    
Hi Sergey. Thanks for your reply. I compiled the above but I really am not sure how to load my URL. The full app is designed to loop through 1200 days of data and I create a file with date, Mean Temperature and Windrun.
Sergey Alexandrovich Kryukov 7-Feb-14 22:39pm    
I don't understand: where to load? What, is the problem that HTML is too big? It it's too big to fit in memory (hard to believe though), it can be much harder. One possibility: is this HTML a well-formed XML?
—SA
parkerig 8-Feb-14 18:49pm    
Hi Sergey. As I am not a full time programmer and when I do program it is in vb.net and usually trivial stuff versus full blown applications, I need a bit more support. Firstly, can the html_parser actually get the values I require ( as described above) If so, how please. I have tried several ways to no avail. I also don't see how I can use the parser on a URL. Only a saved document.
http://howickweather.info/summaries/day.php?d=20121230 If you save the page you don't get the values etc. I see this web scrap as a very interesting exercise as it could be used for numerous ideas. Thanks again. Ian.
Sergey Alexandrovich Kryukov 8-Feb-14 19:08pm    
I already told you, if the big file is a problem, this is not going to be so easy. You did not answer to the question which would give a lot of hope: is HTML a well-formed XML or not?
Yes, you can temporary save the file on disk, why not?
—SA
Sergey Alexandrovich Kryukov 8-Feb-14 19:13pm    
Okay, it this is your file sample...
No, it's not well-formed XML, but, from the other hand, it is not a big file... Try to use the HTML parser...
—SA

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900