Html Agility Pack
This is the ultimate reference of reading web pages.
IF you want to do it yourself , you can try with WebRequest , http://msdn.microsoft.com/en-us/library/system.net.webrequest.aspx . But not all HTML is an XML – so you must find a method to parse. But am I the only one ?No – so I found the HTML Agility Pack , http://www.codeplex.com/htmlagilitypack , that knows how to transform HTML in XML. The code is easy :
HtmlWeb hw = new HtmlWeb();
hw.AutoDetectEncoding = true;
HtmlDocument doc = hw.Load(Url);
HtmlNode NodeRoot = doc.DocumentNode;
And from NodeRoot you can start XPATH with SelectNodes . Try it – it is awesome!