How to read a web page using C#

How does search engine spider crawl web pages ?

On this article, I will try to show How we can achieve similar task using a C# .NET.
The following code will read a web page using a virtual path and get the response then store it locally.

using System.Net;

using System.IO;

WebRequest request = WebRequest.Create(“https://ujjwaladatta.wordpress.com/2011/05/31/membership/”);
using (WebResponse response = request.GetResponse())
{
using (StreamReader responseReader = new  StreamReader(response.GetResponseStream()))
{
string responseData = responseReader.ReadToEnd();
using (StreamWriter writer = new StreamWriter(@”C:\sample.html”))
{
writer.Write(responseData);
}
}
}

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s