How to read a web page using C#

How does search engine spider crawl web pages ?

On this article, I will try to show How we can achieve similar task using a C# .NET.
The following code will read a web page using a virtual path and get the response then store it locally.

using System.Net;

using System.IO;

WebRequest request = WebRequest.Create(“”);
using (WebResponse response = request.GetResponse())
using (StreamReader responseReader = new  StreamReader(response.GetResponseStream()))
string responseData = responseReader.ReadToEnd();
using (StreamWriter writer = new StreamWriter(@”C:\sample.html”))


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s