How to pick only few values from a variable whose dataType is of Var Type using LINQ (ASP.NET)

Question

0.00/5 (No votes)

See more:

I tried This to get the content from a shopping website..

C#

private void DisplayNewLink(string url)
    {
        HtmlWeb web = new HtmlWeb();
        HtmlDocument doc = web.Load(url);

        var results = doc.DocumentNode
            .Descendants("div")
            .Where(n => n.Attributes["class"] != null && n.Attributes["class"].Value !=null)
            .Select(n => n.InnerText.Replace("\r\n", "").Trim())
            .ToArray();
    }

now the values are inside results, i want only few values to display in front end but not getting any idea.. nedd some hepl to figure out this (if possible)

Thank you......

Posted 5-Aug-13 19:32pm

Anjanee Kumar Singh

Updated 5-Aug-13 19:33pm

v2

Add a Solution

Comments

Maarten Kools 6-Aug-13 4:19am

What do you mean by "few values"? Like maybe only ten results? If so, use the Take[^] method, e.g.

var results = doc.DocumentNode
.Descendants("div")
.Where(n => n.Attributes["class"] != null && n.Attributes["class"].Value !=null)
.Select(n => n.InnerText.Replace("\r\n", "").Trim())
.Take(10) // Take the first 10 results
.ToArray();

Anjanee Kumar Singh 6-Aug-13 4:24am

Few Values Means To pick only needed values...

Maarten Kools 6-Aug-13 4:29am

Just put in an extra where-clause then. I cannot guess what qualifies as "needed values".

Anjanee Kumar Singh 6-Aug-13 4:45am

actualy Sir
I am crawling a shopping website say :www.abc.com
i want to extract all the post like Product name, Price etc. for this situation i written the above code...

Maarten Kools 6-Aug-13 4:56am

You cannot do that for a random site, you need to know what the structure looks like.

So if you crawl through all divs, you can, from there, extract the information.

For example, let's say the HTML looks like this (simplified):
<div class="product">
<div class="title">My Product</div>
<div class="price">$4.99</div>
</div>

You can crawl through like so:
var results = doc.DocumentNode.Descendants("div")
.Where(n => n.Attributes["class"] != null && String.Equals(n.Attributes["class"].Value, "product", StringComparison.InvariantCultureIgnoreCase))
.Select(n => new
{
Title = n.ChildNodes.Single(cn => cn.Attributes["class"] != null && String.Equals(cn.Attributes["class"].Value, "title", StringComparison.InvariantCultureIgnoreCase)).InnerText,
Price = n.ChildNodes.Single(cn => cn.Attributes["class"] != null && String.Equals(cn.Attributes["class"].Value, "price", StringComparison.InvariantCultureIgnoreCase)).InnerText
})
.ToArray();

But if you don't know the structure of the HTML, you can't do it.

Anjanee Kumar Singh 6-Aug-13 4:59am

If this the case then how should i proceed to write a generic code for such senario if possible...

pls.

Maarten Kools 6-Aug-13 5:19am

Editing comments suck, sorry.

You cannot make generic code to parse the site. But you can make a framework which allows you to make parsers for each website you want to support. That way retrieving the data will be generic (but the actual parsing will not be)

public class Product
{
public string Name { get; set; }
public double Price { get; set; }
}

public interface IParser
{
IList<Product> Parse(HtmlDocument document);
string SupportedUrl { get; }
}

class Parser : IParser
{
public IList<Product> Parse(HtmlDocument document)
{
return document.DocumentNode.Descendants("div")
.Where(n => n.Attributes["class"] != null && String.Equals(n.Attributes["class"].Value, "product", StringComparison.InvariantCultureIgnoreCase))
.Select(n => new Product
{
Name = n.ChildNodes.Single(cn => cn.Attributes["class"] != null && String.Equals(cn.Attributes["class"].Value, "title", StringComparison.InvariantCultureIgnoreCase)).InnerText,
Price = Double.Parse(n.ChildNodes.Single(cn => cn.Attributes["class"] != null && String.Equals(cn.Attributes["class"].Value, "price", StringComparison.InvariantCultureIgnoreCase)).InnerText)
})
.ToList();
}

public string SupportedUrl
{
get { return "http://www.abc.com"; }
}
}

public class ProductExtractor
{
private static readonly IList<iparser> Parsers;

static ProductExtractor()
{
Parsers = new List<iparser>
{
new Parser()
};
}

public IList<Product> Parse(string url)
{
IParser parser = Parsers.SingleOrDefault(p => String.Equals(p.SupportedUrl, url, StringComparison.InvariantCultureIgnoreCase));
if (parser == null) throw new NotSupportedException(String.Format("The url {0} is not supported.", url));

HtmlDocument document = new HtmlDocument();
document.Load(url);
return parser.Parse(document);
}
}

ProductExtractor pe = new ProductExtractor();
var results = pe.Parse("http://www.abc.com");

Anjanee Kumar Singh 6-Aug-13 5:44am

Parsers.SingleOrDefault(p => String.Equals(p.SupportedUrl, url, StringComparison.InvariantCultureIgnoreCase));

'System.Collections.IList' does not contain a definition for 'SingleOrDefault' and no extension method 'SingleOrDefault' accepting a first argument of type 'System.Collections.IList' could be found (are you missing a using directive or an assembly reference?

and this line too

Parsers = new List { new Parser() };
using the generic type 'System.Collections.Generic.List<t>' requires 1 type arguments

Maarten Kools 6-Aug-13 6:12am

Oh sorry, it needs to be an IList<iparser>, so:

private static readonly IList<iparser> Parsers;

static ProductExtractor()
{
Parsers = new List<Parser>
{
new Parser()
};
}

That must've disappeared somewhere with a copy/paste into this comment field. SingleOrDefault should work after that.

Anjanee Kumar Singh 6-Aug-13 6:16am

sir sorry but SingleOrDefault still raising same error

Maarten Kools 6-Aug-13 9:12am

Did you include the correct using directive? It's System.Linq

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

M.R.A. · Answer 1 · 2013-08-05T23:50:00

Solution 1

use foreach for this like-------
foreach(var v1 in v)
{
txt1.text=v1.value;
}

Posted 5-Aug-13 23:50pm

M.R.A.

Comments

Anjanee Kumar Singh 6-Aug-13 5:52am

sorry i didn't understand this code...