.NET (Core and Framework)

Pete O'Hanlon2-Aug-10 4:18

2-Aug-10 4:18

I'm always amazed that people want to transfer DataTables over the wire. Remember that a DataTable is a structure that is specific to the .NET framework, so a whole lot of plumbing is required in none-MS applications to understand and work with them.

"WPF has many lovers. It's a veritable porn star!" - Josh Smith

As Braveheart once said, "You can take our freedom but you'll never take our Hobnobs!" - Martin Hughes.

devvvy2-Aug-10 15:40

2-Aug-10 15:40

if client app is not to communicate directly with db (not permitted), then what's your soln

dev

Dave Kreskowiak2-Aug-10 18:08

2-Aug-10 18:08

A web service would probably do it, and you're still going to have to cut down on the data being transferred. There's no way you can get away with displaying thousands of records on the screen at once, so why transfer them?!

As someone else has already said, rethink you're design. If you have to aggregate thousands of records, the web service can handle that and return the result. Your solution, for best responsiveness and minimumal network use should lean toward having the service do as much work as possible and transfer only what the user can actually see.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak

devvvy2-Aug-10 19:48

2-Aug-10 19:48

you're saying you can do web service and not via WCF?

rethink what you've learnt. I've done countless screens paging sorting I know like back of my hand so I don't need another lecture of how many rows the grid can or should display.

In short, anyone of you actually has done it, wire a DataTable over the wire (or not)

dev

Dave Kreskowiak3-Aug-10 1:57

3-Aug-10 1:57

Frankly, I don't care what you use. The basic problem still remains. You're trying to transfer data that you shouldn't be trasnfering. There is no reason to send large DataTables over the wire. Send only what you need.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak

devvvy3-Aug-10 13:34

3-Aug-10 13:34

if you dont know let me know

dev

Dave Kreskowiak4-Aug-10 2:11

4-Aug-10 2:11

How many different ways do we have to tell you that sending a DataTable is not a good idea? It's just too big, even if it's empty!

Create your own objects to hold your data plus an IEnumerable collection to hold them. Then you can serialize that over the wire with minimal overhead.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak

devvvy4-Aug-10 14:03

4-Aug-10 14:03

you don't know you sh*t

dev

Dave Kreskowiak4-Aug-10 15:07

4-Aug-10 15:07

Voted to remove message. Too bad I can't vote to remove user...

I already told you how it's normally done. If you want ot continue to bang your head against a wall trying to do it the hard way, fine. Don't let me stop you.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak

devvvy4-Aug-10 20:31

4-Aug-10 20:31

Dave Kreskowiak wrote:
Voted to remove message. Too bad I can't vote to remove user...

why don't you ask babe(and start learning how to wire down a DataTable)

dev

Dave Kreskowiak5-Aug-10 1:53

5-Aug-10 1:53

Why should I, when I can do it faster and more efficiently by NOT using a DatTable?

A guide to posting questions on CodeProject[^]

Dave Kreskowiak

Dave Kreskowiak3-Aug-10 1:59

3-Aug-10 1:59

I wouldn't send a DataTable. I would normally send a custom collection of objects.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak

Pete O'Hanlon3-Aug-10 2:36

3-Aug-10 2:36

Use a strongly typed object that WCF can expose easily, and that can be recognised by more than just .NET.

"WPF has many lovers. It's a veritable porn star!" - Josh Smith

As Braveheart once said, "You can take our freedom but you'll never take our Hobnobs!" - Martin Hughes.

devvvy3-Aug-10 13:59

3-Aug-10 13:59

Thanks, one of our guys send over wcf as binary byte array over wsDualHttpBinding, which worked. I've been advised DataTable can actually be declared a DataTable instead of raw byte array using .NET Contract serializer and is just wondering what's best way

http://weblogs.asp.net/sweinstein/archive/2009/01/03/creating-high-performance-wcf-services.aspx[^]

http://social.msdn.microsoft.com/Forums/en-US/wcf/thread/d13b328b-6639-4b11-9f40-a17ccbc0c12b[^]

dev

David Skelly3-Aug-10 22:17

David Skelly

3-Aug-10 22:17

The first article you link to there is interesting because he actually advises not to send data as a DataTable because it is slow. He gained a six-fold performance increase by converting to POCO objects (an IEnumerable of type MyData in his example) and using a DataContractSerializer to send.

Pete O'Hanlon3-Aug-10 22:38

3-Aug-10 22:38

Just because you can transmit something as a datatable doesn't mean you should. Think about it this way - would you send your house through the post, just to send a letter that's on a desk? The DataTable infrastructure, in this case, is your house and the letter is the actual data you are trying to transmit.

The bottom line - a DataTable is too heavyweight an object to be a useful transmission mechanism; go for straight POCO instead and you can tune it to be as efficient as possible.

"WPF has many lovers. It's a veritable porn star!" - Josh Smith

As Braveheart once said, "You can take our freedom but you'll never take our Hobnobs!" - Martin Hughes.

devvvy5-Aug-10 5:38

5-Aug-10 5:38

i think advise to first convert to csv will work fine but i'm not too keen on writing the extra code just to juice the extra performance (presumeably non-significant) when the whole deal is wired within the almighty corporate intranet - we'd rahter wait till users moans about performance before we do this (besides, this is not the bottleneck at the moment, our primary being sql not clustered)

this said, curious, how much overhead with DataTable - the initial overhead, and per-row overhead (We've no time even to test this out... in fact we alrteady built this wiring DataTable as byte array), any experience to this end? is it appreciable?

dev

Pete O'Hanlon5-Aug-10 11:49

5-Aug-10 11:49

devvvy wrote:
any experience to this end

Yes.

devvvy wrote:
is it appreciable?

Yes.

devvvy wrote:
we'd rahter wait till users moans about performance before we do this

Trouble is - that costs more to fix. It's always more expensive to fix problem code later than sooner - changing from a DataTable later on will cause you all sorts of problems because you have other code that relies on the DataTable.

"WPF has many lovers. It's a veritable porn star!" - Josh Smith

As Braveheart once said, "You can take our freedom but you'll never take our Hobnobs!" - Martin Hughes.

devvvy5-Aug-10 14:09

5-Aug-10 14:09

Thanks - I'd just wire DataTable over wcf as it's already done. But build your wrapper around it so client code don't touch the later directly and change (wired as csv instead but delimitors can be a headache and also conversion and casting overhead ... need justify that as well) can be put in place later on underneath wrapper layer.

Just a thought, WCF is successor to asmx so definitely will choose WCF over asmx. What about socket? If interop/security/transaction/reliability all not *required* (think only WCF service we're using now is async call back- which really can also be easily replaced), would you think socket being a better choice for large data transfer over the wire?

dev

T M Gray3-Aug-10 11:23

T M Gray

3-Aug-10 11:23

Why not use Tab separated values or some other simplified format? DataTable is a very heavy object and serializing it would give you significantly more bytes to transfer. And from my experience Excel is pretty dumb when it comes to things like data types, so all of that info from the database would probably be lost in translation anyway.

devvvy4-Aug-10 13:59