|
I enabled logging but the error occurs before EF gets to emit any SQL for that query.
If the brain were so simple we could understand it, we would be so simple we couldn't. — Lyall Watson
|
|
|
|
|
Hi,
You can try enabling CLR Exception form Debug->Exceptions and enable check box for throw on Common Language Runtime Exception
Regards,
Raj Champaneriya
|
|
|
|
|
It's enabled but it doesn't help: The break occurs only on my code, not in the EF source.
If the brain were so simple we could understand it, we would be so simple we couldn't. — Lyall Watson
|
|
|
|
|
What's the exact error you're getting?
|
|
|
|
|
It's a null-ref. But the reason for that isn't that my query expression is completely flawed but that EF can't deal with a Tuple<> as a container-object for a join-result. If I use a custom class instead of a Tuple it works. I want to find out why it doesn't work with a Tuple, just for "academic insight" - so I would need to see the EF code where this exception originates from.
If the brain were so simple we could understand it, we would be so simple we couldn't. — Lyall Watson
|
|
|
|
|
...a Tuple as a container object for a join?
Interesting. Code snippet please.
I can see a bunch of possible reasons why it won't work, not the least of which is EF has no mapping information for the Tuple and, second, I don't think the SQL provider has any clue how to convert a Tuple<t1, t2=""> into a SQL query that makes sense.
|
|
|
|
|
It's a formally valid query expression. The Tuple is just a replacement for the "anonymous type" which normally is the return value from the "resultSelector"-lambda.
Code follows.
If the brain were so simple we could understand it, we would be so simple we couldn't. — Lyall Watson
|
|
|
|
|
My statement that it worked with a custom type instead of the Tuple wasn't correct. It did work but now I recall it wasn't the same code then; I previously joined on IEnumerables, not on IQueryables so the join wasn't actually performed by EF resp. SQL. However, a different solution doing mostly the same did work and from that I worked out what the problem was: The New-Expression for the join-resultSelector requires the Members-Property[^] to be initialized for the Query-Provider, presumably to access the correct properties of the join-result-object.
The code that didn't work until now is a simplified version of the other which I want to use as a didactic middle step in an article I want to write on dynamic joins. You'll see the code then when I publish it
However, I'd still be interested to figure out why VS doesn't step into the EF source. Any idea?
If the brain were so simple we could understand it, we would be so simple we couldn't. — Lyall Watson
|
|
|
|
|
I've done .NET Framework source debug but never EF.
EF isn't part of the Reference Source for .NET so I'm guessing you would have to download the current source from here (6.1.3)[^] compile it and dump the symbol files into a local folder. You'd probably then have to setup a symbol path to that folder in Visual Studio under Tools -> Options -> Debugging -> Symbols.
I've never done it so I really am just guessing at this!
|
|
|
|
|
I'll try that. Thanks, Dave!
If the brain were so simple we could understand it, we would be so simple we couldn't. — Lyall Watson
|
|
|
|
|
I'm using the NuGet packages rx-main and rx-xaml in my code.
I was basically just reading up on the subject
MSDN Blogs[^], but some of the examples[^] just confused me more than they clarified.
What I (think?) learned were that if you subscribed on the same thread as the never ending function while observing on a different thread, your subscription code would execute. But if I commented out the SubscribeOn code from the example:
.ObserveOn(Scheduler.Default)
It had the exact same effect as if it was there. The subscription seem to run on the same thread as the ObserveOn. So what was the point of SubscribeOn exactly? The problem is that I cant really see any use for it as of now.
IF I wanted to execute something on a different thread I could do this, an example from [WP7Dev] Using the WebClient with Reactive Extensions for Effective Asynchronous Downloads[^]:
public IObservable<string> StartDownload(string uri)
{
WebClient wc = new WebClient();
var o = Observable.FromEventPattern<DownloadStringCompletedEventArgs>(wc, "DownloadStringCompleted")
.ObserveOn(Scheduler.Default)
.Select(newString => ProcessString(newString.EventArgs.Result));
wc.DownloadStringAsync(new Uri(uri));
return o;
}
public string ProcessString(string s)
{
Thread.Sleep(3000);
return s + "<!-- Processing End -->";
}
public void DisplayMyString()
{
var asyncDownload = StartDownload("http://bing.com");
var asyncDownload2 = StartDownload("http://google.com");
var zipped = asyncDownload.Zip(asyncDownload2, (left, right) => left + " - " + right);
zipped.ObserveOnDispatcher().SubscribeOnDispatcher()
.Subscribe(s => label.Text = s);
}
I put SubscribeOnDispatcher in as a test, but it is not needed. Am I missing something here, or is the SubscribeOnDispatcher totally useless?
|
|
|
|
|
Errm, I'm not sure what other behaviour you are expecting. If you are subscribing on Scheduler.CurrentThread, you will use whatever the current thread context is when you subscribe. If you want it to run on a different thread, use SubscribeOn(Scheduler.NewThread) and then marshall it back with ObserveOn(Scheduler.Default) .
This space for rent
|
|
|
|
|
If I try to use Scheduler.NewThread I get a warning that it's obsolete and that I should use Scheduler.Default instead. But I must say the names are really confusing.
That aside, I don't quite see a need for SubscribeOn method at all, I could just as well do something along the lines of:
var o = Observable.FromEventPattern<DownloadStringCompletedEventArgs>(wc, "DownloadStringCompleted")
.ObserveOn(Scheduler.Default)
.Select(newString => BackgroundWorkerThread(newString.EventArgs.Result))
.ObserveOnDispatcher();
If I tried to interact with some UI control elements I need to have the values on the same thread as I observe on?
|
|
|
|
|
It's all to do with what the purposes of the SubscribeOn and ObserveOn were designed for (I know, that seems to be an obvious statement). Basically, SubscribeOn is the interceptor for the Subscribe method so the reason you don't need it in your example is because you aren't using a Subscriber. If you had a Subscribe() method, the SubscribeOn method would have an effect. ObserveOn interacts with the OnNext, OnError and OnComplete methods. It's that simple.
From that, you can probably guess that SubscribeOn is responsible for interacting with the disposable.
This space for rent
|
|
|
|
|
Hrmf, they could just have told me that.
But now I do see the light, it's o' so obvious now.
So thank you.
|
|
|
|
|
You're welcome. It took me a while to figure this one out as well. The problem is, the documentation is all over the place as the examples you tend to find relate to Rx as it was about 6 years ago and it's moved on so much. The API has undergone fairly massive changes in this time.
This space for rent
|
|
|
|
|
Pete O'Hanlon wrote: he documentation is all over the place as the examples you tend to find relate to Rx as it was about 6 years ago and it's moved on so much
This is one of my biggest concerns using Rx, as it is now open source, the documentation could be misleading or directly wrong, depending on the version you are using. I was thinking about writing an overview article about Rx, but I'm really worried that I would just teach people some bad things that they shouldn't do, or somethings that would cause more harm than they help.
|
|
|
|
|
Just a note: The on-line book: Introduction to Rx[^] is really pretty good, even though it is pre-V2!
Some of the names of things have changed, (e.g., Scheduler.NewThread => NewThreadScheduler.Default) but for the basics, it is pretty nice!
"Fairy tales do not tell children the dragons exist. Children already know that dragons exist. Fairy tales tell children the dragons can be killed."
- G.K. Chesterton
|
|
|
|
|
Yes, I have read it quite a bit, should mention that the Lee Cambell also has some valuable answers on the MSDN page as well:
Msdn forums - Reactive Extensions (Rx)[^]
However they all seem to have most of the activity from 2012 or before.
|
|
|
|
|
Hmm, the Scheduler.Default and the NewThreadScheduler.Default seem to be the exact same command? They both live in the System.Reactive.Concurrency namespace, and I get the same error, that its not the UI Thread.
|
|
|
|
|
If you want the UI thread scheduler for the ObserveOn I think what you want instead is to use the .ObserveOnDispatcher() extension method, in the System.Reactive.Linq namespace.
This, apparently (from looking at the version 2 source!), is the recommended way to use the DispatcherScheduler.Current scheduler.
================
[Edit: I see that you already found this!]
"Fairy tales do not tell children the dragons exist. Children already know that dragons exist. Fairy tales tell children the dragons can be killed."
- G.K. Chesterton
|
|
|
|
|
Either Im missing something here, or Im doing something wrong?
var buttonClick = Observable.FromEventPattern<RoutedEventHandler, RoutedEventArgs>(
h => btnMainWindow.Click += h,
h => btnMainWindow.Click -= h);
var EnumerableTest = Enumerable
.Range(1, 10)
.ToObservable()
.ObserveOnDispatcher()
.SubscribeOn(NewThreadScheduler.Default);
var DisposableEnumerable = EnumerableTest
.Subscribe(evt =>
txtMainWindow.Text += evt + Environment.NewLine
);
buttonClick.Subscribe(evt =>
{
DisposableEnumerable.Dispose();
});
There is no change if I replace:
.SubscribeOn(NewThreadScheduler.Default)
with
.SubscribeOnDispatcher()
If I understood things, the SubscribeOn should specify where the disposable lives, so if I set it on a different thread, what happens then?
|
|
|
|
|
I have a class with 11 string properties, together with an IEnumerable (Initialised to a generic List) property, each item of which has an int property, a long property and 3x Nullable int (int?) properties. The typical number of items in this sub property list is 4-6.
I have an IEnumerable containing ~700K of these, and I'm breaking them out into batches of 1000 using the following code;
var enumerator = items.GetEnumerator();
enumerator.MoveNext();
List<object> entityBatch = new List<object>();
DateTime start = DateTime.Now.Ticks;
for (int iterStep = 0; iterStep < batchSize; iterStep++)
{
entityBatch.Add(enumerator.Current);
enumerator.MoveNext();
}
DateTime end = DateTime.Now.Ticks;
However, to process this loop is taking around 12 seconds (as measured using the start / end tick count)....that in itself is bad, but the real question is can anybody shed any light on the fact that if I remove the list property, the same data is processed in 0.25 seconds - so several orders of magnitude more quickly?
I realise there will be some casting going on from objects to my actual types, but the difference makes me think I'm missing something pretty significant.
C# has already designed away most of the tedium of C++.
modified 24-Feb-16 9:33am.
|
|
|
|
|
Don't measure the performance of your code using DateTime ; use the Stopwatch class[^] instead.
Did you make sure to "warm up" the code before you timed it, and run the test over many iterations?
Since you know the maximum number of items you'll be adding to the list, you should set its capacity when you create it:
List<object> entityBatch = new List<object>(batchSize);
If you don't set the capacity, it starts at 4 and doubles each time it runs out of space.
You need to check the value returned from enumerator.MoveNext ; if it returns false , you've reached the end of the sequence. Your code currently continues adding the final items from the sequence to the list until you reach the batch size.
You should consider using proper generic types, rather than a List<object> ; that way, you avoid having to cast the items back to the correct type when you read the batches.
It's probably best to wrap this sort of thing up as a extension method. Here's the one I use:
public static class EnumerableExtensions
{
private static IEnumerable<IEnumerable<TSource>> BatchIterator<TSource>(IEnumerable<TSource> source, int size)
{
int count = 0;
var bucket = new TSource[size];
foreach (var item in source)
{
bucket[count] = item;
checked
{
count++;
}
if (count == size)
{
yield return bucket.Select(x => x);
bucket = new TSource[size];
count = 0;
}
}
if (count != 0)
{
yield return bucket.Take(count);
}
bucket = null;
}
public static IEnumerable<IEnumerable<TSource>> Batch<TSource>(this IEnumerable<TSource> source, int size)
{
if (source == null) throw new ArgumentNullException(nameof(source));
if (size < 1) throw new ArgumentOutOfRangeException(nameof(size));
return BatchIterator(source, size);
}
}
If you can, try to reproduce the problem in a small example, and either post it here, or create a .NET Fiddle[^] and post the link.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Hi Richard - many thanks for the quick reply...
Richard Deeming wrote: Don't measure the performance of your code using DateTime ; use the Stopwatch class[^] instead.
I'm not measuring the performance per-se, just getting an idea, and in any case that part of the code won't be in the prod version...
Richard Deeming wrote: Did you make sure to "warm up" the code before you timed it, and run the test over many iterations?
Absolutely - same perf over many MANY runs
Richard Deeming wrote: Since you know the maximum number of items you'll be adding to the list, you should set its capacity when you create it:
List<object> entityBatch = new List<object>(batchSize);
If you don't set the capacity, it starts at 4 and doubles each time it runs out of space.
Valid point - made the change and no real effect - still around 12s
Richard Deeming wrote: You need to check the value returned from enumerator.MoveNext ; if it returns false , you've reached the end of the sequence. Your code currently continues adding the final items from the sequence to the list until you reach the batch size.
Again, valid point, but in this instance it's not the problem - I can be certain that it's not at the end.
Richard Deeming wrote: You should consider using proper generic types, rather than a List<object> ; that way, you avoid having to cast the items back to the correct type when you read the batches.
Also a valid point, but not possible in this instance - it's in an abstract base class, which operates over several types of similar but not related types and exists in a (necessarily) singleton class....my thinking is leading me to the point that it may well be an area to look at, but given that IEnumerator.Current returns an object type anyway I'm not sure.
Richard Deeming wrote: It's probably best to wrap this sort of thing up as a extension method
Ultimately that's a possibility, but until I can bottom out the cause of the several orders of magnitude difference it's sort of neither here or there - appreciate the comment however.
Many thanks again - I'll have to keep digging into it and see what's what!
C# has already designed away most of the tedium of C++.
|
|
|
|