|
Whats the most efficient way of storing and referecing a list of ~2000 unsorted key value pairs in C#?
I understand that there is not really any definite answer here since it will depend on the data, however, I'm not familiar enough with the .NET framework to know which ones to even test. I'm an MFC C++ developer who is migrating to C#. In MFC C++ I would use a CMap or a muiltidimensional array to store the data and it would be efficient enough for my needs. I'm finding in C#, neither (a series of arrays or a Hashtable) is working out very well. Both seem to be far less efficient than my implementations in C++ but I'm assuming this is mostly because of my inability to code C# as well as I do in C++.
For reference, I need to store 5-6 values for ~2000 keys (3-30 character strings) and receive between 200-10,000 updates a second (averaging about 750) total.
Recomendations? Should I look at another data storage method or should I work on improving my existing attempts?
|
|
|
|
|
I think a Dictionary should be fine. For most of the operations, execution time it proportional to number of elements in it (O(1)).
|
|
|
|
|
if you want to remember only one value (the most recent one) for each key you encounter, then yes a Dictionary is what you want, and
myDictionary[myKey]=myValue;
is all it takes.
Dictionaries are hashed; they are built on top of arrays, which when running out of their capacity are copied to larger ones. Therefore it may make sense to use a constructor overload that takes an initial capacity parameter.
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
|
|
|
|
|
Interested in what kind of 'performance' you are seeing and if you are running these tests in debug or release and if you are timing this in the IDE or not.
I have seen some interesting differences in speed and performance between these configurations before but when doing a test in a full production build outside the IDE they have ended up as very close to a wash. Also you may want to look at allocating new additions to the collections in groups rather than single records at a time. I have also seen that help performance wise.
|
|
|
|
|
Like others said, Dictionary<>() is a good bet. However, I wouldn't recommend using 3 - 30 character strings as the key. Performance will be better if you use something lighter like an int or something. You really haven't given us enough information to make a more informed design.
|
|
|
|
|
WTF?
This is not what I typed, and I can vote on it.
I can't delete or edit it.
Whats is wrong with this?
Also the other Answers do not appear to relate to the Question.
------------------------------------
I will never again mention that I was the poster of the One Millionth Lounge Post, nor that it was complete drivel. Dalek Dave
CCC League Table Link
CCC Link[ ^]
|
|
|
|
|
Dave, whatever it is you are doing in the C# forum, it is disruptive: the top thread is cut short, the next one is decapitated. You may want to get into your topmost message there and fix something. Thanks.
If you could note what the content is, and how things get fixed, it probably would be material for a sucks&bugs thread.
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
modified on Thursday, January 13, 2011 10:11 PM
|
|
|
|
|
I replied showing some code to give a total for the cells in his DGV.
There was nothing like what appears in my post up there.
Something is amiss!
It wasn't me, it was a big boy who did it and then he ran away!
------------------------------------
I will never again mention that I was the poster of the One Millionth Lounge Post, nor that it was complete drivel. Dalek Dave
CCC League Table Link
CCC Link[ ^]
|
|
|
|
|
OK, I'll take it to S&B authorities then.
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
|
|
|
|
|
You have 2000 fixed string keys?
Or you are discarding after 2000 and starting over with a new set?
And are you sizing the collections before you use it? And exactly what are you timing?
|
|
|
|
|
I've searched and found examples on how to write to an excel file, but I'm having trouble figuring out how to code to read all lines of a txt (say 10000 lines) and then write a new row every time a delineator is found which is on a line all by itself. The columns are predefined.
My code is as follows, but is not complete, any help would any appreciated. Thank you.
StreamReader parseFile = new StreamReader(@"C:\temp\output.txt");
string fileRead = parseFile.ReadToEnd();
string line;
string year = DateTime.Now.Year.ToString();
while ((line = parseFile.ReadLine()) != null)
{
int linenumber = 0;
if (line.EndsWith("(null)") != true)
{
linenumber = linenumber + 1;
Excel.Range range = oSheet.get_Range("A" + linenumber, );
range.Value2 = line;
}
}
parseFile.Close();
|
|
|
|
|
This[^] CP Archive Item may help.
------------------------------------
I will never again mention that I was the poster of the One Millionth Lounge Post, nor that it was complete drivel. Dalek Dave
CCC League Table Link
CCC Link[ ^]
|
|
|
|
|
Hello,
I have a winform project (c#) with a media player embedded inside.
When I start playing a song, I can see the equalizer inside the WMP window.
I want to duplicate this window so I will see the same window twice (right side and left side of my winform window).
Can I do that?
I thought about adding another WMP which will play the same songs but I'm afraid that it won't play at the same time.
Thanks
|
|
|
|
|
I am trying to customize the IME UI windows in a C# application which custom draws all of its controls. The main form can get the WM_IME_SETCONTEXT, and a couple of other messages, but I don't see how to trigger the IME to tell my application when to open any of the IME windows (composition, reading, or candidate). Are there any C# samples that I can look at which do the same? Or any C++ for that matter?
Or does anyone have any idea what I might need to do to trigger the IME to request these windows be opened? I set the IME to use PRC Chinese and I see the WM_IME_COMPOSITION message. In response to that message, ImmGetCompositionString() returns zero length. Then I see no additional messages.
I am responding to WM_IME_SETCONTEXT by clearing the lPARAM and then passing it to the DefWinProc, allowing the latter to set the results.
Any help would be appreciated.
Update: It seems that the problem is the IME/IMM is not aware that it should be processing the WM_KEYDOWN, and I am getting WM_CHAR messages instead of WM_IME... messages for composition and more.
modified on Thursday, January 13, 2011 9:58 PM
|
|
|
|
|
I am working on implimenting a 3rd party API written in C++. The API pulls an image from a scanner. I have tried a few different ways of of working with the image struct but every attempt causes memory crashes with P/Invoke. Any suggestions would be appreciated.
Here is the C++ code and sample that comes with the device.
typedef struct {
int width;
int height;
int info;
unsigned char* pbuf;
} MY_IMAGE;
MY_IMAGE img1,img2;
img1.width = 864;
img1.height = 3000;
img1.info = 8;
img1.pbuf = (UCHAR *)new UCHAR[2*nWidth*nHeight];
img2.width = 864;
img2.height = 3000;
img2.info = 8;
img2.pbuf = (UCHAR *)new UCHAR[2*nWidth*nHeight];
ret=WScanSelectBuf(&img1, &img2,0);
---------------------------------------------------------------------
[StructLayout(LayoutKind.Explicit)]
public struct MY_IMAGE
{
[FieldOffset(0)]
public int width;
[FieldOffset(4)]
public int height;
[FieldOffset(8)]
public int info;
[FieldOffset(12), MarshalAs(UnmanagedType.SysUInt)]
public IntPtr pbuf;
}
[DllImport("scandll.dll")]
private extern static int WT_ScanSelectBuf(ref MY_IMAGE image_down, ref MY_IMAGE image_up, int select);
public static int ScanSelectBuf(ref MY_IMAGE image_down, ref MY_IMAGE image_up, int select)
{
return WScanSelectBuf(ref image_down, ref image_up, select);
}
ImageInfo img1 = new ImageInfo();
ImageInfo img2 = new ImageInfo();
int nWidth = 864;
int nHeight = 1000;
img1.width = nWidth;
img1.height = nHeight;
img1.info = 8;
img2.width = nWidth;
img2.height = nHeight;
img2.info = 8;
ret = ScanSelectBuf(ref img1,ref img2, 0);
|
|
|
|
|
At a first glance, this is what I would do:
Declare MY_IMAGE as a class, not as a struct; use LayoutKind.Sequential instead of Explicit; remove FieldOffset attributes; do not use MarshalAs for pbuf field (pbuf should be an IntPtr); import the function without the ref keywords.
To assign the value to pbuf field you need to have the bytes in a byte[] array. Then you just have to set pbuf to the address of that array, using Marshal.UnsafeAddrOfPinnedArrayElement method.
|
|
|
|
|
Hi Erik, thanks for the suggestion. Below are the changes I have made. However, I am still receiving memory errors.
[StructLayout(LayoutKind.Sequential)]
public class MY_IMAGE
{
public int width;
public int height;
public int info;
public IntPtr pbuf;
}
[DllImport("scandll.dll")]
private extern static int WScanSelectBuf(MY_IMAGE image_down, MY_IMAGE image_up, int select);
public static int ScanSelectBuf(ref MY_IMAGE image_down, ref MY_IMAGE image_up, int select)
{
return WScanSelectBuf(image_down, image_up, select);
}
And also
public static int ScanSelectBuf(MY_IMAGE image_down, MY_IMAGE image_up, int select)
{
return WScanSelectBuf(image_down, image_up, select);
}
The API does not provide any other way to fill the MY_IMAGE struct other than calling:
private extern static int WScanSelectBuf(MY_IMAGE image_down, MY_IMAGE image_up, int select);
So, I do not have acces to fill the buffer with a byte[]. This should be happening from caling the method.
Erik - Is there anything I missed? If so, do you have an example?
Luc - Would you be able to provide an example of what you were referring to?
|
|
|
|
|
econner wrote: So, I do not have acces to fill the buffer with a byte[]. This should be happening from caling the method.
I understand that the buffers are filled by WScanSelectBuf, am I right? If so, maybe you have to allocate these buffers before calling the function, I mean, create the instances of byte[] array, pin them with GCHandle and assing the address to the pbuf field of the MY_IMAGE objects. Can you give us a link to the documentation for the C++ library you are using? I would like to know what the three parameters are for, and what the WScanSelectBuf function is supposed to do. I think we might be able to give you a better help with a little more information.
|
|
|
|
|
Correct, on the buffer.
Here is an excert from the docs:
//Scan to buffer.
int WScanSelectBuf(void* image_down, void* image_up ,int select)
(param)
image_down [out] structure for forward direction scan.
image_up [in] NULL (reserved)
select
[in] 0 : forward direction & single scan (use image_down)
1 : backward direction & single scan (use image_up)
2 : both direction & single scan (use image_down & image_up)
3 : forward direction & dual scan (use image_down)
4 : backward direction& dual scan (use image_up)
5: both direction & dual scan (use image_down& image_up)
Return value - Success 0, Fail > 0
|
|
|
|
|
Have you allocated and pinned the byte[] arrays before calling the function?
|
|
|
|
|
No, I have not. From the C++ example, they show setting the pbuf size, but with using IntPtr I was not sure how to accomplish the same thing.
|
|
|
|
|
Create a byte[] array with the same size you need in C++ for UCHAR. Use GCHandle class to pin this array, so the GC will not reallocate it in case of a garbage collection, and set its address to the pbuf field. Have a look at the documentation of GCHandle class.
|
|
|
|
|
I have modified the code as per my understanding of GCHandle. I am still getting memory errors. I believe it has something to do with the IntPtr of the pbuf in the MY_IMAGE struct.
Here is my code so far:
[StructLayout(LayoutKind.Sequential)]
public class MY_IMAGE
{
public int width;
public int height;
public int info;
public IntPtr pbuf;
}
[DllImport("scandll.dll")]
private extern static int WScanSelectBuf(IntPtr image_down, IntPtr image_up, int select);
public static int ScanSelectBuf(MY_IMAGE image_down, MY_IMAGE image_up, int select)
{
GCHandle image_downHandle = GCHandle.Alloc(image_down, GCHandleType.Pinned);
GCHandle image_upHandle = GCHandle.Alloc(image_up, GCHandleType.Pinned);
int ret = WScanSelectBuf(image_downHandle.AddrOfPinnedObject(),
image_upHandle.AddrOfPinnedObject(), select);
image_downHandle.Free();
image_upHandle.Free();
return ret;
}
MY_IMAGE img1 = new MY_IMAGE();
MY_IMAGE img2 = new MY_IMAGE();
int nWidth = 864;
int nHeight = 1000;
img1.width = nWidth;
img1.height = nHeight;
img1.info = 8;
img2.width = nWidth;
img2.height = nHeight;
img2.info = 8;
ret = ScanSelectBuf(img1, img2, 0);
Anything I am missing ???
|
|
|
|
|
Yes, you are pinning the MY_IMAGE objects, but that is not what you have to pin. When you call a function using P/Invoke, the CLR automatically pinnes in the managed heap the references you pass, so there is no need to explicitly pin them as you have done. However, the pbuf field of each MY_IMAGE class is an IntPtr, what means that it only contains the memory address of the buffer allocated to receive de image pixels. As it is only an address, the CLR will not pin these buffers automatically, and that is why the programmer has to pin them manually before using the P/Invoke call. In this case you have not even allocated the buffers, so the values of the pbuf fields of the MY_IMAGE objects you are passing to the unmanaged function are IntPtr.Zero, I mean, the default value for IntPtr.
This is what you have to do:
Before you call the WScanSelectBuf you have to create the two MY_IMAGE objects and allocate the byte[] arrays where that function will place the image pixel:
MY_IMAGE img1 = new MY_IMAGE();
MY_IMAGE img2 = new MY_IMAGE();
byte[] buffer1 = new byte[2*img1.width*img1.height];
byte[] buffer2 = new byte{2*img2.width*img2.height];
GCHandle bufferHandle1 = GCHandle.Alloc(buffer1, GCHandleType.Pinned);
GCHandle bufferHandle2 = GCHandle.Alloc(buffer2, GCHandleType.Pinned);
img1.pbuf = bufferHandle1.AddrOfPinnedObject();
img2.pbuf = bufferHandle2.AddrOfPinnedObject();
int ret = WScanSelectBuf(img1, img2, select);
bufferHandle1.Free();
bufferHandle2.Free();
Tell us if it works.
|
|
|
|
|
I modified the code as per above. The img1.pbuf and img2.pbuf are now showing an address. However, the native API funtion of WScanSelectBuf(img1, img2, select) is still crashing. I have tried passing in:
WScanSelectBuf(img1, img2, select) (MY_IMAGE)
WScanSelectBuf(ref img1, ref img2, select) (ref of MY_IMAGE)
WScanSelectBuf(img1, img2, select) (IntPtr)
and changed the code in the DLLImport and wrapper method without any success. Is there anything I am missing with Marshaling, etc?
I am coming to the conclusion that it could be an issue with the 3rd party's API libary and is doing something odd with the function. However, it does work in their C++ sample.
I have contacted the vendor and they are not familar with .NET. However, they are reviewing this thread and for any suggestions.
|
|
|
|
|