|
It's great to see a well-written, well-supported utility like this. It looks just what I need to deal with some Excel 2007 spreadsheets created by another department. I'm trying to use your code to parse a csv I created from one of them. My code begins:
using (LumenWorks.Framework.IO.Csv.CsvReader csv =
new LumenWorks.Framework.IO.Csv.CsvReader(new StreamReader(inputFileName), true))
{
csv.MissingFieldAction = LumenWorks.Framework.IO.Csv.MissingFieldAction.ReplaceByEmpty;
csv.SkipEmptyLines = true;
int fieldCount = csv.FieldCount;
As soon as it gets to csv.FieldCount, VS2005 throws a "An item with the same key has already been added" exception. Do you have any idea why?
The first three lines of the csv are (each on its own line):
Site,Division,Directorate,Department,Team,Directory Dept Name,Surname,Forename,Level 7,Extension,Official Job Title,Co. Mobile,Category,Phone ID,Note,
,,,,,,,,,,,,,,P = people,
,,,,,,,,,,,,,,J = roles,
|
|
|
|
|
I tried the code snippet you provided and I cannot reproduce the bug you describe. Here is my code:
string data = @"Site,Division,Directorate,Department,Team,Directory Dept Name,Surname,Forename,Level 7,Extension,Official Job Title,Co. Mobile,Category,Phone ID,Note,
,,,,,,,,,,,,,,P = people,
,,,,,,,,,,,,,,J = roles,";
using (CsvReader csv = new CsvReader(new System.IO.StringReader(data), true))
{
csv.MissingFieldAction = MissingFieldAction.ReplaceByEmpty;
csv.SkipEmptyLines = true;
int fieldCount = csv.FieldCount;
}
|
|
|
|
|
Thanks for trying. Ironically, neither can I, now. I recreated the csv file from the source (for the nth time) and now I'm not seeing the problem.
|
|
|
|
|
I am experiencing this same problem. In my case, if I do a raw dump of the csv data, the first line is:
"Product Group,Lit Number (short),Thumbnail URL,Title,Description,LIbraryLink,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
I don't know why it has all those empty headers, but that's why the error is being thrown. Is there any way to suppress "EmptyHeaders" like we can suppress EmptyLines?
-- Joel Clermont
|
|
|
|
|
An option in the constructor let you set the reader to treat header as simple data. By doing so, you will need to access fields by their index, unless you build your own name->index dictionary (and skip those empty headers).
|
|
|
|
|
I'm getting the same!!!
An item with the same key has already been added.
void ReadCsv(string CSV_file)<br />
{<br />
using (CsvReader csv = new CsvReader(new StreamReader(CSV_file), true))<br />
{<br />
csv.MissingFieldAction = MissingFieldAction.ReplaceByNull;<br />
<br />
<br />
<br />
int fieldCount = csv.FieldCount;<br />
string[] headers = csv.GetFieldHeaders();<br />
<br />
while (csv.ReadNextRecord())<br />
{<br />
for (int i = 0; i < fieldCount; i++)<br />
Console.Write(string.Format("{0} = {1};",headers[i],csv[i] == null ? "MISSING" : csv[i]));<br />
Console.WriteLine();<br />
}<br />
}<br />
}
by the way the dll file (LumenWorks.Framework.IO.dll) is version 3.0.0.0 and I just downloaded from here yesterday !!!
|
|
|
|
|
i'm getting exactly the same !!!
here is more details:
System.ArgumentException was unhandled<br />
Message="An item with the same key has already been added."<br />
Source="mscorlib"<br />
StackTrace:<br />
at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)<br />
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)<br />
at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)<br />
at LumenWorks.Framework.IO.Csv.CsvReader.ReadNextRecord(Boolean onlyReadHeaders, Boolean skipToNextLine)<br />
at LumenWorks.Framework.IO.Csv.CsvReader.EnsureInitialize()<br />
at LumenWorks.Framework.IO.Csv.CsvReader.get_FieldCount()<br />
at WindowsApplication1.Form1.ReadCsv(String CSV_file) in Form1.cs:line 32<br />
at WindowsApplication1.Form1.openCSVFileToolStripMenuItem_Click(Object sender, EventArgs e) in Form1.cs:line 56<br />
at System.Windows.Forms.ToolStripItem.RaiseEvent(Object key, EventArgs e)<br />
at System.Windows.Forms.ToolStripMenuItem.OnClick(EventArgs e)<br />
at System.Windows.Forms.ToolStripItem.HandleClick(EventArgs e)<br />
at System.Windows.Forms.ToolStripItem.HandleMouseUp(MouseEventArgs e)<br />
at System.Windows.Forms.ToolStripItem.FireEventInteractive(EventArgs e, ToolStripItemEventType met)<br />
at System.Windows.Forms.ToolStripItem.FireEvent(EventArgs e, ToolStripItemEventType met)<br />
at System.Windows.Forms.ToolStrip.OnMouseUp(MouseEventArgs mea)<br />
at System.Windows.Forms.ToolStripDropDown.OnMouseUp(MouseEventArgs mea)<br />
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)<br />
at System.Windows.Forms.Control.WndProc(Message& m)<br />
at System.Windows.Forms.ScrollableControl.WndProc(Message& m)<br />
at System.Windows.Forms.ToolStrip.WndProc(Message& m)<br />
at System.Windows.Forms.ToolStripDropDown.WndProc(Message& m)<br />
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)<br />
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)<br />
at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)<br />
at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)<br />
at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(Int32 dwComponentID, Int32 reason, Int32 pvLoopData)<br />
at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)<br />
at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)<br />
at System.Windows.Forms.Application.Run(Form mainForm)<br />
at WindowsApplication1.Program.Main() in Program.cs:line 17<br />
at System.AppDomain.nExecuteAssembly(Assembly assembly, String[] args)<br />
at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)<br />
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()<br />
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)<br />
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)<br />
at System.Threading.ThreadHelper.ThreadStart()
|
|
|
|
|
The field header names must be unique.
|
|
|
|
|
now it worked for me
Thanks allot
|
|
|
|
|
Hi
I too am experiencing the same issue. If I set a break point on the line and hover over the csv.FieldCount property the intellisence shows the exception. However, if i then move the mouse off the line and then immediately back on the correct value is shown and the method can continue.
As you can appreciate, this was not a feasible solution post-development.
After quite some time debugging the CSVReader code i narrowed the problem down to line 1460 of CSVReader.cs in the ReadNextRecord(bool,bool) method:
_fieldHeaderIndexes.Add(_fields[i], i);
It quickly became apparent that my CSV file had non-unique headers and this was causing the error. However, this doesn't appear to be your issues. The only thing i could suggest is removing the final ',' from each line.
Sorry i couldn't be of more use.
Cheers,
Martyn...
|
|
|
|
|
Hi friends,
Just before the line _fieldHeaderIndexes.Add(_fields[i], i); i added a condition
if (!_fieldHeaderIndexes.ContainsKey(_fields[i]))
_fieldHeaderIndexes.Add(_fields[i], i); and my issue is resolved.
Thanks,
Saranya.
|
|
|
|
|
Dear all,
is it possible fetching only a number of data to work on it and then fetching the next part and so on?
Is it possible to read a few 100.000 sentences with the csv-reader?
Is it possible to check data if there are any special signs in this data?
Would you please inform me about the costs of this component?
How long will an order last?
Ella Becker
|
|
|
|
|
1- Yes but you have to do it manually, ie read 100 records, process them, repeat
2- I am not sure what you mean by that. If you are talking about data volume, this reader will handle it just fine.
3- You cannot do automatic checks using the reader, but you can process a field value in any way you need once you got it from the reader.
4- free
5- forever
|
|
|
|
|
I should precise that the code is licensed under the MIT license which is included in the download. All it requires is that you include the content of that license with your product, alongside your own.
|
|
|
|
|
Hi,
Congrats on the CSV Reader - very useful and saved me a lot of development time!
I've got a WinForm app with a DataGridView. The datasource is a CachedCsvReader.
Once the CSV is loaded in to the data view I want to brake the link so that I can edit the contents of the grid (I will never want to write back to the file). Any suggestions on how to do this? At present, the grid appears readonly, despite the readonly properties being 'false'.
Thanks,
Graeme
|
|
|
|
|
Thanks for your comment! Unfortunately, I did not implement write support. The relevant code is found in the CachedCsvReader.CsvBindingList class and this is where you would need to add relevant code (ie implement methods such as Add, Remove, Clear, etc.).
|
|
|
|
|
|
It does handle UTF-8, Unicode, etc. just fine. You simply need to use a different encoding for the stream. You can search previous comments to have more information.
|
|
|
|
|
I have a sample file that contains the same quote character " used in quoting the fields as part of the data within fields.
Example: "Monitor 22" diagonal"
Does the CSV Reader not handle this? Excel seems to read the file in without problems as-is. I don't have any control over how the csv files are generated, so adding an excape character is not an option.
Thanks - and, it looks to be very nicely written
|
|
|
|
|
Well, I read some more in the previous postings (didn't see that there was a 'next' button before posting my question)
I see that I can use a different escape character, which I set to '\'. Interestingly, now all fields that are quoted show up in the results with the quotes. So, my code that makes use of the CSVReader expects to remove the quotes for the final result.
Anyway, I have a solution that seems to work, though it'd be nice to not have to replace quotes in the final results.
I am impressed with the speed - thanks for making this code available.
|
|
|
|
|
Hum, the value returned should be without starting and ending quote. Could you post a sample causing this behavior ?
As for your original question, quotes inside quoted field are supported only if escaped. Microsoft has their own way with standards, even when they are dead simple. *sigh*
|
|
|
|
|
Hi!
This really is an extremely fast csv parser.
Can you tell me how to handle a file with variable field count.
Eg:
x,y
,,"a2,pk",a,,
b,
This file is without header and footer. Currently the FieldCount gets set to the count of the first row and the parsing takes place on this field count which is less than the maximum occuring in the second row of the example.
Thanks.
Prashant
|
|
|
|
|
Many people have this issue but unfortunately, that was not a requirement when I made the reader and adding this kind of logic is not so simple if I want to solve this issue completely.
Normally, there should always have commas for the blank fields. Otherwise, how to distinguish between intentionally omitted fields and missing fields ? As it is, the csv you give me as an example is badly formed. If I was going to work on a solution, I would need to provide an option letting users specify how to handle that.
|
|
|
|
|
its works well BUT REALLY VERY FAST
Do you have idea about Calculating MS EXCEL Sheet Records too FAST?
We are developing an Application for Actuaries and the Core Req is
the FASTEST & Robust way...
Give your Suggesions..
Thanx
Bye
ASif Ashraf
MCAD.Net,MCP
Asif.Log@gmail | hotmail.com
asif@colossys.com
92-306-4526526
Sr. developer Colossys LLC
|
|
|
|
|
Hi Sébastien
Your CSV parser is excellent..but how can we find a footer text in the file useing a streamreader.
CSV file parser even doent deals with the field within fileds
ex.
A, B, C, D, E suppose its a header
a , b|d|e, z, d, e Records and second field is conating more fields
Footer text .....
How can this CSV parser will help if i have to find a way for dealing fileds within fields and footer extraction .
Prashant Kumar
|
|
|
|
|