|
From my point of view, the CSV is well formed, but with a logical error due to duplicate names in the header. I agree I could handle this case and throw a more appropriate exception. In next version
|
|
|
|
|
I've got a CSV file that was generated from an Excel file in the following format:
Date,Engine,Remote Campaign,AdGroup,Keyword,Contextual,Impressions,Clicks,Media Cost,PageViews : Count,Visits : Count
26-Feb-07,Yahoo,USA Today - Tech Evergreen,Default,Default,No,0,0,$0.00 ,,
27-Feb-07,Yahoo,USA Today - Tech Evergreen,Default,Default,No,0,0,$0.00 ,,
28-Feb-07,Yahoo,USA Today - Tech Evergreen,Default,Default,No,0,0,$0.00 ,,
1-Mar-07,Yahoo,USA Today - Tech Evergreen,Default,Default,No,0,0,$0.00 ,,
2-Mar-07,Yahoo,USA Today - Tech Evergreen,Default,Default,No,0,0,$0.00 ,,
3-Mar-07,Yahoo,USA Today - Tech Evergreen,Default,Default,No,0,0,$0.00 ,,
4-Mar-07,Yahoo,USA Today - Tech Evergreen,Default,Default,No,0,0,$0.00 ,,
26-Feb-07,Google,USATODAY_Tech,blogmaney,tech blogger,No,3,0,$0.00 ,,
27-Feb-07,Google,USATODAY_Tech,blogmaney,tech blogger,No,2,0,$0.00 ,,
28-Feb-07,Google,USATODAY_Tech,blogmaney,tech blogger,No,2,0,$0.00 ,,
1-Mar-07,Google,USATODAY_Tech,blogmaney,tech blogger,No,2,0,$0.00 ,,
2-Mar-07,Google,USATODAY_Tech,blogmaney,tech blogger,No,6,0,$0.00 ,,
3-Mar-07,Google,USATODAY_Tech,blogmaney,tech blogger,No,3,0,$0.00 ,,
4-Mar-07,Google,USATODAY_Tech,blogmaney,tech blogger,No,3,0,$0.00 ,,
27-Feb-07,Google,USATODAY_Tech,blogmaney,technology blog,No,0,0,$0.00 ,,
28-Feb-07,Google,USATODAY_Tech,blogmaney,technology blog,No,1,0,$0.00
2-Mar-07,Google,USATODAY_Tech,blogmaney,technology blog,No,2,0,$0.00
3-Mar-07,Google,USATODAY_Tech,blogmaney,technology blog,No,1,0,$0.00
4-Mar-07,Google,USATODAY_Tech,blogmaney,technology blog,No,1,0,$0.00
Using the latest
_csvReader = new CsvReader(new StreamReader(fileName), true);<br />
_csvReader.MissingFieldAction = MissingFieldAction.ReplaceByEmpty;
The above file fails to parse correctly. Can you offer any insight?
|
|
|
|
|
Hi, I use this code to read a lines from csv file:
<br />
using (CsvReader csv = new CsvReader(new StringReader(fFileName), false))<br />
<br />
int fieldCount = csv.FieldCount;<br />
string[] headers = csv.GetFieldHeaders();<br />
string cHeader = "";<br />
<br />
while (csv.ReadNextRecord())<br />
for (int i = 0; i < fieldCount; i++)<br />
{<br />
string s = csv[i];<br />
}<br />
}<br />
but don't work.
Using the csv file, called "test3.csv", do not come visualized the lines with the program demo.
It appears, of the dataGridView, only the written one "C:\\test3.csv". How I can do?
Thanks...
|
|
|
|
|
Please use the CachedCsvReader class for databinding purposes.
|
|
|
|
|
Hello,
I tried my luck to process a 25 column CSV file. Sadly without success. The run was interrupted because the program cannot agree that the mentioned dimension of "_fieldHeaderIndexes" in CsvReader.cs should be increased over value "10". The critical code is (line 1456 - 1462):
for (int i = 0; i < _fields.Length; i++)
{
_fieldHeaders[i] = _fields[i];
_fieldHeaderIndexes.Add(_fields[i], i);
}
What can I do to solve the problem?
The first lines of the CSV file are:
Company,Symbol,prop1,prop2,prop3,prop4,prop5,prop6,prop7,prop8,prop9,prop10,prop11,prop12,prop13,prop14,prop15,prop16,prop17,prop18,prop19,prop20,prop21,prop22,prop23
Pre Paid Legal Services,PPD,4496837,73.5,33.89,9.43,,,-14.44,14.44,-0.30,0.30,1.17,1.17,4377658,2.72,13267000,61200,14204958,$39.41,$559817393,NY,Services,Personal Services,234.89,2007-Jan
Holdrs Tr-Wireless,WMH,361996,69.6,,0.61,,,-6.98,6.98,4.38,4.38,3.46,3.46,370023,-2.17,,5200,1700000,$62.05,$105484999,AMX,,,,2007-Jan
Tasty Baking Company - Common Stock,TSTY,869314,68.2,,22.98,,,-10.78,10.78,1.56,1.56,-0.33,0.33,872018,-0.31,,12741,8215261,$9.00,$73937349,NAS,,,,2007-Jan
ZipRealty Inc - Common Stock,ZIPR,2294817,68.1,14.27,-13.91,,,-34.58,34.58,3.07,3.07,-1.34,1.34,2328538,-1.45,16080000,33686,21021021,$7.49,$157447442,NAS,Financial,Real Estate Development,-135.18,2007-Jan
Best regards
Michael
|
|
|
|
|
I am trying to implement the example above, including exception handling. Unfortunately I get an error on the following line:
csv.ParseError += new ParseErrorEventHandler(csv_ParseError);
Error:
Error 3 The type or namespace name 'ParseErrorEventHandler' could not be found (are you missing a using directive or an assembly reference?)
I am new to C# and would appreciate your help.
Regards,
Pieter
|
|
|
|
|
If you use .NET 2.0, the syntax is:
csv.ParseError += new EventHandler<ParseErrorEventArgs>(csv_ParseError);
or simply:
csv.ParseError += csv_ParseError;
If you use .NET 1.1 and you get this error, then please make sure you are referencing the CsvReader library (LumenWorks.Framework.IO.dll) correctly.
|
|
|
|
|
Thank you for your response.
The syntax error has now been resolved. Unfortunately I cannot seem to get the event handling working correctly: I would like to load a CSV file, e.g.:
a,b,c,d
a,b,
a,b,c,d
a,b,c,d
a,b,
a,b,c,d
a,b,c,d
It should then be able to count all the errors. In the case above it should report 2 lines that's not correct.
I have found a previous entry which seems to address a similar issue: http://www.codeproject.com/cs/database/CsvReader.asp?msg=1628796&searchkw=AdvanceToNextLine&sd=26+Dec+2004&ed=26+Mar+2007&stype=1#xx1628796xx, but I do not get the correct results.
I am really stuck and have tried everything.
Pieter
|
|
|
|
|
You should use MissingFieldAction property as follow:
csv.MissingFieldAction = MissingFieldAction.ReplaceByNull;
That way, when a field is null, that means it is missing.
|
|
|
|
|
Thank you for an excellent piece of code and kind help.
|
|
|
|
|
Can you give me some guidenance on how to adjust your code to allow it to read and write upper ascii values. Some of the Greek characters and other miscellaneous characters.?
Chuck
|
|
|
|
|
You need to specify the encoding of the stream. For example:
CsvReader csv = new CsvReader(new StreamReader(fileName, System.Text.Encoding.UTF8), true);
You might have to try other encodings until you get the exact one that you need to use.
|
|
|
|
|
I went with:
CsvReader csv = new CsvReader(new StreamReader(fileName, Encoding.GetEncoding(1252), true);
Once that was done your code is great.
Chuck
|
|
|
|
|
I downloaded the .NET 2.0 demo version, and it won't run because it is looking for a namespace called NUnit.
|
|
|
|
|
Either remove the test project from the solution or install the unit testing framework:
http://www.nunit.org/[^]
|
|
|
|
|
Just letting you know that you did a great job with this.
Cheers
My Blog
A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.
- -Lazarus Long
|
|
|
|
|
I have a comma-separated file, which looks like this:
ORIGIN,DESTINATION
JFK,LAX
ORD,MSP
PHL,FLL,kjhkj kjhkjh,eg,fhgf
NYC,LAX
The third line below the header is malformed in my opinion. When the CsvReader gets to it, it should throw an exception. I tried running it with both SupportsMultiline/hasHeaders set to true and false, but this doesn't change anything at all. Instead while iterating through my CSV file using ReadNextRecords I get this:
JFK,LAX (ok)
ORD,MSP (ok)
PHL,FLL (wrong!)
kjhkj kjhkjh,eg + System.ArgumentOutOfRangeException (OMG!)
Do you have any idea why it misbehaves so?
Here is a method in which I use your CachedCsvReader (I need to use the cached one, because I have to "rewind" the file when it ends.
protected override void OnDoWork(DoWorkEventArgs e)<br />
{<br />
CachedCsvReader csv =<br />
new CachedCsvReader(new StreamReader(AIRPORTS), true);<br />
<br />
csv.DefaultParseErrorAction = ParseErrorAction.AdvanceToNextLine;<br />
csv.MissingFieldAction = MissingFieldAction.ParseError;<br />
csv.SkipEmptyLines = true;<br />
csv.SupportsMultiline = false;<br />
<br />
while (true)<br />
{<br />
if (!CancellationPending)<br />
{<br />
lock (locker)<br />
{<br />
<br />
SwitchProxy();<br />
<br />
if (csv.EndOfStream)<br />
csv.MoveTo(-1);<br />
<br />
if (csv.ReadNextRecord())<br />
{<br />
<br />
origCity = csv[0];<br />
destCity = csv[1];<br />
rate++;<br />
errorCount = 0;<br />
<br />
Thread.Sleep(5000);<br />
ReportProgress(0);<br />
}<br />
<br />
if (paused)<br />
Monitor.Wait(locker);<br />
}<br />
}<br />
else<br />
{<br />
rate = 0;<br />
queries = 0;<br />
e.Cancel = true;<br />
return;<br />
}<br />
}<br />
}
I must say, I was expecting it to throw a MalformedCsvException when it comes to that line, because it evidently doesn't follow the header structure (if we state that comma is the delimiter).
|
|
|
|
|
Using the code below, I do not get the ArgumentOutOfRange exception you mentioned, which would only occur if you pass an index which is equal to or higher than the field count (determined by the first line) or record count.
string data = "ORIGIN,DESTINATION\nJFK,LAX\nORD,MSP\nPHL,FLL,kjhkj kjhkjh,eg,fhgf\nNYC,LAX";
using (CsvReader csv = new CsvReader(new System.IO.StringReader(data), true))
{
csv.DefaultParseErrorAction = ParseErrorAction.AdvanceToNextLine;
csv.MissingFieldAction = MissingFieldAction.ParseError;
csv.SkipEmptyLines = true;
csv.SupportsMultiline = false;
while (csv.ReadNextRecord())
{
for (int i = 0; i < csv.FieldCount; i++)
{
Console.Write(csv[i]);
if (i < csv.FieldCount - 1)
Console.Write(',');
}
Console.WriteLine();
}
}
Output:
JFK,LAX
ORD,MSP
PHL,FLL
kjhkj kjhkjh,eg
fhgf,
That is still bad though. I will post an update tonight with the correction of this bug, so the article should be updated before the end of this week.
|
|
|
|
|
Thank you very much, Sébastien. I am looking forward to your update.
|
|
|
|
|
Is there a bugfix coming at all?
|
|
|
|
|
Your issue needs some thinking before I commit a final update and I have other obligations at the moment. Should not be too long thought. This is the bad side of Open Source: you are not a customer
|
|
|
|
|
No problem. I didn't believe you when you told about releasing a fix "tonight", anyway. I know how software development looks from my own experience. I will wait for the update.
Oh, by the way. I am really surprised you've been supporting this lib for over two years now! It's awesome.
|
|
|
|
|
Thanks for your understanding
So, in your particular case (no multiline fields), you can replace the following:
(in ReadField)
if (initializing || index < _fieldCount - 1)
_nextFieldIndex = Math.Max(index + 1, _nextFieldIndex);
else
_nextFieldIndex = 0;
by
if (eol)
_nextFieldIndex = 0;
else
_nextFieldIndex = Math.Max(index + 1, _nextFieldIndex);
and
(in ReadNextRecord)
else if (_nextFieldIndex > 0 && _nextFieldIndex <= _fieldCount)
by
else if (_nextFieldIndex > 0)
For multiline fields, it is a bit more involved and I need to investigate before I post an update.
|
|
|
|
|
Btw, with the correction in previous message, you will not need to specify
csv.DefaultParseErrorAction = ParseErrorAction.AdvanceToNextLine;
csv.MissingFieldAction = MissingFieldAction.ParseError;
|
|
|
|
|
Hi,
I'm using the CSVReader class to import a list of customers into an application.
I've initialised the object like this:
CsvReader csv = new CsvReader(new StreamReader(data.CsvPath, System.Text.Encoding.Default), false, ';', '"', '\\', '#', true);
csv.SupportsMultiline = false;
From our customer I get a list like this:
"12345";"company";"lastname";"name";"title";"";"street";"zip";"city";"tel";"fax";"email";"country";
But in some cases, unescaped quotes are used in the company field by our customer and I have to deal with it.
For example:
"12345";"company "name"";"lastname";"name";"title";"";"street";"zip";"city";"tel";"fax";"email";"country";
So, if the CSVReader steps through this line, no exception is thrown. When the next record is fetched by ReadNextRecord (which is one without an error) the following exception is thrown:
LumenWorks.Framework.IO.Csv.MissingFieldCsvException: The CSV appears to be corrupt near record '29' field '1 at position '548'.
My problem is that the exception is not thrown on the recordset producing the error.
Any suggestions, how i can handle this without having my customer to change the automatically generated csv files he provides?
Thanks in advance!
|
|
|
|
|