|
Thanx for posting this code!
Your csvreader works like a charm – but I run into trouble with the CachedCsvReader: Im not sure if I’m using the correct version. Just to make sure we’re talking about the same version: I’ve used CachedCsvReader with the following code fragment in the ReadNextRecord(bool,bool) method
<br />
if (base.CurrentRecordIndex > -1)<br />
CopyCurrentRecordTo(record);<br />
else<br />
{<br />
MoveTo(0);<br />
CopyCurrentRecordTo(record);<br />
MoveTo(-1);<br />
}<br />
<br />
_records.Add(record);<br />
<br />
if (!onlyReadHeaders)<br />
_currentRecordIndex++;<br />
The following code on a csv file with 4 records will produce a correct output:
<br />
private void ReadCSVFile(string file){<br />
using (CachedCsvReader csv = new CachedCsvReader( <br />
new StreamReader(file, Encoding.GetEncoding("ISO-8859-1")), true, ';', '\'', '\\', '#', true))<br />
{ <br />
foreach (string[] data2 in csv)<br />
{<br />
DoSomeOutPut(data2);<br />
} <br />
<br />
}<br />
}<br />
output:
<br />
Record1<br />
Record2 <br />
Record3<br />
Record4<br />
When you add a call to csv.GetFieldHeaders() bevor the enumeration the output looks like:
<br />
Record1<br />
Record1 <br />
Record2<br />
Record3<br />
So the first record gets enumerated twice and the last is lost.
What’s the best way to fix this?
Thank in advance.
|
|
|
|
|
Yes, this looks indeed like a bug. I will look into that tomorrow and fix it. Thank you for reporting it!
|
|
|
|
|
If you need it, leave a comment under this thread. Thank you!
|
|
|
|
|
Hi, Sébastien,
Yes, I need it. Our code is .NET 1.1 throughout and we rely on CsvReader in some of our critical applications. I may rewrite it at some point down the road, but it would be very useful if you continued to support the 1.1 version for at least another year or so.
Thank you!
Simon
|
|
|
|
|
Well, as long as there is no bug, I am not going to change anything
|
|
|
|
|
Hi, Sébastien,
It's ok if you do not change anything in the code, but could you please place the sources back online? The references to the 1.1 versions are gone now. (I am talking about the article to which this forum thread is attached.)
Thanks!
Simon
|
|
|
|
|
Hi!
I tried to use the Custom Error Handling Scenario but the first row/record is missing. I have the updated LumenWorks.Framework.IO.dll 3.0.0.0.
Header and the rest of the records are fine.
I need you help. Thanks!
|
|
|
|
|
The first record is usually the header. Have you tried reading the CSV without headers ? It is an argument in the constructor. If that is not the cause of your problem, could you post your code?
|
|
|
|
|
Made my work a lot easier
|
|
|
|
|
Your welcome! Thank you for taking time to leave a comment
|
|
|
|
|
Hi,
Could you please cleanup the ZIP-file CsvReader20_src.zip?
It contains three copies of the source and three different versions of CachedCsvReader.cs (the rest of the files are the same).
I used the tool CloneSpy to cleanup the source myself, first removing all bin directories. Finally I kept the version of CachedCsvReader.cs that had the latest modification date/time. I hope that is the correct one.
modified 27-May-21 21:01pm.
|
|
|
|
|
Hum, I don't know what happened, I double-checked my files before uploading them?! Anyway, it will be fixed soon, sorry about that. Funny you are the first to report it, seems other users are more complacent
|
|
|
|
|
Sorry about the delay, I am still waiting for the article to be updated.
|
|
|
|
|
Article has been updated.
|
|
|
|
|
Hello,
Thanks for the great parser and for keeping it up-to-date!
I have a small 80GB file the I'm working on but the header record is wrapped in quotes which generates an error. Here is the header record and first record:
"Household_Member1","Address","City_State_Zip","Area_Code_and_Phone","Phone_Area_Code","Phone_Number_without_Area_Code","City","State","ZIP","ZIP_+_4","ZIP_+_4_+_DPBC","Delivery_Point_Bar_Code","Carrier_Route","County","MSA","Household_Member_First1","Household_Member_Last1","Household_Member_Gender1","Household_Member_Age1","Household_Member2","Household_Member_First2","Household_Member_Last2","Household_Member_Gender2","Household_Member_Age2","Household_Member3","Household_Member_First3","Household_Member_Last3","Household_Member_Gender3","Household_Member_Age3","Household_Member4","Household_Member_First4","Household_Member_Last4","Household_Member_Gender4","Household_Member_Age4","Household_Member5","Household_Member_First5","Household_Member_Last5","Household_Member_Gender5","Household_Member_Age5","Household_Member6","Household_Member_First6","Household_Member_Last6","Household_Member_Gender6","Household_Member_Age6","Household_Member7","Household_Member_First7","Household_Member_Last7","Household_Member_Gender7","Household_Member_Age7","Household_Member8","Household_Member_First8","Household_Member_Last8","Household_Member_Gender8","Household_Member_Age8","Household_Member9","Household_Member_First9","Household_Member_Last9","Household_Member_Gender9","Household_Member_Age9","Household_Member10","Household_Member_First10","Household_Member_Last10","Household_Member_Gender10","Household_Member_Age10","Residence_Type","Home_Age","Est_Home_Value","Est_Income","Own/Rent","Length_of_Residence","Mortgage_Age","Mortgage_Count","Mortgage_Finance_Type","Mortgage_Loan_to_Value_Ratio","Mortgage_Loan_Type","Personal_Details1","Personal_Details2","Personal_Details3","Personal_Details4","Personal_Details5","Mail_Order_Buyer1","Mail_Order_Buyer2","Mail_Order_Buyer3","Mail_Order_Buyer4","Mail_Order_Buyer5","Mail_Order_Buyer6","Mail_Order_Buyer7","Mail_Order_Buyer8","Mail_Order_Buyer9","Mail_Order_Buyer10","Mail_Order_Buyer11","Mail_Order_Buyer12","Mail_Order_Buyer13","Mail_Order_Buyer14","Mail_Order_Buyer15","Mail_Order_Buyer16","Mail_Order_Buyer17","Mail_Order_Buyer18","Mail_Order_Buyer19","Mail_Order_Buyer20","Mail_Order_Buyer21","Mail_Order_Buyer22","Mail_Order_Buyer23","Mail_Order_Buyer24","Donor_Type1","Donor_Type2","Donor_Type3","Donor_Type4","Donor_Type5","timezone","latitude","longitude"
"Rose M Modarelli",""2822 Bears Den CT","Youngstown, OH 44511-1214","","","","Youngstown","OH","44511","44511-1214","44511-1214-223","223","C090","Mahoning","Youngstown-Warren, OH","Rose","Modarelli","F","65 and older","Dina M Modarelli","Dina","Modarelli","F","25 to 34","Dominic G Modarelli","Dominic","Modarelli","M","25 to 34","Frank J Modarelli","Frank","Modarelli","M","45 to 54","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","Single family dwelling","","","","Home Owner","26 to 30 years","26 to 30 years","Unknown","Unknown","0.01 to 0.09","Unknown","Pool","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","EST",80.696899999999999,41.069099999999999
Works fine if I take the quotes off the header record. I get a lot of these files in so if I can make the parser handle it that would be better than appending a new header to these file since they are so large.
I know you are a busy guy so if you can point me in the general direction I'll try to make the change myself. Thanks again for your time.
Chad
|
|
|
|
|
Thanks for your comment
First of all, knowing what is the exact error would help me ... That said, I can see there is an extra " in ""2822 Bears Den CT" which may cause your error. Hope this help!
|
|
|
|
|
Sebastien,
Thanks for pointing that out. That's what I get for programming so late. Things blend together at that hour! Work fine. Thanks again!
Off to parser 124 million records (90 fields). Your product is solid, I've use this before on 200 million+ with way more fields. No problems.
Great piece of code! Keep up the good work!
Chad
|
|
|
|
|
That reader was meant for multi-gigabytes files. Glad to hear I am not the only one that is dealing with these monsters :p
|
|
|
|
|
Whew I finally figured it out.. thanks for this nice CSVReader..
|
|
|
|
|
I mean id GridView not GridPanel.. sorry
|
|
|
|
|
Im only getting this items in my gridpanel.
System.String[];
how do i use this CSVReader properly using ASP.Net2.0?
Thankyou!
|
|
|
|
|
Do you have any example code/xaml of binding this in wpf to a control such as ListView or ContentControl?
|
|
|
|
|
Sorry for late reply, but for some reason, I did not get notified via email of your post.
Using the class CachedCsvReader, you can use standard binding mechanisms, be it WinForms or WPF. If you do not know how to bind in WPF, take a look at the doc on the Binding class.
|
|
|
|
|
HI there,
I think this is a very usefull great tool!
But I experienced problems, when the first row of a csv file has less columns than the following rows. In this case it only parses as many values per row as given in the first row.
is there a way of setting the number of columns to be read?
I didn't find any...
Thanks to all
ans most of all to the author!!!
Anja
There are only 10 types of people in the world: Those who understand binary, and those who don't...
|
|
|
|
|
Sorry for late reply, but for some reason, I did not get notified via email of your post.
I am sorry, this is one of the few limitations of this CSV reader. I built it unaware that it is so hard for a multi-billion company to follow a spec of 5 lines.
|
|
|
|
|