Click here to Skip to main content
16,004,977 members
Home / Discussions / C#
   

C#

 
GeneralRe: What is the best way to do this? Pin
turbosupramk330-Oct-13 2:30
turbosupramk330-Oct-13 2:30 
GeneralRe: What is the best way to do this? Pin
Pete O'Hanlon30-Oct-13 3:57
mvePete O'Hanlon30-Oct-13 3:57 
GeneralRe: What is the best way to do this? Pin
turbosupramk330-Oct-13 4:20
turbosupramk330-Oct-13 4:20 
QuestionTFS custom control deployment Pin
Merklynn29-Oct-13 3:23
Merklynn29-Oct-13 3:23 
QuestionSQLGeometry as clip path? Pin
SledgeHammer0128-Oct-13 19:49
SledgeHammer0128-Oct-13 19:49 
AnswerRe: SQLGeometry as clip path? Pin
Pete O'Hanlon29-Oct-13 2:25
mvePete O'Hanlon29-Oct-13 2:25 
QuestionReading and filtering 20 m records Pin
nitin_ion28-Oct-13 18:17
nitin_ion28-Oct-13 18:17 
AnswerRe: Reading and filtering 20 m records Pin
Mycroft Holmes28-Oct-13 20:40
professionalMycroft Holmes28-Oct-13 20:40 
With that volume you are going to suffer but I would suggest using an ELT approach and an SSIS package

Extract - you have done
Load - I would BCP or bulk copy the lot into a staging table in SQL Server the staging data should exactly reflect your csv and all fields should be varchar.
Transform - write a stored proc that does the transforms (delete, match, validate formats, dedupe, copy to production table).

NEVER do your transforms in code, SQL is designed to chew through bulk data doing the transforms in c# is just wrong!
Never underestimate the power of human stupidity
RAH

GeneralRe: Reading and filtering 20 m records Pin
nitin_ion28-Oct-13 22:01
nitin_ion28-Oct-13 22:01 
GeneralRe: Reading and filtering 20 m records Pin
Mycroft Holmes28-Oct-13 22:24
professionalMycroft Holmes28-Oct-13 22:24 
GeneralRe: Reading and filtering 20 m records Pin
nitin_ion28-Oct-13 23:02
nitin_ion28-Oct-13 23:02 
GeneralRe: Reading and filtering 20 m records Pin
Mycroft Holmes29-Oct-13 2:03
professionalMycroft Holmes29-Oct-13 2:03 
GeneralRe: Reading and filtering 20 m records Pin
nitin_ion29-Oct-13 18:29
nitin_ion29-Oct-13 18:29 
GeneralRe: Reading and filtering 20 m records Pin
Mycroft Holmes29-Oct-13 19:54
professionalMycroft Holmes29-Oct-13 19:54 
AnswerRe: Reading and filtering 20 m records Pin
V.29-Oct-13 0:46
professionalV.29-Oct-13 0:46 
GeneralRe: Reading and filtering 20 m records Pin
Mycroft Holmes29-Oct-13 2:05
professionalMycroft Holmes29-Oct-13 2:05 
QuestionAutoMapper Error - Missing type map configuration or unsupported mapping Pin
Kevin Marois28-Oct-13 15:26
professionalKevin Marois28-Oct-13 15:26 
AnswerRe: AutoMapper Error - Missing type map configuration or unsupported mapping Pin
CodeBlack29-Oct-13 3:14
professionalCodeBlack29-Oct-13 3:14 
QuestionAccessing users folder. Pin
elficodegeek28-Oct-13 13:00
elficodegeek28-Oct-13 13:00 
AnswerRe: Accessing users folder. Pin
Richard Andrew x6428-Oct-13 13:07
professionalRichard Andrew x6428-Oct-13 13:07 
GeneralRe: Accessing users folder. Pin
elficodegeek30-Oct-13 13:10
elficodegeek30-Oct-13 13:10 
AnswerRe: Accessing users folder. Pin
Bernhard Hiller29-Oct-13 21:49
Bernhard Hiller29-Oct-13 21:49 
GeneralRe: Accessing users folder. Pin
elficodegeek30-Oct-13 13:01
elficodegeek30-Oct-13 13:01 
GeneralRe: Accessing users folder. Pin
Bernhard Hiller30-Oct-13 22:19
Bernhard Hiller30-Oct-13 22:19 
GeneralRe: Accessing users folder. Pin
elficodegeek1-Nov-13 13:06
elficodegeek1-Nov-13 13:06 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.