Click here to Skip to main content
16,006,709 members
Home / Discussions / Database
   

Database

 
AnswerRe: update same table in 3 different databases Pin
Ashfield10-Jun-08 21:02
Ashfield10-Jun-08 21:02 
GeneralRe: update same table in 3 different databases Pin
Dushan12310-Jun-08 23:47
Dushan12310-Jun-08 23:47 
GeneralRe: update same table in 3 different databases Pin
Ashfield11-Jun-08 3:02
Ashfield11-Jun-08 3:02 
GeneralRe: update same table in 3 different databases Pin
Dushan12311-Jun-08 3:30
Dushan12311-Jun-08 3:30 
QuestionMixing Select into and Create Pin
prithaa10-Jun-08 20:03
prithaa10-Jun-08 20:03 
AnswerRe: Mixing Select into and Create Pin
Ashfield10-Jun-08 20:59
Ashfield10-Jun-08 20:59 
GeneralRe: Mixing Select into and Create Pin
prithaa10-Jun-08 21:52
prithaa10-Jun-08 21:52 
QuestionLocal ADO.NET Performance Issue Pin
I don't want a display name10-Jun-08 11:01
I don't want a display name10-Jun-08 11:01 
I'm having difficulty identifying the best way to process a dataset I've been provided. It's a fairly large dataset (1.76m records) with two columns (pk1,pk2) in the primary key. e.g., {{1,a},{1,b},{1,c},{1,d},{2,a},{2,b},{3,a},{4,a},{4,b}}

What I need to do is read all the records for pk1 in the dataset {{1,1},{1,2},{1,3},{1,4}}, run some calculations on these, write the results to a different table, then move on to the next pk1. I've tried a couple methods, and run into problems in each. I'm wondering if anyone has any ideas for this.

I'm restricted using the local file system, and cannot install anything new locally: e.g., MS Access, text files (connecting through system.data.oledb), cannot install MySQL or SQLServer.

What I've done thus far, is:
- create a DataTable containing a list of distinct pk1 values [MainList]. {1,2,3,4}
- For each unique value in [MainList],
- Execute a sproc via OleDbDataAdapter.Fill method to populate a DataTable containing all records in the original dataset associated with the current pk1 {{1,a},{1,b},{1,c},{1,d}}
- Process this datatable to create the user's output
- Write the results to a separate set of tables in the same database.
This method takes an unacceptable amount of time, with 98% taken up by the .Fill method.

Next, I tried the same process, but using the OleDbDataReader in lieu of the .Fill(DataTable) process. This improves the speed slightly, but still not where we need it.

Running this file through sequentially, using a text file, and checking the current records key against the prior records key, we processed the entire file in about an hour.

We are currently running VB.NET 2005 with ADO.NET 2.0.

Also, are there any other options on databases available, provided we're not allowed to install something locally, and can only use the local system?

Any input would be appreciated.
AnswerRe: Local ADO.NET Performance Issue Pin
Mycroft Holmes10-Jun-08 16:19
professionalMycroft Holmes10-Jun-08 16:19 
GeneralRe: Local ADO.NET Performance Issue Pin
I don't want a display name11-Jun-08 3:05
I don't want a display name11-Jun-08 3:05 
GeneralRe: Local ADO.NET Performance Issue Pin
Mycroft Holmes11-Jun-08 14:05
professionalMycroft Holmes11-Jun-08 14:05 
NewsSQL Server 2008 Release Candidate Available to Public Pin
brucedkyle10-Jun-08 6:00
brucedkyle10-Jun-08 6:00 
QuestionRecover Oracle Connection in a Windows Service. Pin
jamorey10-Jun-08 4:21
jamorey10-Jun-08 4:21 
AnswerRe: Recover Oracle Connection in a Windows Service. Pin
Mycroft Holmes10-Jun-08 16:26
professionalMycroft Holmes10-Jun-08 16:26 
QuestionDate Between Problem Pin
Babu.R.K10-Jun-08 3:48
Babu.R.K10-Jun-08 3:48 
AnswerRe: Date Between Problem Pin
Ashfield10-Jun-08 3:59
Ashfield10-Jun-08 3:59 
AnswerRe: Date Between Problem Pin
Kjetil Svendsen11-Jun-08 3:15
Kjetil Svendsen11-Jun-08 3:15 
GeneralRe: Date Between Problem Pin
Ashfield11-Jun-08 5:01
Ashfield11-Jun-08 5:01 
AnswerRe: Date Between Problem Pin
Niraj_Silver26-Jun-08 1:10
Niraj_Silver26-Jun-08 1:10 
QuestionCan't read some cells in Excel [modified] Pin
gaby1192310-Jun-08 3:28
gaby1192310-Jun-08 3:28 
AnswerRe: Can't read some cells in Excel Pin
gaby119232-Jul-08 8:05
gaby119232-Jul-08 8:05 
Questionregistering a ".mdf" file in desktop engine Pin
asdf175310-Jun-08 3:05
asdf175310-Jun-08 3:05 
Questionproblem with output parameter Pin
Mr.Kode9-Jun-08 23:54
Mr.Kode9-Jun-08 23:54 
AnswerRe: problem with output parameter Pin
Mike Dimmick10-Jun-08 0:29
Mike Dimmick10-Jun-08 0:29 
GeneralRe: problem with output parameter Pin
Mr.Kode10-Jun-08 0:53
Mr.Kode10-Jun-08 0:53 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.