|
Hi all,
it's been a lot of learning already, but here's a point I can't figure out even with the help all the sources I reviewed by now. Maybe one of you can help me find my error?
In my little application (written in VB 2008 Express) I use Backgroundworkers. One of them is supposed to read / filter Outlook-Mails and put extracted customer data into datatable rows. The other one gets the rows and checks a DB for previous inquiries of that customer. Between both of them I put a buffer in order to avoid race conditions. I hope I expressed the basic concept well enough to be understandable - and although it's working I'd appreciate comments and ideas to make the code easier.
What doesn't work is: The RunWorkerCompleted-event in the class "Consumer" is not fired, which I tried to work around in the ProgressChanged-event, but that can't be it. Here's the code:
First I give you the buffer class in which I guess the error must be - suspecting some SyncLock or Monitor activities (which I took from a sample without fully understanding them):
Public Class Puffer
Private mBuffer As New List(Of DataRow) With {.capacity = 1}
Property Puffer() As DataRow
Get
SyncLock (Me)
If mBuffer.Count = 0 Then
MsgBox("Alle Puffer sind leer. " & Thread.CurrentThread.Name & " wartet.")
Monitor.Wait(Me)
End If
Dim readValue As DataRow = mBuffer(0)
MsgBox(String.Format("{0} liest Element '{1}'.", Thread.CurrentThread.Name, readValue.Item("Name")))
OutCounter += 1
mBuffer.Remove(mBuffer(0))
Monitor.Pulse(Me)
Return readValue
End SyncLock
End Get
Set(ByVal WriteValue As DataRow)
SyncLock (Me)
If mBuffer.Count = mBuffer.Capacity Then
MsgBox("Alle Puffer sind voll. " & Thread.CurrentThread.Name & " wartet.")
Monitor.Wait(Me)
End If
MsgBox(String.Format("{0} schreibt Element '{1}'.", Thread.CurrentThread.Name, WriteValue.Item("Name")))
mBuffer.Add(WriteValue)
InCounter += 1
Monitor.Pulse(Me)
End SyncLock
End Set
End Property ' Buffer
End Class
Producer class (shortened):
Public Class Producer
Dim WithEvents ProducerThread As New BackgroundWorker
Private Puffer As Puffer
Public Sub New(ByVal sharedObject As Puffer)
With ProducerThread
.WorkerReportsProgress = True
.WorkerSupportsCancellation = True
.RunWorkerAsync()
End With
Puffer = sharedObject
End Sub
Public Sub Produce(ByVal sender As Object, ByVal e As System.ComponentModel.DoWorkEventArgs) Handles ProducerThread.DoWork
Thread.CurrentThread.Name = "Produzent"
'Return a reference to the MAPI layer
Dim oApp As New Outlook.Application
Dim oNameSpace As Outlook.NameSpace = oApp.GetNamespace("MAPI")
....
Try
oNameSpace.Logon(Nothing, Nothing, False, False) ' TODO: Ggf ändern...
....
' Get Messages collection of Inbox and count them
Dim oItems As Outlook.Items = oInFolder.Items
Dim intTotal As Integer = oItems.Count
' Loop each unread message.
For i As Integer = 1 To oItems.Count
If ProducerThread.CancellationPending Then Exit Sub
If CurrentMedia <> "" Then
Dim CurrentRow As DataRow = MailsTable.NewRow()
CurrentRow("Absender") = Inquiry.ClientMail
....
' erst einmal in den Puffer
Puffer.Puffer = CurrentRow <----- write into the buffer
CurrentRow = Nothing
End If
oMsg = Nothing
Next
oItems = Nothing
Catch ex As System.Exception
MsgBox("Fehler in der Routine 'Produce'", MsgBoxStyle.Critical, String.Format("Klasse '{0}'", Me.GetType.Name))
End Try
End Sub
Private Sub ProducerThread_RunWorkerCompleted(ByVal sender As Object, ByVal e As System.ComponentModel.RunWorkerCompletedEventArgs) Handles ProducerThread.RunWorkerCompleted
'Thread.CurrentThread.Name = "Produzent"
frmProcessOutlAtt.btnRead.Text = "Lesen"
End Sub
End Class
Consumer Class (shortened):
Public Class Consumer
Dim WithEvents ConsumerThread As New BackgroundWorker
Private Puffer As Puffer
Private rowCounterValid As Integer
Private rowCounterInvalid As Integer
Public Sub New(ByVal sharedObject As Puffer)
With ConsumerThread
.WorkerReportsProgress = True
.WorkerSupportsCancellation = True
.RunWorkerAsync()
End With
Puffer = sharedObject
End Sub
Public Sub Consume(ByVal sender As Object, ByVal e As System.ComponentModel.DoWorkEventArgs) Handles ConsumerThread.DoWork
MailsTable.Clear()
Dim DA As m2prolapDataSetTableAdapters.AdressenTableAdapter = New m2prolapDataSetTableAdapters.AdressenTableAdapter
Try
If ConsumerThread.CancellationPending then Exit Sub
Dim myrow As DataRow = Puffer.Puffer <----- read from the buffer
While myrow IsNot Nothing
If DA.GetDataByNamesStreetsMail(myrow("Name"), myrow("Strasse"), myrow("Vorname"), myrow("Absender")).Count > 0 Then
rowCounterValid += 1
myrow("Antworten") = False
ConsumerThread.ReportProgress(1, myrow)
Else
rowCounterInvalid += 1
myrow("Antworten") = True
ConsumerThread.ReportProgress(0, myrow)
End If
myrow = Puffer.Puffer
End While
Catch ex As System.Exception
MsgBox("Fehler in der Routine 'Consume'", MsgBoxStyle.Critical, String.Format("Klasse '{0}'", Me.GetType.Name))
End Try
End Sub
Private Sub ConsumerThread_ProgressChanged(ByVal sender As Object, ByVal e As System.ComponentModel.ProgressChangedEventArgs) Handles ConsumerThread.ProgressChanged
Dim myRow As DataRow = e.UserState
Dim Marker As Boolean = CBool(e.ProgressPercentage)
MailsTable.Rows.Add(myRow)
With frmProcessOutlAtt
.DataGrid.DataSource = MailsTable
.DataGrid.AutoResizeColumns(DataGridViewAutoSizeColumnsMode.DisplayedCells)
.DataGrid.Refresh()
End With
'MessageBox.Show(String.Format("Datensatz '{0}' wurde ausgewertet.", myRow("Name")))
'OutCounter = MailsTable.Rows.Count
End Sub
'THIS ONE IS NOT FIRED AT ALL:
Private Sub ConsumerThread_RunWorkerCompleted(ByVal sender As Object, ByVal e As System.ComponentModel.RunWorkerCompletedEventArgs) Handles ConsumerThread.RunWorkerCompleted
Thread.CurrentThread.Name = "Konsument"
MsgBox(String.Format("{0} Datensätze von {1} verarbeitet." & vbCrLf & "{1} ist beendet.", _
OutCounter, Thread.CurrentThread.Name), MsgBoxStyle.Information)
End Sub
End Class
I really hope that one of you can find the time and examine what happens behind the borders of my knowledge here... As written before, I think that something about "SyncLock" or "Monitor" prevents the Consumers "RunWorkerCompleted" from getting fired. But as we all know there are so many trees sometimes.... maybe you just find a typing error
Thank you in advance
Mick
|
|
|
|
|
You can remove the Monitor lines - you don't need them.
From what I've read in your code, it looks like you can also change the SyncLocks. You may also want to cahnge your design a bit. You can have one thread adding items to a Queue collection, wrapped by SyncLocks on the Queue synch object. Then you can have the other thread pulling items out of the Queue to process them, again wrapping the calls to Queue in SyncLocks.
|
|
|
|
|
Thank you Dave for the brief review. I'll try your suggestions as soon as possible. But you didn't mention the 'RunWorkerCompleted' problem - could you see anything in my code that might cause that problem?
Thanks again
Mick
|
|
|
|
|
Get rid of the Monitors...
|
|
|
|
|
Following your first suggestion - to take out the Monitor lines - brought up some news.
First: It solved my previous problem, i.e. the terminal event of the counsumer class fires now.
Second: Unfortunately the whole code doesn't work anymore It seems that the "consumer" is running quicker than the "producer" and tries to retrieve a data row from the buffer before it's filled, resulting in zero rows processed (in my sample it's supposed to be 4 of them). But at least the "RunWorkerCompleted" event is processed . Still I guess I would have needed the Monitor lines, no?
I'm going to read a bit about the Queue you mentioned.
|
|
|
|
|
You have a situation where you're running two threads, expecting the Completed to be fired by the Consumer thread that processes the rows. This thread shouldn't stop, so long as the applicaiton is running, IMHO. It can't stop because it cannot know, in your current design, if there are any more records coming from the Producer. It's the Producer that should be raising the Completed event. Only it knows if there is anything left to process on it's end.
|
|
|
|
|
Thanks again, Dave! This I guess is an important step for my understanding.
So I understand that in my code the 'consumer' thread keeps waiting until it's purposly ended, e.g. with a call from another thread or function. And in case there are further rows fed into the buffer, it will keep processing them, right?
This would still be ok as long as a waiting thread doesn't eat up too much memory and/or performance. But first I'll switch and try the queue concept you recommended.
Have a nice day,
Mick
|
|
|
|
|
All it has to do is wake up every so often to see if there are records in the queue. If so, process them. If not, go back to sleep for a little while. You may also want to check not only the queue, but a flag that tells the thread it should terminate nicely.
|
|
|
|
|
Daves hint about queues lead me to a real nice solution for the producer/consumer issue I've been fighting with. A first test with some sample code works perfectly between different threads.
For those interested, here's the sample (translated from C# to VB.NET):
Imports System.Threading
Public Class Test
Private Shared queue As ProducerConsumer
Shared Sub Main()
queue = New ProducerConsumer()
CType(New Thread(New ThreadStart(AddressOf ConsumerJob)), Thread).Start()
Dim rng As New Random(0)
For i As Integer = 0 To 9
Console.WriteLine("Producing {0} in thread {1}", i, Thread.CurrentThread.ManagedThreadId.ToString)
queue.Produce(i)
Thread.Sleep(rng.Next(1000))
Next i
End Sub
Private Shared Sub ConsumerJob()
' Make sure we get a different random seed from the first thread
Dim rng As New Random(1)
' We happen to know we've only got 10 items to receive
For i As Integer = 0 To 9
Dim o As Object = queue.Consume()
Console.WriteLine(Constants.vbTab + Constants.vbTab + Constants.vbTab + Constants.vbTab & "Consuming {0} in thread {1}", o, Thread.CurrentThread.ManagedThreadId.ToString)
Thread.Sleep(rng.Next(1000))
Next i
End Sub
End Class
Public Class ProducerConsumer
Private ReadOnly listLock As Object = New Object()
Private queue As New Queue()
Public Sub Produce(ByVal o As Object)
SyncLock listLock
queue.Enqueue(o)
' We always need to pulse, even if the queue wasn't empty before. Otherwise, if we add several items
' in quick succession, we may only pulse once, waking a single thread up, even if there are multiple
' threads() waiting for items.
Monitor.Pulse(listLock)
End SyncLock
End Sub
Public Function Consume() As Object
SyncLock listLock
' If the queue is empty, wait for an item to be added. Note that this is a while loop, as we may
' be(pulsed)but not wake up before another thread has come in and consumed the newly added object.
' In that case, we'll have to wait for another pulse.
Do While queue.Count = 0
' This releases listLock, only reacquiring it after being woken up by a call to Pulse
Monitor.Wait(listLock)
Loop
Return queue.Dequeue()
End SyncLock
End Function
End Class An extra warning / quote from the author: "Note that using these methods can easily lead to deadlock - if thread A holds locks X and Y, and waits on Y, but thread B needs to acquire lock X before acquiring and then pulsing Y, thread B won't be able to do anything. Only the lock which is waited on is released, not all the locks the waiting thread owns. Usually you should ensure that prior to waiting, a thread only owns the lock it's going to wait on. Sometimes this isn't possible, but in those cases you should think extra carefully about how everything is going to work."
But for my purposes I seem to have a proper basis
Kind regards and thank you, Dave!
Mick
|
|
|
|
|
Hi all,
modified 28-Nov-12 8:07am.
|
|
|
|
|
simple
1. replace "c:\DB\Output1\outputDB.txt" to "c:\DB\Output1\outputDB.xls"
2. replace Space(x) function with chrw(9).
in the your above code.
Rajesh B --> A Poor Workman Blames His Tools <--
|
|
|
|
|
Thanks for the reply.
In the above case I read the database and wrote as excel.
I would like to read a text file and write it into excel file.
|
|
|
|
|
Read the data from the text file using Streamreader.,
write the data in manner of you want , with tab delimiter.
Rajesh B --> A Poor Workman Blames His Tools <--
|
|
|
|
|
Just to make sure I understand you correctly...
Do you want to pick up each line in the TextFile, and have it's contents put into a series of cells in XL where the spaces in the line are disposed of?
ie Number of children in one cell, number of balloons in next etc?
So that the cell contents are just the value as VALUE, not as TEXT?
Cos if so there are two methods, one is to just load the text into one cell and then split it via VBA in XL, or to pick up the text form the text file, scan it, apply each set of non space characters to a variable, then fire the variable into a cell in xl, continue until end of line, next line etc.
Which is the one you would like?
------------------------------------
"Possessions make you poor, wealth is measurable only in experience."
Sun Tzu 621BC
|
|
|
|
|
I want each line in the text file has to be entered in the excel based on the fields.
If the excel file field is Parent, the first field from the text file has to be entered in that cell and so on.
I want the text to be entered and not the count/number.
|
|
|
|
|
I need help with the following code. I am building a custom control. I am storing controls in arrays as I need more rows. When I type into the second text box as soon as i run the program the text is typed into the first text box. Please help. I thank you in advance.
Imports System.Drawing
Imports System.Windows.Forms
Public Class UserControl1
Dim arrText1(1), arrtext2(1) As TextBox
Public Sub SetTextBoxes()
Dim text1 As New TextBox()
With text1
.Name = "text1"
.Left = 8
.Font = New Font(.Font.Name, 14, FontStyle.Regular, .Font.Unit)
.Size = New Point(80, 30)
Me.Controls.Add(text1)
arrText1(0) = text1
End With
Dim text2 As New TextBox()
With text2
.Name = "text2"
.Left = 100
.Font = New Font(.Font.Name, 14, FontStyle.Regular, .Font.Unit)
.Size = New Point(80, 30)
Me.Controls.Add(text2)
arrtext2(0) = text2
End With
End Sub
Private Sub ucGrid_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
SetTextBoxes()
End Sub
End Class
|
|
|
|
|
The code you posted here appears to work just fine.
Whatever is causing the problem must be somewhere else.
What are the arrays for?
My advice is free, and you may get what you paid for.
|
|
|
|
|
Thanks for the fast reply. It happens only the first time you run the code and type into the second text box. Otherwise it works fine. I am using the arrays in order to have more rows in my control (sort of control array of vb6). This code works fine with my regular windows form in vb.net 2008. But it does not work in my user control form in vb.net 2008. Please help.
modified on Monday, May 18, 2009 7:48 AM
|
|
|
|
|
So apparently the error is caused by some difference between the standard windows form and your user control form.
I suppose you'll need to check what differences there are between the two.
My advice is free, and you may get what you paid for.
|
|
|
|
|
Thank you Johan. I will try to figure out the difference between both. Thanks once again.
|
|
|
|
|
Hi Johan, this is for your information. I have placed two text boxes in the user control (Design time) without any coding. Still the text i type in the second text box is getting typed in the first text box and the cursor in the second text box is moving as per the characters typed (giving out blank spaces in the first text box). Any idea what the problem could be. Do you think i should reinstall vb.net ?!
|
|
|
|
|
Hi guys,
I am importing a file and need to import the file onto a db.The problem is i am reading the file line by line and different lines may contain data that is to be inserted as one record in the db.The tag of the line helps me to know what to insert where. The problem is some lines don't have a tag and should be read as lines from the previous tag..The qsn is when do i insert the line into the db.
Here is a sample of the file.
-----------------------------------------------------------------------
:61:0807040704D370,13NTRFTT 3676//FAAM8186027286
/OCMT/EUR400,/
:86:210?00PAYMENT BY ELECTRONIC MEDIA?1097171?20/FAG:GONZALO HERNANDE
Z CIRI?21ZA/FAG:033529423T/FAG:P.O. ?22BOX 30437/FBB:EUR370,13/FU
K?23:1,/FCC:FR?32MRS KONAN IRENE/ MRS MAGGIE?33NAIROBI
:61:0807040704D700,NCHK6810021334//400877010900EUR
/OCMT/EUR700,/
:86:002?00GVC-LANGTEXT NICHT VORHANDE?1095186?20SCHECKAUSSTELLER?3066
010075?31173757?32SCHECK-NR. 0006810021334?34000
-----------------------------------------------------------
Tag (:61 contains the details of each transaction.But as you can see this transaction is contained into two lines
Tag (:86 contains info of a the owner. But as you can see this tag here contains two lines.Some records have up to 5 lines .
Any ideas will be highly appreciated.
|
|
|
|
|
The actual problem appears to be that the lines that belong together, are mixed up.
You could just create a datatable (or a temporary database table), and insert all lines separately, including the tag numbers.
Then sort the table by tag numbers, concatenate the rows where necessary, and then insert the concatenated values into the production database table.
My advice is free, and you may get what you paid for.
|
|
|
|
|
Something like (with possible syntax errors ):
using sr as stringreader = new stringreader("MyFile)
dim inLine as string = sr.readline() 'assumes first line has tag
while sr.peek >=0
dim s as string = sr.readline
if s.startwith(":")
'write inLine as db record
inLine = s
else
inline = inLine & s
end if
end while
end using
Regards
David R
---------------------------------------------------------------
"Every program eventually becomes rococo, and then rubble." - Alan Perlis
|
|
|
|
|
Thanks for the reply,its been very helpful.
That should handle the first transaction block with a (":").I have a query though.See every tag(":")has its own transaction eg :61: and :86: are totally different transactions and this two should be stored in different tables in the db.What happens when i reads the next(":") and want to store it on a different table.How can i map this line to the tag that appeared just before it?
Regards
|
|
|
|
|