I need to build an application that loads a 200, 000 line
(or more) text file and lets the user interactively search
through it in memory, generating sub-lists of those 200, 000
lines with various search criteria.
I'm just wondering if it's wishful thinking to hope that
a standard list box can handle that many lines, and be
fast enough to traverse. If not, I wonder what other solutions or suggestions might there be.
Has anyone seen any applications that sound similar in
functionality to this?
All the string searching going on makes me think to use Perl,
but I don't know Perl at all. Also, I'm probably wrong, but
it's a command line only tool, isn't it?
Sounds like an excellent candidate for using a DATABASE. Like SQL Server. That's what these programs do for a living....
If you really need to do this yourself, you should probably create a series of memory structures to house the data and create indexes on the various fields. The MFC/OO approach would be to create objects similar to the following:
class CommandObj
{
public:
CString m_cCommandTime;
CString m_cCommandName;
WORD m_wParam1;
WORD m_wParam2;
WORD m_wParam3;
};
class CommandDB
{
BOOL CheckMatch(CommandObj *pCmdObj, LPCSTR cpCommand, WORD *wpParam1, WORD *wpParam2, WORD *wpParam3);
public:
CObList m_oCommands; // list of Command objects
CMapStringToOb m_oCmdNameIndex; // index of cmd name to a CObList of Command objects that have this cmd name
CMapWordToOb m_oParam1Index; // index of param1 values to a CObList of Command objects that have this valuevirtual ~CommandDB(); // clean up our listsvoid AddCommandObj(CommandObj *pCmdObj);
BOOL SearchCommands(CObList *pOutputResults, LPCSTR cpCommand, WORD *wpParam1, WORD *wpParam2, WORD *wpParam3);
};
void CommandDB::AddCommandObj(CommandObj *pCmdObj)
{
CObList *pIdxList = NULL;
// update our indexes...if (!m_oCmdNameIndex.Lookup(pCmdObj->m_cCommandName,pIdxList))
{
pIdxList = new CObList;
m_oCmdNameIndex.SetAt(pCmdObj->m_cCommandName,pIdxList);
}
pIdxList->AddTail(pCmdObj);
if (!m_oParam1Index.Lookup(pCmdObj->m_wParam1,pIdxList))
{
pIdxList = new CObList;
m_oParam1Index.SetAt(pCmdObj->m_wParam1,pIdxList);
}
pIdxList->AddTail(pCmdObj);
// and add to "table"
m_oCommands.AddTail(pCmdObj);
}
BOOL CommandDB::SearchCommands(CObList *pOutputResults, LPCSTR cpCommand, WORD *wpParam1, WORD *wpParam2, WORD *wpParam3)
{
BOOL bFound = TRUE;
CommandObj *pCmdObj;
CObList *pKeyList = NULL;
if (cpCommand != NULL && *cpCommand)
m_oCmdNameIndex.Lookup(cpCommand,pKeyList);
elseif (wpParam1 != NULL)
m_oParam1Index.Lookup(*wpParam1,pKeyList);
else
pKeyList = &m_oCommands; // table scan time...// scan keyset for matches
POSITION pPos = pKeyList->GetHeadPosition();
while (pPos)
{
pCmdObj = (CommandObj *)pKeyList->GetNext(pPos);
if (CheckMatch(pCmdObj,cpCommand,wpParam1,wpParam2,wpParam3))
{
bFound = TRUE;
pOutputResults->AddTail(pCmdObj);
}
}
return bFound;
}
BOOL CommandDB::CheckMatch(CommandObj *pCmdObj, LPCSTR cpCommand, WORD *wpParam1, WORD *wpParam2, WORD *wpParam3)
{
// you can extend this to do "like" clauses/pattern matching, etc...if (cpCommand && *cpCommand)
{
if (pCmdObj->Compare(cpCommand))
return FALSE;
}
if (wpParam1)
{
if (*wpParam1 != pCmdObj->m_wParam1)
return FALSE;
}
if (wpParam2)
{
if (*wpParam2 != pCmdObj->m_wParam2)
return FALSE;
}
if (wpParam3)
{
if (*wpParam3 != pCmdObj->m_wParam3)
return FALSE;
}
return TRUE;
}
This is like a little "hardcoded" isam file...only in memory.
Is it possible to draw several shapes directly on a window during a loop without having any flicker? I have tried to use the ValidateRect and InvalidateRect function, but I can’t seem to make them prevent flicker, maybe I am missing something. How do you prevent flicker when drawing directly on a window?
One way is to use a memory device context (search codeproject for CMemDC), which is essentially a bitmap in memory. Instead of drawing directly to the screen, you draw everything to the memory DC, and then as the last step in the drawing process, you BitBlt the memory DC onto the real DC. This means that if the same part of the DC is drawn multiple times during the drawing process, you won't see any flicker.
But shouldn't it be possible to use ValidateRect or some other function to prevent an graphic area from updating to the screen? So that you can draw stuff on that area and update it when your done, sort of like when you use BeginPaint and EndPaint?
Aidman wrote: But shouldn't it be possible to use ValidateRect or some other function to prevent an graphic area from updating to the screen?
Yes, but the part you don't validate will still flicker if you draw over the same part multiple times or if you don't handle WM_ERASEBKGND to prevent the background being erased.
It's almost always better to use a memory DC, combined with selective invalidating like you suggested.
Ryan
Being little and getting pushed around by big guys all my life I guess I compensate by pushing electrons and holes around. What a bully I am, but I do enjoy making subatomic particles hop at my bidding - Roger Wright (2nd April 2003, The Lounge)
Punctuality is only a virtue for those who aren't smart enough to think of good excuses for being late - John Nichol "Point Of Impact"
John M. Drescher wrote: YES! This is very important to avoid filcker even if you use a memory dc.
Absolutely! I probably didn't make that clear enough did I
Ryan
Being little and getting pushed around by big guys all my life I guess I compensate by pushing electrons and holes around. What a bully I am, but I do enjoy making subatomic particles hop at my bidding - Roger Wright (2nd April 2003, The Lounge)
Punctuality is only a virtue for those who aren't smart enough to think of good excuses for being late - John Nichol "Point Of Impact"
I need to play a video file in .avi file format I am writing a DirectX win32 program so I do not want to use the control that ships with VC++ as I need full control.
also the AVI file is compressed using a custom codec.how can I extract the frames although it is compressed.
Hi. Recently due to my newbie design, I got stuck with all my windows as dialogs. Now I want to create a modal help dialog that has CHTMLView in it. I successfully did this with WebBrowser ActiveX object, but it requires external file (which I dont like), since it doesnt have LoadFromResource interface. Now most of the examples shown have Doc/View architecture which makes it simple to replace CChildView with CHTMLView. But almost none is inside a CDialog. The hints like creating CFrameWnd inside a CDialog sounds good but I couldn't make it work. Can anyone help / post an article on this? Any solutions will do. Thanks!!!
I'm looking for what I should use for a collection class which holds small objects of my own creation (all of the one class type.) Trouble is that the collection is expected to have up to 500000 objects in it.
What would be recommended for a collection class that minimizes the searching and retrieval time for so many objects in the collection?
It depends on how you want to access them. If you need to do lookups based on some key value, and the values do not need to be sorted, then you can use a hashmap, as it has constant time insertion, deletion, and retrieval.
If you need them sorted, then you could use a balanced tree, as that will have logN insertion/deletion/retrieval, and allows traversing the tree in order (keeping them sorted).
If you don't care how they are sorted, and don't need to do lookups, then you could use a simple linked list, or even an array if you know at the start how many items there will be.
If you can provide some more details with how you plan to use it, then it will be easier to recommend a specific collection to use.