|
Why do you need to Map and UnMap? Can't each thread Map the file, and then you synchronize access across the threads?
Regardless of whether memory mapped files are used or not you still need to handle thread synchronization, so I don't see what that has to do with it.
I thought MMF couldn't go beyond 2G, but could well be wrong on this.
What is in this 4G file???
BTW I haven't used MMF, but I am very interested in this technique and keep reading all I can find about it. I'll be very interested in your experience. Doesn't seem to me that many people are using MMF and it looks so simple and elegant, and fast.
Neville Franks, Author of ED for Windows. www.getsoft.com
|
|
|
|
|
The data is historic market data for 5 years. Depending on the number of symbols that will be there, this could be a large file. Now, the mapping will be done only on certain parts of the file, depending on the user query. There will be a small part of the file, that will be updated from real-time market data. I can maintain that part of the file always in memory and do thread sync and stuff. Actually I already have code that has a limit of 2 GB as you said (the largest that can be mapped now). But, if I change that to mapping parts of the file, then there could potentially be a number of map/unmap going on. Hence before I went ahead with this, I just wanted to know whether anyone here has idea about the overhead involved in this.
My logic was: Even if a map existed, if the data has not been queried before, it would not be in memory. So, the overhead is the actual creation of the map. But, when I unmap (the case being the data requested is not in the current map), I assume that the OS unloads the memory related to that part (although I am not sure - it could be delayed too.)
I was not planning to map and unmap. The process would be to maintain a map until there is a query that requires me to map another part.
Thomas
modified 29-Aug-18 21:01pm.
|
|
|
|
|
I don't know precisely what happens when you map/unmap but my guess is that for map nothing would happen, and for unmap any writes would be delayed. Why not write some test code and then profile it.
This is some info on MMF I've been collecting.
--------------------------------------------------------------------------------
Title: Tuesday, 20 August 2002 9:34:05 PM C/C++ Users Journal Web Site
Text: The Standard Librarian: File-Based Containers
Matt Austern
Looking for a way to create a file-based container? You might need to look outside
the Standard C++ library, and memory mapping may be the answer.
Comment: STL style container using MMF. Unix implementation.
From: http://www.cuj.com/experts/1907/austern.htm?topic=experts
--------------------------------------------------------------------------------
Title: Tuesday, 20 August 2002 9:30:39 PM Managing Memory-Mapped Files in Win32
Text: Managing Memory-Mapped Files in Win32
Comment: Good MS article on MMF
From:
http://msdn.microsoft.com/library/en-us/dngenlib/html/msdn_manamemo.asp?frame=true
File Mapping:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/file_mapping.asp
--------------------------------------------------------------------------------
Title: Sunday, 18 August 2002 7:40:45 PM SourceForge.net: Project Info -
Common C++ Libraries
Text: The Common C++ project has been merged with APE, and the combined project
now uses the Common C++ name. Common C++ offers a highly portable application
framework for threading, sockets, realtime network streaming, persistance, and file
access.
Comment: Includes MMF code
From: http://sourceforge.net/projects/cplusplus/
--------------------------------------------------------------------------------
Title: Sunday, 14 July 2002 9:10:26 AM Reliquary Class Library
Text: FileMaps use the operating system's caching mechanism to map a file to a
section of virtual memory. Applications access the memory, while the operating
system maintains the file.
Comment: Part of Reliquary Library
From:
http://www.fidnet.com/~bbreeden/reliquary/classlib/filemap.htm#FileMap_class
--------------------------------------------------------------------------------
Title: Sunday, 7 July 2002 9:36:54 AM System Abstraction Layer
Text: Mapped on memory file provides direct access to the file data using
virtual memory mechanism. That is the most efficient way to access file, because it
requires no context switching and data copying. This class provides all the methods
from os_file class and has one additional method get_mmap_addr()
Comment:
From: http://www.garret.ru/~knizhnik/SAL/ReadMe.htm#mmap_file
--------------------------------------------------------------------------------
Neville Franks, Author of ED for Windows. www.getsoft.com
|
|
|
|
|
I did a sample profiling program:
The results:
Scenario a: [map - update - unmap] in a loop - 60 us per iteration
Scenario b: map - [update in a loop] - unmap - 40 ns per update
If the update is done on two memory locations in the view more than 4 KB apart, the time in the first case goes up to 80 us.
To confirm this, I added a FlushViewofFile in the scenario b loop and the time was 11 ms!.
Now I have to draw my conclusions.
- mapping and unmapping are not that bad, if they are inevitable.
- if you can map the whole file to memory, then do it. it gives best performance
- Limit the number of FlushViewofFile calls to what is absolutely necessary.
Thomas
modified 29-Aug-18 21:01pm.
|
|
|
|
|
Hi Thomas, Thanks for that - very interesting, and good to boot. My main interest in MMF is in database code (BTree etc.) where instead of managing my own cache (which I do now), get rid of that and use MMF. This greatly simplifies my code and lets the OS cache look after caching instead of me. It also skips having to move data back and forth between my app<->cache<->disk file, which should also help improve performance, even if just by a small amount.
The main issue is whose cache provides optimal performance. I've heard and read that for some large database/file systems using your own caching can actually hinder performance. All very interesting stuff indeed.
Neville Franks, Author of ED for Windows. www.getsoft.com
|
|
|
|
|
Hi All,
I need to pass a parameter to a method which takes the value as LPCWSTR.But the data is in the simple character string char[10].How do I convert the data type by type casting.
Thanks
Abhishek.
Learning is a never ending process of Life.
|
|
|
|
|
Simple typecasting won't work. You can use:
USES_CONVERSION;
CallSomeMethodThatAcceptsWideString( A2W( pszSomeCharArray ) );
Wenn ist das Nunstück git und Slotermeyer? Ja! Beierhund das oder die Flipperwaldt gersput!
|
|
|
|
|
Hi all,
I am writing an application on Win98.
I have problems with my RegisterClass .
I get "0" value when I call RegisterClass method and "GetLastError" method give 0x0000078 (decimal 120) value.
This code has worked fine in another application.
Why I get such a strange behaviour?
Code follows as below:
BOOL
InitApplication( HANDLE hInstance )
/*++
This function initializes window data and registers the window class
--*/
{
WNDCLASS wc;
BOOL tmp = FALSE;
DWORD result;
// Fill in window class structure with parameters that describe the
// main window.
wc.style = 0; // Class style(s).
wc.lpfnWndProc = MainWndProc; // Function to retrieve messages for
// windows of this class.
wc.cbClsExtra = 0; // No per-class extra data.
wc.cbWndExtra = 0; // No window extra data
wc.hInstance = hInstance; // Application that owns the class.
wc.hCursor = LoadCursor(NULL, IDC_ARROW);
wc.hbrBackground = GetStockObject(WHITE_BRUSH);
wc.lpszMenuName = "CexmplMenu"; // Name of menu resource in .RC file.
wc.lpszClassName = "CexmplWClass"; // Name used in call to CreateWindow.
// Register the window class and return success/failure code.
//
tmp = RegisterClass(&wc);
result = GetLastError();
// MessageBox(NULL,
return (tmp);
}
Any comments?
regards
Saekth
|
|
|
|
|
The error you're being returned is ERROR_CALL_NOT_IMPLEMENTED. Are you by chance writing an NT service or something exotical like a device driver?
Joaquín M López Muñoz
Telefónica, Investigación y Desarrollo
|
|
|
|
|
hi there. i am new to this forum.
i was wondering if there was a way you could use C++ to retrieve an image off the web and store it in a file? For Instance, if i want a hurricane image off of weather.com then it would be neat if there was a C++ call or calls to allow you to grab that image(.jpeg or bitmap or whatever) off the web at any given point and store it in a folder or wherever.
Is this possible?  PLEASE HELP!!!!!!!! 
|
|
|
|
|
http://www.codeproject.com/internet/webgrab.asp
Papa
Murex Co.
while (TRUE)
Papa.WillLove ( Bebe ) ;
|
|
|
|
|
You can use WinInet calls (starting with InternetOpenUrl or HttpOpenRequest), or use URLDownloadToFile to have IE do it for you. (See this article[^] for more info)
--Mike--
Just released - RightClick-Encrypt v1.4 - Adds fast & easy file encryption to Explorer
My really out-of-date homepage
Sonork-100.19012 Acid_Helm
|
|
|
|
|
Sure, anything is possible. Internet Explorer, after all, is just C++ under the hood. That said, it's not a trivial task  If you're not against using MFC, take a look at CInternetSession, CHttpConnection, and CHttpFile. If the name of the JPG you're looking for constantly changes, you'll have to parse the containing HTML to figure it out every time. If the name stays the same, you're golden and only have to pull down that single file.
Ty
"The significant problems we face cannot be solved at the same level of thinking we were at when we created them." -Albert Einstein
|
|
|
|
|
hey thanks all to have replyed so far  .
well in fact this is my situation in more detail. i am looking to get the picture off the web of the current hurricane over cuba and heading into the Gulf of Mexico. The Picture from this website changes every 15 mins, coming from a satellite picture. so what i want to do is for my C++ program to be able to go out to this site and grab the NEW "frame" or updated frame of the picture. the name of the file doesnt change i dont think but of course where the hurricane is does. every 15 mins i want to store the pics in a folder and then i can use them later for whatever i want. if i have been unclear this here is the link where i am looking at.
http://www.goes.noaa.gov/browsh2.html
write back and thanks....again, this helps 
|
|
|
|
|
hey thanks. ill check out those links...if i just want to do this the first way how to i implement the InternetOpenUrl or HttpOpenRequest you spoke of?? i am a pretty good C++ programmer but this aspect of the language i have never tackled. thanks for your info. also i wrote to tyMatthews a more detailed write up of what i am doing. thanks again
|
|
|
|
|
hey thanks.
ill check out those links...if i just want to do this the first way how to i implement the InternetOpenUrl or HttpOpenRequest you spoke of??
i am a pretty good C++ programmer but this aspect of the language i have never tackled. thanks for your info. also i wrote to tyMatthews a more detailed write up of what i am doing. thanks again
|
|
|
|
|
This wont compile:
{
char * dataBaseName = "abc";
BSTR b_dataBaseName = SysAllocString (dataBaseName);
}
error is:
SysAllocString' : cannot convert parameter 1 from 'char *' to 'const unsigned short *'
Any help is appreciated.
Thanks,
ns
Thanks,
ns
|
|
|
|
|
The parameter must be a unicode string.
Jason Henderson
start page
articles
"If you are going through hell, keep going." - Sir Winston Churchill
|
|
|
|
|
SysAllocString (L"abc");
Papa
Murex Co.
while (TRUE)
Papa.WillLove ( Bebe ) ;
|
|
|
|
|
Thanks. Actually its coming from an editbox so
mEdit.GetWindowText(myTExt);
char* newString = new char[myText.GetLength()+1);
so how do I make newString into a unicode string or the L("abc") like you said?
Thanks,
ns
|
|
|
|
|
If you have a CString, then it has a built in member to give you a BSTR,
namely
AllocSysString (done that way to avoid/promote confusion, I guess).
The BSTR contains a copy of the original string, so changes to the CString won't affect the BSTR, and remember that at some stage, you'll need to call SysFreeString and have it destroy the BSTR.
Steve S
[This signature space available for rent]
|
|
|
|
|
If I understood you, I tried this:
CString abc = "anb";
BSTR b_dataBaseName = SysAllocString (abc);
}
but got:
: error C2664: 'SysAllocString' : cannot convert parameter 1 from 'class CString' to 'const unsigned short *'
Thanks,
ns
|
|
|
|
|
myText.AllocSysString() will return a new BSTR for you. At least it did in VC6.
|
|
|
|
|
Your method worked. It compiles! I was using SysAllocString (also exists). Now question is how to free this?
I didnt see a FreeSysString or SysFreeString matching the AllocSysString.....?
I was told that SysAllocString MUST be followed by SysFreeString, so I'm looking for this sort of matching function to go with AllocSysString...
Thanks,
ns
|
|
|
|
|
Use SysFreeString.
Jason Henderson
start page
articles
"If you are going through hell, keep going." - Sir Winston Churchill
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
