C / C++ / MFC

valikac19-Sep-02 10:06

valikac

19-Sep-02 10:06

The overhead includes performance loss and possible synchronization problems. The program could suffer a performance loss since more than one thread could be reallocation larger virtual memory.

As the threads access the mapped files, you will need to implement a synchronization algorithm. Depending on the number of processes the program creates, you will need a CRITICAL_SECTION or a mutex.

Can you implement a solution such that the program will allocate memory before creating the threads? Thus, you will not have to allocate more virtual memory as often.

Kuphryn

Neville Franks19-Sep-02 11:27

Neville Franks

19-Sep-02 11:27

Why do you need to Map and UnMap? Can't each thread Map the file, and then you synchronize access across the threads?

Regardless of whether memory mapped files are used or not you still need to handle thread synchronization, so I don't see what that has to do with it.

I thought MMF couldn't go beyond 2G, but could well be wrong on this.

What is in this 4G file???

BTW I haven't used MMF, but I am very interested in this technique and keep reading all I can find about it. I'll be very interested in your experience. Doesn't seem to me that many people are using MMF and it looks so simple and elegant, and fast.

Neville Franks, Author of ED for Windows. www.getsoft.com

User 988519-Sep-02 15:35

User 9885

19-Sep-02 15:35

The data is historic market data for 5 years. Depending on the number of symbols that will be there, this could be a large file. Now, the mapping will be done only on certain parts of the file, depending on the user query. There will be a small part of the file, that will be updated from real-time market data. I can maintain that part of the file always in memory and do thread sync and stuff. Actually I already have code that has a limit of 2 GB as you said (the largest that can be mapped now). But, if I change that to mapping parts of the file, then there could potentially be a number of map/unmap going on. Hence before I went ahead with this, I just wanted to know whether anyone here has idea about the overhead involved in this.

My logic was: Even if a map existed, if the data has not been queried before, it would not be in memory. So, the overhead is the actual creation of the map. But, when I unmap (the case being the data requested is not in the current map), I assume that the OS unloads the memory related to that part (although I am not sure - it could be delayed too.)

I was not planning to map and unmap. The process would be to maintain a map until there is a query that requires me to map another part.

Thomas

modified 29-Aug-18 21:01pm.

Neville Franks20-Sep-02 11:07

Neville Franks

20-Sep-02 11:07

I don't know precisely what happens when you map/unmap but my guess is that for map nothing would happen, and for unmap any writes would be delayed. Why not write some test code and then profile it.

This is some info on MMF I've been collecting.

--------------------------------------------------------------------------------
Title: Tuesday, 20 August 2002 9:34:05 PM C/C++ Users Journal Web Site
Text: The Standard Librarian: File-Based Containers
Matt Austern
Looking for a way to create a file-based container? You might need to look outside
the Standard C++ library, and memory mapping may be the answer.
Comment: STL style container using MMF. Unix implementation.
From: http://www.cuj.com/experts/1907/austern.htm?topic=experts

--------------------------------------------------------------------------------
Title: Tuesday, 20 August 2002 9:30:39 PM Managing Memory-Mapped Files in Win32
Text: Managing Memory-Mapped Files in Win32
Comment: Good MS article on MMF
From:
http://msdn.microsoft.com/library/en-us/dngenlib/html/msdn_manamemo.asp?frame=true

File Mapping:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/file_mapping.asp
--------------------------------------------------------------------------------
Title: Sunday, 18 August 2002 7:40:45 PM SourceForge.net: Project Info -
Common C++ Libraries
Text: The Common C++ project has been merged with APE, and the combined project
now uses the Common C++ name. Common C++ offers a highly portable application
framework for threading, sockets, realtime network streaming, persistance, and file
access.
Comment: Includes MMF code
From: http://sourceforge.net/projects/cplusplus/

--------------------------------------------------------------------------------
Title: Sunday, 14 July 2002 9:10:26 AM Reliquary Class Library
Text: FileMaps use the operating system's caching mechanism to map a file to a
section of virtual memory. Applications access the memory, while the operating
system maintains the file.
Comment: Part of Reliquary Library
From:
http://www.fidnet.com/~bbreeden/reliquary/classlib/filemap.htm#FileMap_class

--------------------------------------------------------------------------------
Title: Sunday, 7 July 2002 9:36:54 AM System Abstraction Layer
Text: Mapped on memory file provides direct access to the file data using
virtual memory mechanism. That is the most efficient way to access file, because it
requires no context switching and data copying. This class provides all the methods
from os_file class and has one additional method get_mmap_addr()
Comment:
From: http://www.garret.ru/~knizhnik/SAL/ReadMe.htm#mmap_file

--------------------------------------------------------------------------------

Neville Franks, Author of ED for Windows. www.getsoft.com

User 988520-Sep-02 12:11

User 9885

20-Sep-02 12:11

I did a sample profiling program:

The results:

Scenario a: [map - update - unmap] in a loop - 60 us per iteration
Scenario b: map - [update in a loop] - unmap - 40 ns per update

If the update is done on two memory locations in the view more than 4 KB apart, the time in the first case goes up to 80 us.

To confirm this, I added a FlushViewofFile in the scenario b loop and the time was 11 ms!.

Now I have to draw my conclusions.

- mapping and unmapping are not that bad, if they are inevitable.
- if you can map the whole file to memory, then do it. it gives best performance
- Limit the number of FlushViewofFile calls to what is absolutely necessary.

Thomas

modified 29-Aug-18 21:01pm.

Assigning character string to LPCWSTR

Neville Franks20-Sep-02 12:23

Neville Franks

20-Sep-02 12:23

Hi Thomas, Thanks for that - very interesting, and good to boot. My main interest in MMF is in database code (BTree etc.) where instead of managing my own cache (which I do now), get rid of that and use MMF. This greatly simplifies my code and lets the OS cache look after caching instead of me. It also skips having to move data back and forth between my app<->cache<->disk file, which should also help improve performance, even if just by a small amount.

The main issue is whose cache provides optimal performance. I've heard and read that for some large database/file systems using your own caching can actually hinder performance. All very interesting stuff indeed.

Neville Franks, Author of ED for Windows. www.getsoft.com

abhinarulkar19-Sep-02 6:32

abhinarulkar

19-Sep-02 6:32

Hi All,
I need to pass a parameter to a method which takes the value as LPCWSTR.But the data is in the simple character string char[10].How do I convert the data type by type casting.
Thanks
Abhishek.

Learning is a never ending process of Life.

Re: Assigning character string to LPCWSTR

dabs19-Sep-02 6:40

dabs

19-Sep-02 6:40

Simple typecasting won't work. You can use:

USES_CONVERSION;

CallSomeMethodThatAcceptsWideString( A2W( pszSomeCharArray ) );

Wenn ist das Nunstück git und Slotermeyer? Ja! Beierhund das oder die Flipperwaldt gersput!

RegisterClass problem

saekth19-Sep-02 6:30

saekth

19-Sep-02 6:30

Hi all,

I am writing an application on Win98.

I have problems with my RegisterClass .
I get "0" value when I call RegisterClass method and "GetLastError" method give 0x0000078 (decimal 120) value.
This code has worked fine in another application.
Why I get such a strange behaviour?

Code follows as below:

BOOL
InitApplication( HANDLE hInstance )
/*++
This function initializes window data and registers the window class
--*/
{
WNDCLASS wc;
BOOL tmp = FALSE;
DWORD result;
// Fill in window class structure with parameters that describe the
// main window.

wc.style = 0; // Class style(s).
wc.lpfnWndProc = MainWndProc; // Function to retrieve messages for
// windows of this class.
wc.cbClsExtra = 0; // No per-class extra data.
wc.cbWndExtra = 0; // No window extra data
wc.hInstance = hInstance; // Application that owns the class.
wc.hCursor = LoadCursor(NULL, IDC_ARROW);
wc.hbrBackground = GetStockObject(WHITE_BRUSH);
wc.lpszMenuName = "CexmplMenu"; // Name of menu resource in .RC file.
wc.lpszClassName = "CexmplWClass"; // Name used in call to CreateWindow.

// Register the window class and return success/failure code.
//
tmp = RegisterClass(&wc);
result = GetLastError();
// MessageBox(NULL,

return (tmp);
}

Any comments?

regards

Saekth

Re: RegisterClass problem

Joaquín M López Muñoz19-Sep-02 9:14

Joaquín M López Muñoz

19-Sep-02 9:14

The error you're being returned is ERROR_CALL_NOT_IMPLEMENTED. Are you by chance writing an NT service or something exotical like a device driver?

Joaquín M López Muñoz
Telefónica, Investigación y Desarrollo

C++ And Getting Images Off The Web

OpenGL19-Sep-02 5:24

19-Sep-02 5:24

hi there. i am new to this forum.

i was wondering if there was a way you could use C++ to retrieve an image off the web and store it in a file? For Instance, if i want a hurricane image off of weather.com then it would be neat if there was a C++ call or calls to allow you to grab that image(.jpeg or bitmap or whatever) off the web at any given point and store it in a folder or wherever.

Is this possible? Unsure | :~

PLEASE HELP!!!!!!!! Cry | :((

jmkhael19-Sep-02 5:38

jmkhael

19-Sep-02 5:38

http://www.codeproject.com/internet/webgrab.asp

Papa
Murex Co.

while (TRUE)
Papa.WillLove ( Bebe ) ;

Michael Dunn19-Sep-02 5:36

Michael Dunn

19-Sep-02 5:36

You can use WinInet calls (starting with InternetOpenUrl or HttpOpenRequest), or use URLDownloadToFile to have IE do it for you. (See this article[^] for more info)

--Mike--
Just released - RightClick-Encrypt v1.4 - Adds fast & easy file encryption to Explorer
My really out-of-date homepage
Sonork-100.19012 Acid_Helm

TyMatthews19-Sep-02 5:48

TyMatthews

19-Sep-02 5:48

Sure, anything is possible. Internet Explorer, after all, is just C++ under the hood. That said, it's not a trivial task Smile | :)

If you're not against using MFC, take a look at CInternetSession, CHttpConnection, and CHttpFile. If the name of the JPG you're looking for constantly changes, you'll have to parse the containing HTML to figure it out every time. If the name stays the same, you're golden and only have to pull down that single file.

"The significant problems we face cannot be solved at the same level of thinking we were at when we created them." -Albert Einstein

OpenGL19-Sep-02 6:47

19-Sep-02 6:47

hey thanks all to have replyed so far Poke tongue | ;-P

.

well in fact this is my situation in more detail. i am looking to get the picture off the web of the current hurricane over cuba and heading into the Gulf of Mexico. The Picture from this website changes every 15 mins, coming from a satellite picture. so what i want to do is for my C++ program to be able to go out to this site and grab the NEW "frame" or updated frame of the picture. the name of the file doesnt change i dont think but of course where the hurricane is does. every 15 mins i want to store the pics in a folder and then i can use them later for whatever i want. if i have been unclear this here is the link where i am looking at.

http://www.goes.noaa.gov/browsh2.html

write back and thanks....again, this helps Big Grin | :-D

OpenGL19-Sep-02 6:50

19-Sep-02 6:50

hey thanks. ill check out those links...if i just want to do this the first way how to i implement the InternetOpenUrl or HttpOpenRequest you spoke of?? i am a pretty good C++ programmer but this aspect of the language i have never tackled. thanks for your info. also i wrote to tyMatthews a more detailed write up of what i am doing. thanks again

OpenGL19-Sep-02 8:53

SysAllocString doesnt compile

19-Sep-02 8:53

hey thanks.

ill check out those links...if i just want to do this the first way how to i implement the InternetOpenUrl or HttpOpenRequest you spoke of??

i am a pretty good C++ programmer but this aspect of the language i have never tackled. thanks for your info. also i wrote to tyMatthews a more detailed write up of what i am doing. thanks again

ns19-Sep-02 5:18

19-Sep-02 5:18

This wont compile:

{


	char *	dataBaseName = "abc";
	BSTR b_dataBaseName = SysAllocString (dataBaseName);	
}

error is:

SysAllocString' : cannot convert parameter 1 from 'char *' to 'const unsigned short *'

Any help is appreciated.
Thanks,
ns

Thanks,
ns

Jason Henderson19-Sep-02 5:26

Jason Henderson

19-Sep-02 5:26

The parameter must be a unicode string.

Jason Henderson
_{start page}
^articles
^{"If you are going through hell, keep going." - Sir Winston Churchill}

jmkhael19-Sep-02 5:29

jmkhael

19-Sep-02 5:29

SysAllocString (L"abc");

Papa
Murex Co.

while (TRUE)
Papa.WillLove ( Bebe ) ;

ns19-Sep-02 6:09

19-Sep-02 6:09

Thanks. Actually its coming from an editbox so

mEdit.GetWindowText(myTExt);
 //convert CString to char*

 char* newString = new char[myText.GetLength()+1);

so how do I make newString into a unicode string or the L("abc") like you said?

Thanks,
ns