|
Ok, there are probably several ways to solve this, but first let me clear up a misconception:
The amount of physical memory available to your computer does not limit the amount of memory available to your application. Therefore, forcing a library to not work on physical memory - if that were possible - wouldn't change a thing! It is the OS that decides where allocated objects reside, and, yes, that does include the hard disk. The OS already moves objects from physical memory to the hard disk if it is running low on physical memory. And it does that at runtime.
Your actual problem is that you are running out of virtual memory. Virtual memory is an abstract address space that the OS grants every application. Depending on your Windows version and settings, that address space may encompass anything between 2 and 3 GB at most, even if your machine has 16 GB of physical RAM.
There are two probable causes for your problem:
1. The amount of data you have to deal with is simply more than this library can handle.
If this is the problem, then you should consider ways how to partition your data into smaller chunks, and adapt your operations to process that data bit by bit. Loading a 4 GB file into memory simply does not work!
2. You may not be using the library as intended: many library functions create objects and return them as results. The intention is that you call appropriate destruction functions once you are finished with those objects. If you don't do that, the amount of memory you need will continually increase over time until you run out of memory. (and you may think it's the library that is the problem because the library does the allocation
You should check the library documentation to see which of the functions create objects and expect you to release those objecs again, or call corresponding functions that clean them up properly. Also check your algorithms to see whether they 'lock' these objects unnecessarily, or for too long.
|
|
|
|
|
Thank you for your advice. Yes, maybe I can do some cleanup work.
|
|
|
|
|
That's the best place to start.
Make sure you have no memory leaks.
If there is some reason that you need gigabytes of data in memory to do some heavy complex processing (perhaps DNA analysis, simulations etc...,) then upgrade your hardware and go 64 bit.
|
|
|
|
|
|
Stefan_Lang wrote: 2. The OS will also move memory from and to hard disk as needed in order to keep
the entire system running smoothly. Repeatedly. At runtime! Again, you cannot
influence that short of rewriting the OS
I am rather certain that all desktop OSes provide access to some very raw memory management APIs. Without that there are whole classifications of applications that could not be written.
As an example of one API for windows, I presume there are others, the following insures that memory is not written to the hard drive.
http://msdn.microsoft.com/en-us/library/aa366895%28VS.85%29.aspx[^]
|
|
|
|
|
Quote: I am rather certain that all desktop OSes provide access to some very raw memory management APIs.
yes, i can't agree more. But I don't find what i need.
|
|
|
|
|
If your problem is that your application fails because over time this library keeps consuming memory then the solution is to remove the library from your application. You can do that as follows.
1. Create an executable that wraps the library.
2. Provide a communication idiom: files, stdio or sockets. That provides the necessary API for your business needs (basically a proxy for what you are already doing.)
3. In your actual application use WinProcess (or whatever the method is called) to execute 1 and manage it.
4. Use the communication from 2 from your application in place of the straight to library calls you are currently using.
Other variations on the above depend on your needs
- Create multiple instance
- Stop each instance after X (1 or more) uses
- Have min/max instances and use a locked queue to send requests for processing
- Detect failures, restart and retry a request.
- Detect excessive memory usage, terminate and restart.
|
|
|
|
|
Thank you for the specific suggestions. i will have a try.
|
|
|
|
|
I want to convert a binary file (.dll) to a base64 string and store it in my application, then write it to the hard disk using the stored base64 string but i am failing miserably. any help is appreciated thanks.
edit: basically, exactly what this site does: http://www.motobit.com/util/base64-decoder-encoder.asp[^]
|
|
|
|
|
base64 conversion is actually simple, you shouldn't fail. However you may also find ready-to-use implemantations, for instance, a quick Google search gave me: "Encoding and decoding base64 with C++"[^].
Veni, vidi, vici.
|
|
|
|
|
Member 8417954 wrote: but i am failing miserably. any help is appreciated thanks. It's impossible to help unless you explain exactly what the failure is. You might also like to explain exactly what problem you are trying to solve, as this seems a rather pointless exercise.
Use the best guess
|
|
|
|
|
CString ftitle;
fscanf_s(f_ptr_seed_inf,"%s\n",ftitle[ifl].GetBuffer(MAX_PATH));
ftitle[ifl].ReleaseBuffer();
I got error message:
Unhandled exception at 0x776015de in MSW_Vib_Model.exe: 0x00000000: The operation completed successfully.
Please help
|
|
|
|
|
Try taking out the [ifl] from the ftitle object. I don't think you can call GetBuffer() on a character returned from the [] operator.
CString ftitle;
fscanf_s(f_ptr_seed_inf,"%s\n",ftitle.GetBuffer(MAX_PATH));
ftitle.ReleaseBuffer();
|
|
|
|
|
|
mrby123 wrote: fscanf_s(f_ptr_seed_inf,"%s\n",ftitle[ifl].GetBuffer(MAX_PATH)); How does this even compile? I would have expected a C2228 error.
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Show me a community that obeys the Ten Commandments and I'll show you a less crowded prison system." - Anonymous
|
|
|
|
|
As Richard said, I've already answered this question.
You need to change the fscanf_s line as follows -
fscanf_s(f_ptr_seed_inf,"%s\n",ftitle[ifl].GetBuffer(MAX_PATH), MAX_PATH);
|
|
|
|
|
I have read a lot about abstract class, but still have questions, as I have not really used in my c++ code practically.
What is the use of abstract class ? Any sample I can take a look at ?
Thanks.
|
|
|
|
|
|
AdvancedCPP wrote: What is the use of abstract class ?
class Shape
{
public:
Shape( int nWidth, int nHeight )
{
m_nWidth = nWidth;
m_nHeight = nHeight;
}
virtual double getArea() = 0;
protected:
int m_nWidth;
int m_nHeight;
};
class Circle : public Shape
{
public:
Circle( int nWidth, int nHeight ) : Shape(nWidth, nHeight) {};
double getArea()
{
int nRadius = m_nWidth / 2;
return 3.1415962 * nRadius * nRadius;
}
};
class Square : public Shape
{
public:
Square( int nWidth, int nHeight ) : Shape(nWidth, nHeight) {};
double getArea() { return m_nWidth * m_nHeight; }
};
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Show me a community that obeys the Ten Commandments and I'll show you a less crowded prison system." - Anonymous
|
|
|
|
|
One of the points of an abstract class is you'd never create one (e.g. shape in example) - you can't create a 'shape' (what would it look like?) it has to be a circle or a square (as in the example) or some other real thing. But you do create objects that inherit from (use the code in) an abstract class. So the advantage is you only have to write the abstract class code (shape in the example) once, no matter how many different actual concrete classes which inherit the class, you can create such as square, circle etc all which inherit from it.
|
|
|
|
|
AdvancedCPP wrote: I have not really used in my c++ code practically
You have either to change that or your nickname.
Veni, vidi, vici.
|
|
|
|
|
I am creating an application that is multithreaded and all the threads are accessing the same shared memory. The data coming into the memory should be processed in sequential order. Like if 1,2,3,4,5 comes in the memory then it should be processed in same order.
I have used critical section around my memory, but I still get into a race condition where data is not grabbed by the thread in sequential order.
Am I missing anything else to prevent threads from race condition?
Thanks.
ACpp
|
|
|
|
|
Yes, you are.
Google 'multi thread patterns'.
It is a long path, but when you arrive, you're likely to feel it was worth it.
Pablo.
"Accident: An inevitable occurrence due to the action of immutable natural laws." (Ambrose Bierce, circa 1899).
|
|
|
|
|
Can I run two command line instructions on a file in a VS2008 project? I can set each command line instruction up as a rule in the .rules file (see below) but can only select one rule or the other not both. From the input files properties in VS2008 Project View I can select GraphVizSVGRule or GraphVizPNGRule . I want to create both a .svg file and a .png file from the input .gv file (GraphViz)
="1.0"="utf-8"
<VisualStudioToolFile
Name="GraphViz Rule File"
Version="8.00"
>
<Rules>
<CustomBuildRule
Name="GraphViz Dot to SVG Rule"
DisplayName="GraphVizSVGRule"
CommandLine="dot.exe -T svg "$(InputDir)\$(InputName).gv" -o "$(InputDir)\images\$(InputName).svg""
Outputs="$(InputDir)\images\$(InputName) .svg"
FileExtensions="*.gv"
ExecutionDescription="builds svg file"
>
<Properties>
</Properties>
</CustomBuildRule>
<CustomBuildRule
Name="GraphViz Dot to PNG Rule"
DisplayName="GraphVizPNGRule"
CommandLine="dot.exe -T png "$(InputDir)\$(InputName).gv" -o "$(InputDir)\images\$(InputName).png""
Outputs="$(InputDir)\images\$(InputName) .png"
FileExtensions="*.gv"
ExecutionDescription="builds png file"
>
<Properties>
</Properties>
</CustomBuildRule>
</Rules>
</VisualStudioToolFile>
|
|
|
|
|
The only method I have found to run more than one is to create a command script that accepts parameters, so you can pass in the file and directory names.
Use the best guess
|
|
|
|