0

I have solved the problem. Switching the target platform from "any" to x64 will allow me to successfully run this program (allocating 64 GB, which is double my physical memory limit):

byte[][] test = new byte[64][];

for (int i = 0; i < 64; i++)
  test[i] = new byte[1000000000];

I'm still confused about why this issue seemingly popped out of nowhere, but there it is.

Edit: I have tried the large address aware flag, and also the gc allow large objects. It breaks the second application where i = 2 (in other words, when allocating the third set of 500 million bools, or 1.5 billion total bools, or the first 1.5 GB of RAM). This is no where near the limit of 3GB that comes as default with .NET 4.5. If you read carefully, the first application worked just fine until a day ago, when all of a sudden, it started throwing an out of memory exception when memory usage rose above 1 GB.

Edit1: Most seem to be missing the point of my question. The first application ran just fine (for a month) until yesterday, when it started throwing out of memory exceptions at appx. 1GB of memory usage. What could have changed between two days ago (when it ran fine) and yesterday, when it would no longer run?

Edit2: This question is not a duplicate. You have to realize that my application was working fine for a month until yesterday. The only change to my machine at all was upgrading FLStudio from FL 11 to FL 12. I will uninstall FL completely and see if that changes anything, but I highly doubt that it will.

To start, some background info: The problem is not a lack of memory. I have 32 GB of RAM, and am running a 64 bit version of windows. I wish the problem was that simple, if it were, I wouldn't have had to come here for help.

So I built this application:

uint arraySize = 2000000000;
uint squareRoot = Convert.ToUInt32(Math.Sqrt(arraySize));
bool[] sieveContainer = new bool[arraySize];
ulong sum = 0;

for (int baseMultiple = 2; baseMultiple < squareRoot; baseMultiple++)
{
    if (sieveContainer[baseMultiple] == false)
    {
        for (int multiples = baseMultiple * 2; multiples < arraySize; multiples += baseMultiple)
            sieveContainer[multiples] = true;
    }
}

for (int j = 2; j < arraySize; j++)
{
    if (sieveContainer[j] == false)
        sum++;
}

This worked perfectly alright and was happy to consume about 2GB of RAM. Up until yesterday. As soon as it breaks 1GB (sometimes even .75GB) I get an out of memory exception. I started looking for potential solutions, and found that I didn't really have enough information about my problem to find one. So I built this test application: 

byte[][] test = new byte[16][];

for (int i = 0; i < 16; i++)
    test[i] = new byte[500000000];

Once again, at 1GB of RAM being allocated, I get a System.OutOfMemoryException error. No other information is provided. I have tried everything I can think of/find from reputable sources on the internet, and haven't been able to find a solution. (Solutions yielding no result were immediately reversed).

Why can I only use 1GB of memory for an entire application, when yesterday I could use +2GB? I haven't messed around with any settings in Visual Studio, nor have I changed anything in my machine. I ran MalwareBytes, which found nothing. I'm out of ideas, do you have any?

superstewie
  • 71
  • 1
  • 6
  • 6
    You are aware that, in the example code you provided, this `new bool[arraySize]` allocates almost 2 GB, and this `new byte[500000000]` allocates almost 500 MB in a loop that is performed 16 times (i.e. it tries to allocate a total of almost 8 GB) ? – Alex Jun 07 '15 at 01:51
  • 2
    What does this question have to do with Visual Studio? – John Saunders Jun 07 '15 at 02:40
  • John Saunders I think it has to do with some runtime problems or VS settings that have been inadvertently changed or some similar problem. If you'll read carefully, this application ran fine for months (I figure I ran it 100+ times). Until yesterday, when it broke and will no longer run. – superstewie Jun 07 '15 at 02:46
  • Why would changes to "VS settings" affect the running application? Were you running it from within VS (maybe by F5)? – John Saunders Jun 07 '15 at 02:49
  • 1
    Allocating large byte arrays is a horrible horrible idea. You are asking for memory fragmentation problems. Try to keep contiguous allocations to less that 60K or so (for LOH issues) and for fragmentation avoidance. – Yishai Galatzer Jun 07 '15 at 03:00
  • You aren't listening. Let us know if that changes. – David Heffernan Jun 07 '15 at 04:54
  • 1
    The main problem you have appears to be a lack of appreciation of the constraints of a 32 bit process, and the issue of fragmentation. If you understood these issues well you would not be attempting to code the way you do. That's the starting point. – David Heffernan Jun 07 '15 at 05:00
  • When you installed FLStudio did you also install a newer version of .net too? For example, installing .NET 4.5 changes the GC behavior for programs built in .NET 4.0. – Scott Chamberlain Jun 07 '15 at 06:05
  • Please don't think the solution is to go back to an earlier .net runtime!! That is quite plausibly why you observed a behaviour change, but your problems run deeper. – David Heffernan Jun 07 '15 at 07:20
  • In response to "you aren't listening" comments: correct me if I'm wrong, but the only way to prime sieve is one ideologically contiguous block of memory. This would allow you to use a jagged array, with many smaller arrays, to avoid memory fragmentation issues. The other thing to think about is heck yeah I didn't know about these limits until you pointed them out, causing me start researching them. I've been coding for all of 5 weeks. But none of your answers explained why this problem appeared out of nowhere in an application that previously worked 100% of the time I ran it. – superstewie Jun 07 '15 at 11:20

0 Answers0