Memory Allocation and System Limits

We want to hear from you! Please send us your FEEDBACK.

The following article may contain actual software programs in source code form. This source code is made available for developers to use as needed, pursuant to the terms and conditions of this license.
 

Overview

Machines with large amounts of memory and disk are becoming very common. Many 32-bit programs are starting to reach the limits of the environment. Understanding some of the basic memory allocations made by the system on behalf of a user process can be useful. The following addresses the limits of the system, along with some information on how address space is allocated and used.
 

Ultra[tm] Workstations and Memory Allocation

In general, on the Ultra[tm] workstation series of processors, a 32-bit application can address four gigabytes of virtual memory space. A 64-bit application can address up to the size of the machines virtual memory, theoretically into terabytes of space. The memory address space of a process is composed of many different parts and pieces. Following is a summary of memory allocations which the developer may have some control over, and how the different allocations fill the process address space.

pmap

% sleep 60 &
[1] 5893
% pmap 5893
5893:   sleep 60
00010000      8K read/exec         /usr/bin/sleep
00020000      8K read/write/exec   /usr/bin/sleep
00022000      8K read/write/exec     [ heap ]
FF280000    672K read/exec         /usr/lib/libc.so.1
FF338000     32K read/write/exec   /usr/lib/libc.so.1
FF370000     16K read/exec         /usr/platform/sun4u/lib/libc_psr.so.1
FF390000      8K read/exec         /usr/lib/libdl.so.1
FF3A0000      8K read/write/exec     [ anon ]
FF3B0000    136K read/exec         /usr/lib/ld.so.1
FF3E2000      8K read/write/exec   /usr/lib/ld.so.1
FFBEE000      8K read/write/exec     [ stack ]
 total      912K

The argument to pmap is the process ID of the sleep process. That is usually returned by the shell when a process is executed in the background, else it can be found by using the ps command in a terminal window. The total four gigabyte address range runs from 0x00000000 to 0xffffffff. In the output above, you can see that a couple pages of the executable mapped in at 0x00010000 and 0x00020000. These are pages from the sleep binary. There is a small heap segment and then various system libraries mapped in. This program has only 912K mapped in.

Stack and ulimit

Following are some brief examples on the Solaris 8 OE with 32-bit executables. These examples show situations that developers may run into, and how to work with the memory limitations.

In a "normal" program, memory is used from both the stack and the heap. In general, stack allocations are local variables which have been allocated. Stack allocations should be kept to a minimum due to the default Solaris OE stack limit of eight megabytes. The default allocations can be seen using the ulimit command:

% ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes) 8192
coredump(blocks) unlimited
nofiles(descriptors) 256
memory(kbytes) unlimited

Here, you can see the limitation of the stack at eight megabytes. Note that the ulimit command is a built-in shell, so the output and syntax may vary between shells. To increase the stack size for a given shell, the "-s" argument to ulimit can be used.

% ulimit -s 16384
% ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes) 16384
coredump(blocks) unlimited
nofiles(descriptors) 256
memory(kbytes) unlimited

The stack size limit has been increased to 16 megabytes.

The primary usage of stack is for local variables. For example, the following code simply allocates a one megabyte array in main, sleeps 10 seconds, then allocates a one megabyte array in a function, sleeps again and then exits.

/*
 *  usestack.c:  Example program which uses stack space by allocating local
 *               variables.
 *          
 *               cc usestack.c -o usestack
 */ 

#include <stdio.h>


main()
{

    /*  Allocate a 1 meg char array  */
    char ch[1024 * 1024];

    touchmem(ch, 1024 * 1024);

    printf("In main\n");
    sleep(10);

    f1();

}

f1()
{

    /*  Allocate a 1 meg char array  */
    char ch3[1024 * 1024];

    touchmem(ch3, 1024 * 1024);

    printf("In f1\n");

    sleep(10);

}

/*
 *  touchmem touches each page a few times to force allocation.
 */
touchmem(char *start, int size)
{
    int i;

    for (i = 0; i < size; i += 1024) {

        start[i] = 'x';
    }

}

Following is a sample run and output snippets from pmaps of this application.

% cc usestack.c -o usestack
% usestack &
In main
% pmap 630
.
.
.
FFAEE000   1032K read/write/exec     [ stack ]
.
.
In f1
% pmap 630
.
.
.

FF9EE000   2056K read/write/exec     [ stack ]
.
.
.

In the example run, the message "In main" is printed. At this point the stack usage is just over one megabyte at 1032k. When the f1() function is called the second one megabyte allocation is made, we can see that the memory usage for stack is up to two megabytes. Given this example and the default Solaris OE stack size limitation, we can see why it is best to keep local variable allocations low.

A problem with limited stack space is that the program can fail in strange ways. Quite often, an unexpected SIGSEGV with a deep stack trace could point to a stack overflow situation. Heavy use of local variables can also cause stack overflows with shallow stacks.

Heap and malloc

The heap contains most memory allocations and global variable allocations. Following is an example program which puts out two heap allocations. Both allocations are one megabyte. One allocation is a global character array (ch) and the other is a malloc().

/*
 *  useheap.c:  Example program which uses heap space by allocating 
 *              a global variable and a dynamic allocation.
 *
 *               cc useheap.c -o useheap
 */

char ch[1024 * 1024];

main ()
{

  char *ch2;

  ch2 = (char *) malloc (1024 * 1024);

  sleep (100);

}

Here is a sample run and pmap output from the useheap program:

% cc useheap.c -o useheap
% useheap &
% pmap 7376
.
.
.
00022000   2056K read/write/exec     [ heap ]
.
.
.

This output shows two megabytes taken out of the heap. One megabyte is the global allocation and the other is from the malloc().

After a malloc, there can be a corresponding free which frees the memory for the application to reuse. Note that the free does not give the memory back to the system; it only allows future mallocs to reuse the allocated space. So, if memory is malloc()'ed and free()'ed, you will not see a decrease in the heap size. If there is a need for memory which can be freed to the system, mmap() can be used. There is also libmapmalloc which uses mmap() as a backend to malloc().

The previous examples are on 32-bit executables. With 64-bit machines becoming prevalent, there is an option to move to 64-bit executables which have none of the 32-bit heap limitations. Following is a short example program which shows a 64-bit program which allocates six gigabytes of heap space.

/*
 *  useheap2.c:  Example program which uses heap space by allocating 
 *              a global variable and a dynamic allocation.
 *
 *               cc -xarch=v9 useheap2.c -o useheap2
 */

#include 

main ()
{

  char *ch2;
  char *ch3;

  ch2 = (char *) malloc (5ll*1024ll*1024ll*1024ll);
  ch3 = (char *) malloc (1024*1024*1024);
  sleep(10);

}

Here is an example compilation, run and pmap output:

% cc -xarch=v9 useheap2.c -l useheap2
% useheap2 &
 18531
% pmap 18531
.
.
.
0000000100102000 5242896K read/write/exec     [ heap ]
0000000240106000 1048576K read/write/exec     [ heap ]
.
.
.

With 64-bit executables, the addresses within the pmap output are now shown as 64-bit quantities. The output above shows the two large heap allocations which total six gigabytes. This is far beyond the limit of 32-bit executables. Note that stack size limits still apply.

References:

Solaris manpages: proc(1), malloc(3C), mmap(2), libmapmalloc(4), ulimit, pmap

DOC ID #1407