libslack(mem) - memory module
#include <slack/std.h>
#include <slack/mem.h>
typedef struct Pool Pool;
#define null NULL
#define nul '\0'
#define mem_new(type)
#define mem_create(size, type)
#define mem_resize(mem, size)
void *mem_resize_fn(void **mem, size_t size);
#define mem_release(mem)
void *mem_destroy(void **mem);
void *mem_create_secure(size_t size);
void mem_release_secure(void *mem);
void *mem_destroy_secure(void **mem);
char *mem_strdup(const char *str);
#define mem_create2d(type, x, y)
#define mem_create3d(type, x, y, z)
#define mem_create4d(type, x, y, z, a)
void *mem_create_space(size_t size, ...);
size_t mem_space_start(size_t size, ...);
#define mem_release2d(space)
#define mem_release3d(space)
#define mem_release4d(space)
#define mem_release_space(space)
#define mem_destroy2d(space)
#define mem_destroy3d(space)
#define mem_destroy4d(space)
#define mem_destroy_space(space)
Pool *pool_create(size_t size);
Pool *pool_create_with_locker(Locker *locker, size_t size);
void pool_release(Pool *pool);
void *pool_destroy(Pool **pool);
Pool *pool_create_secure(size_t size);
Pool *pool_create_secure_with_locker(Locker *locker, size_t size);
void pool_release_secure(Pool *pool);
void *pool_destroy_secure(Pool **pool);
void pool_clear_secure(Pool *pool);
#define pool_new(pool, type)
#define pool_newsz(pool, size, type)
void *pool_alloc(Pool *pool, size_t size);
void pool_clear(Pool *pool);This module is mostly just an interface to malloc(3), realloc(3) and free(3) that tries to ensure that pointers that don't point to anything get set to null. It also provides dynamically allocated multi-dimensional arrays, memory pools and secure memory for the more adventurous.
#define null NULLEasier to type. Easier to read. Feel free to keep using NULL if you prefer.
#define nul '\0'A name for the nul character.
#define mem_new(type)Allocates enough memory (with malloc(3)) to store an object of type type. It is the caller's responsibility to deallocate the allocated memory with free(3), mem_release(3), or mem_destroy(3). It is strongly recommended to use mem_destroy(3), because it also sets the pointer variable to null. On success, returns the address of the allocated memory. On error, returns null.
#define mem_create(size, type)Allocates enough memory (with malloc(3)) to store size objects of type type. It is the caller's responsibility to deallocate the allocated memory with free(3), mem_release(3), or mem_destroy(3). It is strongly recommended to use mem_destroy(3), because it also sets the pointer variable to null. On success, returns the address of the allocated memory. On error, returns null.
#define mem_resize(mem, num)Alters the amount of memory pointed to by *mem. If *mem is null, new memory is allocated and assigned to *mem. If size is zero, *mem is deallocated and null is assigned to *mem. Otherwise, *mem is reallocated and assigned back to *mem. On success, returns *mem (which will be null if size is zero). On error, returns null with errno set appropriately, and *mem is not altered.
void *mem_resize_fn(void **mem, size_t size)An interface to realloc(3) that also assigns to a pointer variable unless an error occurred. mem points to the pointer to be affected. size is the requested size in bytes. If size is zero, *mem is deallocated and set to null. This function is exposed as an implementation side effect. Don't call it directly. Call mem_resize(3) instead. On error, returns null with errno set appropriately.
#define mem_release(mem)Releases (deallocates) mem. Same as free(3). Only to be used in destructor functions. In other cases, use mem_destroy(3) which also sets mem to null.
void *mem_destroy(void **mem)Calls free(3) on the pointer, *mem. Then assigns null to this pointer. Returns null.
void *mem_create_secure(size_t size)Allocates size bytes of memory (with malloc(3)), and then locks it into RAM with mlock(2) so that it can't be paged to disk, where some nefarious local user with root access might read its contents. Note that additional operating system dependent measures might be required to prevent the root user from accessing the RAM of arbitrary processes (e.g. On Linux: sysctl kernel.yama.ptrace_scope=3). It is the caller's responsibility to deallocate the secure memory with mem_release_secure(3) or mem_destroy_secure(3) which will clear the memory and unlock it before deallocating it. It is strongly recommended to use mem_destroy_secure(3), because it also sets the pointer variable to null. On success, returns the address of the secure allocated memory. On error, returns null with errno set appropriately.
Note that entire memory pages are locked by mlock(2), so don't create many, small pieces of secure memory or many entire pages will be locked. Use a secure memory pool instead. Also note that on old systems, secure memory requires root privileges.
On some systems (e.g. Solaris), memory locks must start on page boundaries. So we need to malloc(3) enough memory to extend from whatever address malloc(3) may return to the next page boundary (worst case: pagesize - sizeof(int)) and then the actual number of bytes requested. We need an additional sizeof(void *) + sizeof(size_t) bytes (e.g. 8 or 16) to store the address returned by malloc(3) (so we can free(3) it later), and the size passed to mlock(2) so we can pass it to munlock(2) later. Unfortunately, we need to store the address and the size after the page boundary and not before it, because malloc(3) might return a page boundary or an address less than sizeof(void *) + sizeof(size_t) bytes to the left of a page boundary.
It will look like:
for free()
+-------+ +- size+n for munlock()
v | v
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |* * * *|# # # #| | | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
^ ^ ^ . . . size bytes . . . . . .
| +- next page |
+- malloc() +- address returnedIf your system doesn't require page boundaries (e.g. Linux), the address returned by malloc(3) is locked, and only the size is stored.
void mem_release_secure(void *mem)Sets the memory pointed to by mem to 0xff bytes, then to 0xaa bytes, then to 0x55 bytes, then to nul bytes, then unlocks and releases (deallocates) mem. Only to be used on memory returned by mem_create_secure(3). Only to be used in destructor functions. In other cases, use mem_destroy_secure(3) which also sets mem to null.
void *mem_destroy_secure(void **mem)Sets the memory pointed to by *mem to 0xff bytes, then to 0xaa bytes, then to 0x55 bytes, then to nul bytes, then unlocks and destroys (deallocates and sets to null) *mem. Only to be used on memory returned by mem_create_secure(3). Returns null.
char *mem_strdup(const char *str)Returns a dynamically allocated copy of str. It is the caller's responsibility to deallocate the new string with free(3), mem_release(3), or mem_destroy(3). It is strongly recommended to use mem_destroy(3), because it also sets the pointer variable to null. This function exists because strdup(3) is not part of the ISO C standard. On error, returns null with errno set appropriately.
#define mem_create2d(i, j, type)Alias for allocating a 2-dimensional array. See mem_create_space(3).
#define mem_create3d(i, j, k, type)Alias for allocating a 3-dimensional array. See mem_create_space(3).
#define mem_create4d(i, j, k, l, type)Alias for allocating a 4-dimensional array. See mem_create_space(3).
void *mem_create_space(size_t size, ...)Allocates a multi-dimensional array of elements of size size and sets the memory to nul bytes. The remaining arguments specify the sizes of each dimension. The last argument must be zero. There is an arbitrary limit of 32 dimensions. The memory returned is set to nul bytes. The memory returned needs to be cast or assigned into the appropriate pointer type. You can then set and access elements exactly like a real multi-dimensional C array. Finally, it must be deallocated with mem_destroy_space(3) or mem_release_space(3) or mem_destroy(3) or mem_release(3) or free(3). It is strongly recommended to use mem_destroy_space(3) or mem_destroy(3), because they also set the pointer variable to null.
Note: You must not use memset(3) on all of the returned memory because the start of this memory contains pointers into the remainder. The exact amount of this overhead depends on the number and size of dimensions. The memory is allocated with calloc(3) to reduce the need to memset(3) the elements, but if you need to know where the elements begin, use mem_space_start(3).
The memory returned looks like (e.g.):
char ***a = mem_create3d(2, 2, 3, char);
+-------------------------+
+-------|-------------------+ |
a +-------|-------|-------------+ | |
| +-------|-------|-------|-------+ | | |
v | | | | V V V V
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| a[0] | a[1] |a[0][0]|a[0][1]|a[1][0]|a[1][1]| | | | | | | | | | | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | ^ ^ a a a a a a a a a a a a
+-------|-------+ | 0 0 0 0 0 0 1 1 1 1 1 1
+-----------------------+ 0 0 0 1 1 1 0 0 0 1 1 1
0 1 2 0 1 2 0 1 2 0 1 2size_t mem_space_start(size_t size, ...)Calculates the amount of overhead required for a dynamically allocated multi-dimensional array created by a call to mem_create_space(3) with the same arguments. If you need to reset all elements in such an array to nul bytes:
int ****space = mem_create_space(sizeof(int), 2, 3, 4, 5, 0);
size_t start = mem_space_start(sizeof(int), 2, 3, 4, 5, 0);
memset((char *)space + start, '\0', sizeof(int) * 2 * 3 * 4 * 5); #define mem_release2d(space)Alias for releasing (deallocating) a dynamically allocated 2-dimensional array. See mem_release_space(3).
#define mem_release3d(space)Alias for releasing (deallocating) a dynamically allocated 3-dimensional array. See mem_release_space(3).
#define mem_release4d(space)Alias for releasing (deallocating) a dynamically allocated 4-dimensional array. See mem_release_space(3).
#define mem_release_space(space)Releases (deallocates) a multi-dimensional array, space, allocated with mem_create_space. Same as free(3). Only to be used in destructor functions. In other cases, use mem_destroy_space(3) or mem_destroy which also set space to null.
#define mem_destroy2d(space)Alias for destroying (deallocating and setting to null) a 2-dimensional array. See mem_destroy_space(3).
#define mem_destroy3d(space)Alias for destroying (deallocating and setting to null) a 3-dimensional array. See mem_destroy_space(3).
#define mem_destroy4d(space)Alias for destroying (deallocating and setting to null) a 4-dimensional array. See mem_destroy_space(3).
#define mem_destroy_space(mem)Destroys (deallocates and sets to null) the multi-dimensional array pointed to by space.
Pool *pool_create(size_t size)Creates a memory pool of size size from which smaller chunks of memory may be subsequently allocated (with pool_alloc(3)) without resorting to the use of malloc(3). Useful when you have many small objects to allocate, but malloc(3) is slowing your program down too much. It is the caller's responsibility to deallocate the new pool with pool_release(3) or pool_destroy(3). It is strongly recommended to use pool_destroy(3), because it also sets the pointer variable to null. On success, returns the pool. On error, returns null.
The size of a pool can't be changed after it is created, and the individual chunks of memory allocated from within a pool can't be separately deallocated. The entire pool can be emptied with pool_clear(3).
Pool *pool_create_with_locker(Locker *locker, size_t size)Equivalent to pool_create(3) except that multiple threads accessing the new pool will be synchronised by locker.
void pool_release(Pool *pool)Releases (deallocates) pool. Only to be used in destructor functions. In other cases, use pool_destroy(3) which also sets pool to null.
void *pool_destroy(Pool **pool)Destroys (deallocates and sets to null) *pool. Returns null. Note: pools shared by multiple threads must not be destroyed until after all threads have finished with it.
Pool *pool_create_secure(size_t size)Creates a memory pool of size size just like pool_create(3) except that the memory pool is locked into RAM with mlock(2) so that it can't be paged to disk where some nefarious local user with root access might read its contents. Note that additional operating system dependent measures might be required to prevent the root user from accessing the RAM of arbitrary processes (e.g. On Linux: sysctl kernel.yama.ptrace_scope=3). It is the caller's responsibility to deallocate the new pool with pool_release_secure(3) or pool_destroy_secure(3) which will clear the memory pool and unlock it before deallocating it. In all other ways, the pool returned is exactly like a pool returned by pool_create(3). On success, returns the pool. On error, returns null with errno set appropriately. Note that on old systems, secure memory requires root privileges.
Pool *pool_create_secure_with_locker(Locker *locker, size_t size)Equivalent to pool_create_secure(3) except that multiple threads accessing the new pool will be synchronised by locker.
void pool_release_secure(Pool *pool)Sets the contents of the memory pool to 0xff bytes, then to 0xaa bytes, then to 0x55 bytes, then to nul bytes, then unlocks and releases (deallocates) pool. Only to be used on pools returned by pool_create_secure(3). Only to be used in destructor functions. In other cases, use pool_destroy_secure(3) which also sets pool to null.
void *pool_destroy_secure(Pool **pool)Sets the contents of the memory pool to 0xff bytes, then to 0xaa bytes, then to 0x55 bytes, then to nul bytes, then unlocks and destroys (deallocates and sets to null) *pool. Returns null. Note: secure pools shared by multiple threads must not be destroyed until after all threads have finished with it.
void pool_clear_secure(Pool *pool)Fills the secure pool with 0xff bytes, then 0xaa bytes, then 0x55 bytes, then nul bytes, and deallocates all of the chunks of secure memory previously allocated from pool so that it can be reused. Does not use free(3).
#define pool_new(pool, type)Allocates enough memory from pool to store an object of type type. On success, returns the address of the allocated memory. On error, returns null with errno set appropriately.
#define pool_newsz(pool, size, type)Allocates enough memory from pool to store size objects of type type. On success, returns the address of the allocated memory. On error, returns null with errno set appropriately.
void *pool_alloc(Pool *pool, size_t size)Allocates a chunk of memory of size bytes from pool. Does not use malloc(3). The pointer returned must not be passed to free(3) or realloc(3). Only the entire pool can be deallocated with pool_release(3) or pool_destroy(3). All of the chunks can be deallocated in one go with pool_clear(3) without deallocating the pool itself.
On success, returns the pointer to the allocated pool memory. On error, returns null with errno set appropriately (i.e. EINVAL if pool is null, ENOSPC if pool does not have enough unused memory to allocate size bytes).
It is the caller's responsibility to ensure the correct alignment if necessary by allocating the right numbers of bytes. The easiest way to do ensure is to use separate pools for each specific data type that requires specific alignment.
void pool_clear(Pool *pool)Deallocates all of the chunks of memory previously allocated from pool so that it can be reused. Does not use free(3).
On error, errno is set by underlying functions or as follows:
EINVALWhen arguments are invalid.
ENOSPCWhen there is insufficient available space in a pool for pool_alloc(3) to satisfy a request.
ENOSYSReturned by mem_create_secure(3) and pool_create_secure(3) when mlock(2) is not supported (e.g. Mac OS X).
MT-Safe (mem)
MT-Disciplined (pool) man locker(3) for details.
1D array of longs:
long *mem = mem_create(100, long);
mem_resize(&mem, 200);
mem_destroy(&mem);3D array of ints:
int ***space = mem_create3d(10, 20, 30, int);
int i, j, k;
for (i = 0; i < 10; ++i)
for (j = 0; j < 20; ++j)
for (k = 0; k < 30; ++k)
space[i][j][k] = i + j + j;
mem_destroy3d(&space);A pool of a million integers:
void pool()
{
Pool *pool;
int i, *p;
if (!(pool = pool_create(1024 * 1024 * sizeof(int))))
return;
for (i = 0; i < 1024 * 1024; ++i)
{
if (!(p = pool_new(pool, int)))
break;
*p = i;
}
pool_destroy(&pool);
}Secure memory:
char *secure_passwd = mem_create_secure(32);
if (!secure_passwd)
exit(EXIT_FAILURE);
get_passwd(secure_passwd, 32);
use_passwd(secure_passwd);
mem_destroy_secure(&secure_passwd);Secure memory pool:
Pool *secure_pool;
char *secure_passwd;
if (!(secure_pool = pool_create_secure(1024 * 1024)))
exit(EXIT_FAILURE);
secure_passwd = pool_alloc(secure_pool, 32);
get_passwd(secure_passwd, 32);
use_passwd(secure_passwd);
pool_destroy_secure(&secure_pool);libslack(3), malloc(3), realloc(3), calloc(3), free(3), mlock(2), munlock(2), locker(3)
20230824 raf <raf@raf.org>