Abstract. To solve problems that require far more memory than a single machine can supply, data can be swapped to disk in some manner, it can be compressed, and/or the memory of multiple parallel machines can be used to provide enough memory and storage space. Instead of implementing either functionality anew and specific for each application, or instead of relying on the operating system’s swapping algorithms (which are inflexible, not algorithm-aware, and often limited in their fixed storage capacity), our solution is a Large Virtual Machine (LVM) that transparently provides a large address space to applications and that is more flexible and efficient than operating system approaches. LVM is a virtual machine for Java that is designed to support large address spaces for billions of objects. It swaps objects out to disk, compresses objects where needed, and uses multiple parallel machines in a Distributed Shared Memory (DSM) setting. The latter is the main focus of this paper. ...