Grzegorz Milos working in conjunction with Michael Fetterman
University of Cambridge Computer Laboratory
Memory CoW in Xen
******
Memory is widely recognised to be the most scarce resource in virtual
environments. While memory available in commodity desktops and servers
grows according to Moore's Law (increases twofold every two years[4]),
the growing demands placed by modern OSes (>1GB for Windows Vista) as
well as the partitioning between multiple VMs, highlights the
importance of efficient memory management.

The research community responded by developing dynamic memory
management strategies based on ballooning[1], hotplugging[2] or guest
page hinting[3]. Additionally, the opportunity to relieve memory
pressure in VMMs by Copy-on-Write (CoW) memory sharing has been
popularised by Waldspurger[1].

In homogenous environments CoW offers substantial savings (Waldspurger
claims up to 60%), however such setups are extremely rare in real
world. It therefore follows that CoW mechanism must be efficient to be
generally applicable. We are proposing a paravirtual

CoW scheme designed to lower the three major costs of memory sharing:
detecting duplicate memory pages, servicing CoW faults and managing
the surplus memory.  Memory sharing opportunities are commonly
detected by comparing content hashes. This is indeed the approach used
by Waldspurger. However, the constant growth of the search space, as
well as increasing performance gap between CPU and memory[5] calls for
a better solution. We believe that we can detect the majority of
duplicate pages, at the fraction of the cost, by leveraging extra
information provided by the guest OSes. In particular interception of
blk read operations allowed us to find 40% of sharings. The additional
pieces of information provided by guests are also valuable in
servicing CoW faults. Our fault handler design is based on a simple
observation: if CoW fault is the result of a page being deallocated by
the guest memory manager, there is no need to mem-copy, since the
content of the page is conceptually discarded. Finally, in order to
make the full use of surplus memory from CoW, a mechanism by which Xen
can asynchronously reclaim memory is required.

We achieve that by putting an obligation on each guest to provide
ahead of time repayment guarantees for each bonus page received. The
repayment guarantees take the form of page repayment FIFO. Since the
pages included in the FIFO must be volatile (i.e. discardable) our
mechanism shares many characteristics with the "guest page hinting"
project under development by IBM[3].

Our prototype implementation of the CoW support in Xen uses
paravirtual guests and shadow translate mode. This allows us to
transparently share/unshare memory entirely in Xen. At the time of
writing the hypervisor extension responsible for these low level
operation is already completed. A prototype of CoW manager interposing
on the blktap reads exists too. Our efforts are therefore
concentrating on the repayment FIFO implementation.