Protein/RNA clusters arise frequently in spatially regulated biological processes, from the asymmetric distribution of P granules and PAR proteins in developing embryos to localized receptor oligomers in migratory cells. This co-occurrence suggests that protein clusters might possess intrinsic properties that make them a useful substrate for spatial regulation. Here, we demonstrate that protein droplets show a robust form of spatial memory, maintaining the spatial pattern of an inhibitor of droplet formation long after it has been removed. Despite this persistence, droplets can be highly dynamic, continuously exchanging monomers with the diffuse phase. We investigate the principles of biophysical spatial memory in three contexts: a computational model of phase separation; a novel optogenetic system where light can drive rapid, localized dissociation of liquid-like protein droplets; and membrane-localized signal transduction from clusters of receptor tyrosine kinases. Our results suggest that the persistent polarization underlying many cellular and developmental processes could arise through a simple biophysical process, without any additional biochemical feedback loops.