Find us at NeurIPS, Thur 4:30 pm #2115! We know networks have to be both plastic and stable but we're used to thinking about computations, such as memory, as additional requirements. Instead, we find that almost all stable & plastic networks display simple memory abilities.