Remembering Transformer for Continual Learning