DumbFB: don't unmap if not mapped