Yeah, it might peel off, or at least feel weird on the fabric. You would be better off using ink of some sort. something without polymers in it which will create a film. A friend of mine purchased a plain mat and drew/colored Pokemon on it with felt tip permanent markers. It came out pretty awesome looking. There are also a few companies that do custom prints as well.