Until I test all the various things from my first post, most of which are specifically sold for rubber stamping and/or easy printmaking, I wanted to point out a few materials that might seem like good choices, but are probably best avoided (unless you're going for a specific effect, in which case you may want to experiment anyway).
If you read the other posts on this blog, or follow my Twitter or Instagram (both @anagramforink), you probably know I do a lot of linocuts. So it may seem odd that I'm now telling you not to use it. The thing is, though, lino is great for relief printmaking, but terrible for stamping.
Lino is a lot harder to cut than rubber stamp materials, it'll dull your tools a lot faster (and you *really* need sharp tools to cut it in the first place), and you probably won't be happy with the results unless you're printing on a press or using a hand-printing rub technique. For simple stamping, it's just not soft enough to make a nice clean image.
So, unless you plan to take up printmaking proper, save yourself some aggravation and steer clear of linoleum.
One of the things on my list for stamp making materials is white plastic erasers, and they're excellent for rubber stamping. But they're pretty much the only kind of eraser worth spending money on. Personally they're also the only kind I use for erasing, too (except, very occasionally, kneaded rubber).
Most other kinds of erasers are too hard, too crumbly, or too textured to give good results (though, possibly, they might be useful for specific effects if you want to spend a lot of time experimenting). Those horrible pink erasers are among the worst, though different brands have different properties and a few of them might be worth trying if you're in the experimenting mood.
Also, there are some white erasers that are not the good kind -- they're pretty much just white versions of the pink ones, and nearly as bad to work with. They're usually the same shape as the pink ones, and often sold with ink erasers attached to one end.
Some coloured erasers, like novelty kinds, are very similar material to the good white plastic ones and would probably work, so if you find them really cheap it might be worth trying them, but in general I'd just say just get the white ones. You can even find them at the dollar store in a pinch.
Those sheets of craft foam might be tempting, and so might styrofoam trays and that sort of thing. They certainly could be fun to play with, but won't give very good results for rubber stamps you actually want to keep and re-use.
If you've got any more materials to avoid, or have achieved interesting results with something I said was no good, please feel free to leave a comment and share your results.