I've never gone out of my way to remove them, but I know to install the A/V jacks on my Colecovision, a bit of the shielding had to be removed. Also, I (like most people) don't have the metal plates used to connect the Genesis to the Sega CD, and I've never had a problem.
Like others have said, they were legally required to be there, but if you have to remove part of them to do a mod, I wouldn't worry about it personally.