I have been considering one ever since I first heard about it. I haven't used one yet, but the most common complaint I hear about is game compatibility. Since you have to use a PS3 or PS4 controller to play the games, any game that requires the touch pads may not work. I've read that some games incorporate the L3 and R3 buttons for the touch screen. In other words, as you hold down the L3 or R3 buttons and move the sticks around, that acts as your finger touching the screen. However, for games that require quick touch pad actions, the L3 and R3 buttons probably won't be quick enough.
At the end of the day, there just aren't enough games on the Vita (PS TV compatible or not) that I'm interested in enough for me to buy one.