In the past few years, I think I've started but neglected to finish Fringe, Chuck, and Battlestar Galactica. None of these were really given up on out of frustration, though; I just kind of lost interest at some point along the way. Fringe I stopped about halfway through the first season when the dick FBI supervisor guy showed up. Chuck I made it halfway through Season 3, I think? After Chuck and the girl got together. BSG I'm almost at the end of, and should probably go back and catch the last few episodes on Netflix, although I hear somewhat bad things about it.
By comparison, I've caught up on Mad Men entirely over the last year, and I'm looking forward to getting Breaking Bad season 4 on Netflix or DVD. I would hope that I wouldn't have to buy it, but hopefully it'll hit streaming soon. It's rare for me to get any length into a television show and then not finish it; I usually feel obligated. But lately there just hasn't been a huge amount of time for me to watch TV lately, so I don't feel too bad about abandoning something I'm not 100% behind.