I think you actually can (must?) buy a pass that gives your entry to the mansions (or, as they call them, cottages) of note. You don't need to purchase them a la carte.
Also, kind of a one-off, but go to the Tennis Hall of Fame. I couldn't care less about tennis, at least as compared to most other sports, but it's very good and well worth the visit.
Food: Eat at the White Horse Tavern, corner of Marlborough and Farewell Streets. A little dressy (compared to most other places), but the food is dynamite - best in town. Building is also ancient, so it oozes history. I also recommend the Red Parrot for dinner, corner of Thames and Memorial Blvd. West. If the weather is nice, they'll open up all the front windows facing Thames Street, which will be crowded with foot traffic. It's a very fun atmosphere, and the food is good.
One of the biggest touristy things to do is to try the clam chowder at the Black Pearl on (I think) Bowen's Wharf, but it actually is very good. The restaurant in general is also quite good.
Take a cruise of Narragansett Bay. You can down a couple beers and some oysters while you wait to board, and the ride is fun.
Also, if you like colonial architecture and historic preservation, take a walk through The Point neighborhood. It's on the bay, with a great view of the Claiborne Pell Bridge, but the entire neighborhood has been preserved by the historic preservation society (I can't think of the actual name). Really cool, though most of the homes are actually rather modest. Tim O'Shea, the Bryant basketball coach (formerly of Ohio U.) lives there, not that it matters.
Of course, the cliff walk and all that, as others have mentioned, are good fun, too. Great walking town (lousy for driving), and you ought to enjoy it.