For those with kids (or just a love of all things nautical), the Maritime Museum is worth adding to the list. Made up of close to a dozen ships, schooners, and submarines (consider it a floating museum), each vessel houses an exhibit. The Star of India, which is the oldest active sailing ship in the world, was featured on Pirates of the Caribbean: On Stranger Tides, and is a favorite of visitors and locals alike.
Local Tip: After exploring the ships, explore the area. The Embarcadero, which lines the harbor, is a perfect place for admiring sailboats and an excellent view of downtown.
