Annecy Castle is just one of the attractions in Annecy Old Town, Annecy so you might want to stop by during your trip. If you want to experience more of the culture Annecy has to offer, head to Palais de l Ile, Notre-Dame-de-Liesse Church, and Church of St. Maurice.
If museums are on your list of things to see, you might want to browse the collections at Palais de l Ile in Annecy Old Town. If you want to see more of the museums in Annecy, you can easily walk to Conservatoire Art et Histoire.