The Rye area is perfect for day trips to local castles and other historic sites, of which there are a few examples below:
Camber Castle: There is a picturesque walk from Rye town to Camber Castle. Note, Castle only open a few days a year: http://www.visit1066country.com/attractions/camber-castle-p44063
Ypres Tower in Rye, has great views: http://www.ryemuseum.co.uk/ypres-tower/
Bodiam Castle: http://www.nationaltrust.org.uk/bodiam-castle/
1066 Battle of Hastings, Abbey & Battlefield: https://www.english-heritage.org.uk/daysout/properties/1066-battle-of-hastings-abbey-and-battlefield/
Hastings old town is very interesting and worth a visit.
Scotney Castle near Tunbridge Wells: http://www.nationaltrust.org.uk/scotney-castle/
Hever Castle near Edenbridge: http://www.hevercastle.co.uk/