The Rye area is perfect for day trips to local castles and other historic sites, of which there are a few examples below:

Camber Castle: There is a picturesque walk from Rye town to Camber Castle.  Note, Castle only open a few days a year: http://www.visit1066country.com/attractions/camber-castle-p44063

Ypres Tower in Rye, has great views: http://www.ryemuseum.co.uk/ypres-tower/

Bodiam Castle: http://www.nationaltrust.org.uk/bodiam-castle/

1066 Battle of Hastings, Abbey & Battlefield: https://www.english-heritage.org.uk/daysout/properties/1066-battle-of-hastings-abbey-and-battlefield/ 

Hastings old town is very interesting and worth a visit.

Scotney Castle near Tunbridge Wells: http://www.nationaltrust.org.uk/scotney-castle/

Hever Castle near Edenbridge: http://www.hevercastle.co.uk/