I am sure as well. No wonder it all fell down and rolled away. The Templers use rounded boulders without LOTS of cement and masonery for foundations or walls. Though must account for why nothing lasted above ground.
Here's Mortimer's Tower, built in the earth 14th century at Kneilworth. The Normans learned to start with a circular "keep" or motte and bailey, or tower and enclosure, which are strong and fast to build, and expand the castle from that.
What you have there I could not call a "castle" or any part of one. Though I must say, when General Sullivan marched through this valley in 1779 he destroyed an "Indian Castle" in what is now called "Castle Creek, NY". It was three wood sheds full of pumpkins; by other written accounts. So maybe castles on this side of the ocean are less impressive.