Probably built on the site of an old Roman fort by William Strickland in 1397, although there is some evidence that it was built by Ralph Neville in 1396. In the 15th century the Neville family added the Red Tower and later Richard, Duke of Gloucester ( later Richard III), added a new gatehouse as well as more living quarters. The castle was used to defend the area from Scottish raiders
In 1648, during the Civil War, the castle served as an army headquarters building for the Commonwealth. Like other castles in the area when it fell into disrepair the stone was used for local building.
Penrith castle is sometimes referred to as the "Castle of Kings"