I think this one is fairly simple.
Most of the site is within 150m of a residential area, so you can’t overfly the building site in Open A3, even when it’s closed.
If you can be confident there’s nobody present and stay 50m from the housing, then you can overfly in Open A2 (50m horizontally from uninvolved people) if you have an A2 CofC.
Your best option is if you can use an Article 16 Operational Authorisation. That applies outside “built-up areas” and defines those as “areas substantially used for industrial, recreational, commercial or residential purposes.” I’d personally say it’s clearly none of those, certainly at a weekend when nobody is working there. Yes, there may be a security guard, but so long as you are at least 30m up in the air, flying manually and not for commercial work, Article 16 covers you.
I’ve shared some comments in another thread which may help with your thought process: