The culprit was the LS Cache plugin setting - “Serve Stale Cache” located on Cache > Purge > Server Stale"
This setting serves “Old Cache” until the new cache is available in the background.
Turning this off allows to test if cache is actually being purged.