Fatal flaw in WaveServiceProvider.php caused deployment to take down my site
I know the title sounds dramatic but this caused a panic for me tonight and took me a few hours to resolve. I was able to track down the cause of the problem but I'm not sure what changes need to be made to prevent future problems.
I use Deployer to run automated deployments for my site. One of the features of Deployer, as well as many other similar services that manage server deployments, is zero-downtime deployments. This is achieved by creating a file architecture on the server in the format of ~/my_site.com/releases/1
with numbers increasing for each deployment. Once the files are built, composer is run, etc. then the symlink for the document root is set to the newest release folder.
In my case, I have Deployer preserve the last 10 releases in case a rollback is needed, and the oldest release folder is automatically deleted. Tonight as I deployed a new release, I got this error:
In WaveServiceProvider.php line 156:
require_once(/home/deployer/mysite/releases/29/wave/src/Helpers/globals.php): Failed to open stream: No such file or directory
What was strange was that I was on release 39, so 29 was just deleted. How was it being referenced? After scouring through files and trying to figure this out, I found in my database that in the cache
table there was a key titled claimzap_cache_wave_helpers
with value "a:1:{i:0;s:67:""/home/deployer/mysite/releases/29/wave/src/Helpers/globals.php"";}"
expiration 2076449816
.
I deleted this entry, redeployed, and everything was restored. I checked the database again, and this time there was a new entry with my new deployment release number. I did another release, and this entry did not change. I now know that in 8 more releases, I am going to experience this problem again.
I suspect this line in WaveServiceProvider.php
is the issue:
$helpers = Cache::rememberForever('wave_helpers', function () {
return glob(__DIR__.'/Helpers/*.php');
});
Caching this forever seems to be a problem if your deployment path does not remain the same. What is the solution to prevent this from happening?