Caching sticks around even with plugin deactivated

This isn't exactly a pre-sales question as I already got Speed of Light, but I didn't want to open another ticket as this isn't that urgent.

I'm having some super sticky caching issue that makes it a bit hard to develop. At least some of the caching sticks around even after I clear cache, and more than that, even after the plugin is disabled.

I had some images in my theme which I updated and changed the live version to a new version. But the new version would not load, even when I completely cleared browser cache (even viewed on different device) and clicked clear cache on WP toolbar, and then I disabled the plugin and still the images were cached.

At that point I actually thought it was my host because if a plugin is disabled, how can it still be doing something? But my host said nothing like that is possible so I took another look. Sure enough, as soon as I deleted WP Speed of light plugin, the image instantly updated on the next hard refresh.

Only by deleting the plugin entirely was I able to cause my theme's images to update.

I can always just delete the plugin and reinstall it, but I'd love to resolve it at some point to where I can rely on the plugin's "clear cache" button. Does anyone know what's up with this behavior? Did I set something up wrong?

Thanks in advance