Can I measure the performance gain?

Partially, the online speed test like Pingdom measure only a first page load (no browser cache) for non loggedin users (no loggedin cache) so you may find few difference, but your users will!

The best way to measure the performance, is to test by yourself the time to render a page in your browser.