The site is hosted on S3 and cloudfront right now. The poor performance might be because the page transaction and animations.
My initial vision is display the album similar to a photo book, now seems nature scrolling might be more acceptable.
I, unlike some others, don't mind the paginated scrolling. I just think it would be good to be able to quickly flip through pages without delay just because the images are loading.
A good example would be a Facebook photo album. If I click the left arrow as fast as I can, the page keeps flipping instantly. Instead of seeing each picture, I see a small loading icon and a box where the picture will be. When I finally stop mashing the arrow, the picture I'm settled on will load.
I hope you don't find my comments overly critical. It's a beautiful design, just trying to be constructive.