Store Pixels Values Currently On Screen And Compare Them To Values On The First Frame?

I need a way to store the pixels values currently on the screen and compare them to the values on the first frame. Right now I'm using glreadpixels as follows:

currentBuffer= BufferTools.reserveByteData(mapSize);
glReadPixels(mapStartX, mapStartY, mapWidth, mapHeight, GL_BLUE, GL_UNSIGNED_BYTE, currentBuffer);
for (int i = 0; i < mapSize; i++) {
if (currentBuffer.get(i) != baseBuffer.get(i)) {
//Do nothing
continue;
}
//Do something
}

This works perfectly fine but turns out to be a real bottleneck, dropping the fps to a third of what it was. Is there any quicker way? All I'm after is speed, I don't even need to show it on the screen if the comparison is made "behind the scene".