for example you have next files in /var/log/apache2/ folder:

site.access.log
site.access.log.1
site.access.log.2.gz
site.access.log.3.gz
...
site.access.log.89.gz

And you want to grep through all of them, you can use next command:

zcat -f /var/log/apache2/site.access.* |grep 123.123.123.123

Normally zcat shows only .gz logs, but -f options allow to ignore error if file is not gz and considers it as plain text.