What do you think about not logging page views for bots? The reason I'm asking is because they make so many entries in the database and sometimes can bring down a site if a lot of them hit at once. I've had to put in crawl delays, but on one site, the security scanning company brings down the site every time it scans. There end up being a lot of database timeout errors.

I don't want to get rid of bots completely because we need google, bing, etc. Just trying to minimize the impact of the on the site.

Judy,

I have been very lucky with Cloudflare for this purpose, I did have to upgrade to the business plan to have a bit more control. I wish of course I had more granule control, but with the tools provided in business I was able to reduce my bots to less than 10 a day.

There is a fine line between a good bot and a mischief bot, so that is an issue that you would have to contend with, but for the most part, under the "Configure Super Bot Fight Mode" section, the "Definitely automated" automated option does an excellent job.

I have created rules to let other bots in, the only thing that might get in the way is if you use a service like SEMrush where you might manually have to turn it off, for services like those to complete their tasks and reporting.

I looked at the project honeypot and decided it was not for us, Funny though they seem to advertise Cloudflare on their site? Wonder why?

I hope some of this helps,
-Ray