Well, I don't know apache, but you might want to see about placing these restrictions in tcp wrappers rather than in apache - if you have tcp wrappers installed, read the man for hosts.deny and set up an http deny for all but those 30 clients. You're only moving the overhead around rather than eliminating it, but it could be that tcp wrappers deal with the overhead more efficiently. Like I said though, I don't know apache, this is just a guess....
Mathias