Well, no. You certainly can't do it at install, and even if there's a hidden setting that can only be invoked by installing a user-created app, changing the settings post-install is too late - your address book has been downloaded and stored permanently by the app maker.
What you describe may exist in a future version of Android but does not in 4.3.
It's not too late immediately after install, as the app can't do anything until you've run it. That's including background tasks, they can only be kicked off the first time the app is explicitly opened (correct me if there's some loophole here I'm not aware of).
You have a valid point that it could be easier, but I wanted to clarify that.
As an app maker, I'd like it if I could declare mandatory permissions and optional ones, as well as the iOS model of asking for permissions on demand. There are features I'd like to add, but I can't easily do so because (a) it would prevent auto-updates - all existing users would have to explicitly okay it, so many will naturally leave it at current version for months or years and never receive any update; (b) more permissions naturally makes users more cautious. So I'd rather be able to just ask to do something when the user has expressed they need it, but there's no such option.
What you describe may exist in a future version of Android but does not in 4.3.