Whoops, you are right, my posterior should have been Beta[2,100e6+1] (I chose a uniform prior). With that I get a median probability of 1.7e-8, and a mean probability of 2e-8. Good catch!
Note that this is not particularly sensitive to whether you choose beta[1,1] or Beta[1,3] as the prior.
Yeah, that makes virtually no difference. I'm not familiar with picking these priors, so I could imagine picking (say) Beta[0.5,1] or Beta[5,30] as prior too, and that does make some difference (as in the prior alpha has some impact).
Is there some principled reason to pick one over the other, or is it just "whatever looks reasonable"?
I also can't explain why plain numerical simulation doesn't come up with almost identical p values - I get 1/1.5e8 to 1/1.7e8, but that's 6.25e-9. I mean, it's the same order of magnitude, but it's nevertheless quite different.
Oh well, unless you've got some insight there, I guess I'm going to give up on that analysis - that kind of nitty gritty stuff is a timesink ;-).
Note that this is not particularly sensitive to whether you choose beta[1,1] or Beta[1,3] as the prior.