The 3.5" screen is better for me because it easily fits in any pocket, so I always have it with me, and the battery easily lasts all day even with heavy web browsing.
I am yet to see a male claiming Galaxy Note (5.3-inch screen) does not fit in his pocket. By the way, I am assuming you're a man because women don't carry their phones in their pocket.