You are spot on with this; after learning a second language (Arabic) in my 20's and a third (Hebrew) in my 40's though I do find it "easier" in the sense that I know how to learn a hard language a little better than at the first. Same with programming. I learned Fortran in my teens and then Java in my late 30's and of course there is much more to Java than Fortran but the background helped me learn Java much faster than if I started cold. They key though is desire, time and focus - if you can not get those things you will not be able to learn anything that takes a long time to master.