There must be a window of time in your late teens when it grabs you, because while I appreciated it and really like some of his short stories, it didn't "change my life" like some people I've heard of.
I also hear about people who claim to have read The Hobbit or Kim when they were five. Five! And while I started reading before Kindergarten, I wasn't gulping down whole novels when I was five. Or even having them read out loud to me.
One funny story this reminds me of is when I was talking to a precocious fourth grader my wife had. We were talking about books and somehow Ender's Game came up in conversation, so he went off, read it, and told me about this achievement. I said, "Well, I 'm glad you enjoyed it. That might be a good book to revisit when you're a little older. You might find you get more out of it."
He came up to me the next year, when he was in fifth grade and said, "You were so right about me re-reading Ender's Game. I got so much more out of it now that I'm older."