Sam only remembered fragments from his previous life. He recalled how he had lived in that past existence, but when it came to memories of the novel world, his recollection was hazy. This was understandable, given that he had devoured an unknown number of novels; expecting him to remember every detail was unrealistic. However, Sam had an unusual ability: he only remembered crucial plot points from these novels right before they were about to happen, or when his power increased, or in worst case: It's ominous direction. This power helped him to restore his previous life memories.
What Sam couldn't recall, to his frustration, was what someone could do with the items from the forbidden land. It was a gaping hole in his knowledge, and it gnawed at him.