At a modern-day nursing home, an elderly man named Duke (James Garner) begins to read a romantic story from his notebook to a fellow patient (Gena Rowlands).
The story he tells begins in 1940. In Seabrook Island, South Carolina, local country boy Noah Calhoun (Ryan Gosling) is smitten with seventeen-year-old heiress Allie Hamilton (Rachel McAdams) after seeing her at a carnival, and they share an idyllic summer love affair. Noah takes Allie to an abandoned house, which he explains that he intends to buy for them. Later that evening, she asks him to make love to her, but they are interrupted by Noah's friend Fin (Kevin Connolly) with the news that Allie's parents have the police out looking for her. When Allie and Noah return to her parents' mansion, they ban her from seeing Noah, whom they called "trash, trash, trash". The two break up and the next morning, Allie's mother announces that the family is returning home to Charleston.
Noah and Allie have no choice but to move on with their lives; Noah and Fin enlist to fight in World War II and Fin is killed in battle. Allie volunteers in a hospital for wounded soldiers, where she meets an officer named Lon Hammond, Jr. (James Marsden), a young lawyer who is handsome, sophisticated, charming and comes from old Southern money. The two eventually become engaged, to the delight of Allie's parents, but Allie sees Noah's face when Lon asks her to marry him.
When Noah returns home from the war, he discovers his father has sold their home so that Noah can buy the abandoned house, fulfilling his lifelong dream to buy it for the departed Allie, whom by now he hasn't seen for several years. While visiting Charleston, Noah witnesses Allie and Lon kissing at a restaurant; he convinces himself that if he restores the house, Allie will come back to him. Later, Allie is startled to read in the newspaper that Noah has completed the house, and she visits him in Seabrook.
In the present, it is made clear that the elderly woman is in fact Allie, who is suffering from dementia and cannot remember any of the events being read to her. Duke, the man who is reading to her, is her husband, but Allie cannot recognize him.
Back in the forties, Allie returns to Seabrook to find Noah living in the restored house. The two renew their relationship and make love. In the morning, Allie's mother appears on Noah's doorstep, warning Allie that a jealous Lon has followed her to Seabrook. Allie confesses to Lon that she has been spending time with Noah. He is upset but says that he still loves her. Allie tells him she knows she should be with him, but she remains indecisive.
In the present, Allie briefly becomes lucid. She remembers that the story Duke is reading is the story of how they met. Duke tells her how she appeared at Noah's doorstep, having left Lon at the hotel, and Allie suddenly remembers her past. When she had found out about her illness, she herself wrote their story in the notebook with instructions for Noah to "read this to me, and I'll come back to you." But Allie soon relapses, losing her memories of Noah. She panics, not understanding who he is, and has to be sedated. Duke - who it is in fact Noah, aged - is hospitalized with what seems to be another heart attack.
When he is released from the hospital, Noah visits Allie to find her lucid again. Allie questions Noah about what will happen to them when she loses her memory completely and he reassures her that he will never leave her. She asks him if he thinks their love for each other is strong enough to "take them away together"; he replies that he thinks their love could do anything. After telling each other that they love one another, they both go to sleep in Allie's bed. The next morning a nurse finds that they have died peacefully together.