For Greatest Hits, Vol. 3, Billy Joel started where Greatest Hits, Vols. 1 & 2 left off, with songs from his An Innocent Man album, such as "Keeping the Faith" and the title track. There were just three additional studio albums to account for (The Bridge, Storm Front, and River of Dreams), and Joel decided to substitute some album tracks for the actual charting singles. Joel never cared for "Modern Woman" (a Top 10 hit from the soundtrack to Ruthless People), so he removed it from consideration. His version of Leonard Cohen's "Light as the Breeze" was for the 1995 Cohen tribute album Tower of Song, while two previously unreleased tracks—a cover of Bob Dylan's "To Make You Feel My Love" and Goffin & King's "Hey Girl," first recorded by Freddie Scott in 1963—fill out the album and serve as Joel's final recordings in the pop-rock arena.