His first, The Paul Simon Songbook, was released in the UK in 1965, early in his career; it featured songs like "I Am a Rock" and "The Sound of Silence" that would later be rerecorded for Simon and Garfunkel albums.
After his success with Garfunkel and the subsequent dissolution of their partnership, Simon released Paul Simon featuring "Me and Julio Down By the Schoolyard" among other songs.
So, for most Americans, There Goes Rhymin' Simon was the second solo album.
Since Graceland and The Rhythm of the Saints, Simon has been known as a bit of a musical explorer, finding rhythms and textures from other cultures and incorporating them into his own musical stew, already filled with 50s Doo-wop, the Everly Brothers, and Greenwich Village coffeehouse folk. There Goes Rhymin' Simon shows how this sort of musical borrowing and recombining has always been part of Simon's songwriting method.
The first song he recorded for the album ended up being the opening of the side 2, "American Tune," a State of the Union type address set to a melody by Bach:
Reportedly, he wanted to play with the same black players who backed the Staples Singers' on "I'll Take You There":
Anyway, Al Bell at Stax told Paul that he could get him the same musicians that the Staples family used, but that the guys were "mighty pale."
Here's a trailer for a documentary about Muscle Shoals:
Now Muscle Shoals has got the Swampers;
They've been known to pick a song or two.
Lord they get me off so much.
They pick me up when I'm feeling blue
Now how about you?
Admit it, you always wondered what they heck that was about. You're welcome.
Anyway, those were the side players who backed Simon on "Kodachrome" and "Love Me Like a Rock."
"Kodachrome" and "Love Me Like a Rock" were both released as singles and both got to #2 on the charts.
"Kodachrome" was kept from the top spot by Billy Preston:
Anyway, the point is, Paul Simon has always been searching. Just as Bob Dylan adapted Woody Guthrie, Simon has always been exploring different musical genres and writing songs in different "voices."