The song is "But Yours." It's the last song in the show, sung by the irresponsible, former drunk Sid (played by Jackie Gleason) who has finally decided to stop his wandering ways and settle down with Lilly (Eileen Herlie, here), because he realizes he's in love with her.
Sid, it turns out, just isn't really ready to settle down. He's been too much of a drifter all his life, And so, in the end, he realizes he has to leave this small Connecticut town. And everyone sees him off at the train station.
Hey, you've put up with all of these posting and songs of charming, wistful, evocative, lovely and fun Take Me Along, you deserve the finale. Here's how it ends.