To get hold of them legally you are going to browse for them in online webstores. Most of the songs were originally issued as 'B'-sides on CD hit single releases, but I guess most of the songs can now be downloaded separately.

It probably doesn't require much of an effort to find the whole batch of b-sides, remixes and rarities on less legal online servers by simply Googling for them.

Another (less legal) option is to use a tool like YouTube downloader or Freemake Video Downloader to store the sound/video from YT or other tube sites on your local hard drive. If the song is in decent quality you might go with that.

The typical problem with purchasing all the songs legally is usually that they aren't always available as separate tracks for download. Plus sometimes the songs are copy-protected. Not all websites sell songs for keeps, unfortunately. I can't stand that.

(No offensive text in this signature field from now on)