A quick update as I've been messing about with this tonight.
What I have been doing so far is putting my tracks in folders as such:
Artist folder ----- Album Folder(s) ----- tracks.
So I've maybe had 3 or 4 songs from each artist that have all come from different albums so each artist could have around 5 folders.
I had 400 different artists on my SD card so in theory could have had around 2000 folders!!!!
After RTFM

it states
"Memory cards with max 500 folders and max 2048 files" so that was probably the reason for the lengthy (20 minute delay)
I deleted all but 100 artists that I knew only had 1 album folder so I now had a maximum 200 folders. Plugged it into the RCD510 and it only took around 1 minute to access.
However I thought if I went up a selection or two I would get access to just a list of songs, the way I do with my ipod but it would only go as far back as "Artists" (which I can understand why now). Also doing it the way I had done it meant the songs would only play from that particular artist i.e. it was stuck in the Artist---Album---track and to move on to another song, I had to manually go up a selection or two to get another artist.
So I think after all that I have sussed it

However is there a way I can sort by artist or song the way I can with the ipod or am I best just to have no folders for artists that just have 1 or 2 songs?
How do folk arrange theirs?
Incidentally, I plugged in my card reader with the SDHC in through the MDI port and rather than take a minute to access it like the RCD510, it was accessed straight away!!!