Surprisingly, I found that when adding links to the sidebar, I found that there was nothing built in to WordPress to change the order that they are displayed in. Luckily, a plugin is available that will do just that. It’s called My Link Order and allows you to drag and drop the links into the order that you desire.
I did have a problem originally, in that I was rearranging the links but not seeing any difference in the links displayed on my site. I was missing something completely obvious – My Link Order only arranges the order of the links if you use the My Link Order widget! So removing the Links widget and replacing it with the My Link Order widget fixed things for me.