November 2012: Joomla 2.5.8 to 3.0: Essentially go to Control Panel / Joomla Update / Options / Update Server / Short Term Support
The steps below work for Joomla 1.7.0 to 1.7.1 to 1.7.3 to 2.5.0 to 2.5.1 to 2.5.x to whatever:
On Joomla's backend (login as administrator in http://www.joomlasite.com/administrator), click on Extensions / Extension Manager
Click on Update
On the top right of the Update tab, click on "Find Updates"
After a few seconds, the available update will appear. Check the box on its left.
Click on "Update" on the top right.
If everything goes ok, a successful message appears.
Alternative to all of this is to upload the upgrade package file taken from http://www.joomla.org/download.html (e.g., Joomla_2.5.x_to_2.5.7-Stable-Patch_Package.zip) through Extensions / Extension Manager / Install.
Check for Errors in Extension Manager
Always check for Errors in "Extensions / Extension Manager / Warnings" after updates.
Error Copy failed while upgrading
While updating Joomla, the error "-1 - An error has occurred. Copy failed" may appear. I had modified a file in one of the templates folders and left it with the wrong owner. Resetting the owner to the web server user (on the console, chown apache. /path/filename) did the trick.
Error SQL DB function failed with error number 1061 etc while upgrading
Error message: JInstaller: :Install: Error SQL DB function failed with error number 1061