Wget is a versatile command line tool.  It is a free utility for non-interactive download of files from the Web.  It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies. One of the often made uses of wget is to download Linux distribution ISOs from the web.

Wget is non-interactive, meaning that it can work in the background, while the user is not logged on.  This allows you to start a retrieval and disconnect from the system, letting wget finish the work.  By contrast, most of the Web browsers require constant user’s presence, which can be a great hindrance when transferring a lot of data.

Wget has numerous switches and employing them in numerous combination makes it possible to make powerful use of this versatile tool.

Here is how you can use wget to download an entire website.

$ wget --recursive \
       --no-clobber \
       --page-requisites \
       --html-extension \
       --convert-links \
       --restrict-file-names=windows \
       --domains mywebsite.com \
       --no-parent www.mywebsite.com/howtos/

Let’s look at the switches / options used with wget in the preceding command one by one.

• --recursive – Download the entire website.
• --no-clobber -  Don’t overwrite any existing files. Used in case of interrupted and resumed downloads.
• --page-requisites -  Get all the elements that compose the page such as JavaScript, CSS and so on.
• --html-extension – Save files with html extension.
• --convert-links -  Convert links so that they work locally, offline.
• --restrict-file-names=windows – Modify file names so that they will work in Windows as well.
• --domains mywebsite.com – Don’t follow links outside this domain.
• --no-parent -  Don’t follow links outside the directory howtos/.