How to use curl in linux

15 Tips On How to Use ‘Curl’ Command in Linux

Back in the mid-1990’s when the Internet was still in its infancy, a Swedish programmer named Daniel Stenberg started a project that eventually grew into what we know as curl today.

Initially, he aimed at developing a bot that would download currency exchange rates from a web page periodically and would provide Swedish Kronor equivalents in US dollars to IRC users.

Long story short, the project thrived, adding several protocols and features along the way – and the rest is history. Now let’s dive in with both feet and learn how to use curl to transfer data and more in Linux!

We have put together the following list of 15 curl commands for you.

1. View curl Version

The -V or —version options will not only return the version, but also the supported protocols and features in your current version.

2. Download a File

If you want to download a file, you can use curl with the -O or -o options. The former will save the file in the current working directory with the same name as in the remote location, whereas the latter allows you to specify a different filename and/or location.

3. Resume an Interrupted Download

If a download was interrupted for some reason (for example, using Ctrl + c ), you can resume it very easily. The use of -C – (dash C, space dash) tells curl to resume the download beginning where it left off.

Download File Using Curl Command

4. Download Multiple Files

With the following command you will download info.html and about.html from http://yoursite.com and http://mysite.com, respectively, in one go.

5. Download URLs From a File

If you combine curl with xargs, you can download files from a list of URLs in a file.

Download Multiple Files with Curl

6. Use a Proxy with or without Authentication

If you are behind a proxy server listening on port 8080 at proxy.yourdomain.com, do.

where you can skip -U user:password if your proxy does not require authentication.

7. Query HTTP Headers

HTTP headers allow the remote web server to send additional information about itself along with the actual request. This provides the client with details on how the request is being handled.

To query the HTTP headers from a website, do:

Curl Query HTTP Headers

This information is also available in your browser’s developer tools.

8. Make a POST request with Parameters

The following command will send the firstName and lastName parameters, along with their corresponding values, to https://yourdomain.com/info.php.

You can use this tip to simulate the behavior of a regular HTML form.

9. Download Files from an FTP Server with or without Authentication

If a remote FTP server is expecting connections at ftp://yourftpserver, the following command will download yourfile.tar.gz in the current working directory.

where you can skip -u username:password if the FTP server allows anonymous logins.

10. Upload Files to an FTP server with or without Authentication

To upload a local file named mylocalfile.tar.gz to ftp://yourftpserver using curl, do:

11. Specify User Agent

The user agent is part of the information that is sent along with an HTTP request. This indicates which browser the client used to make the request. Let’s see what our current curl version uses as default, and let’s change it later to “I am a new web browser”:

Curl Check User Agent

12. Store Website Cookies

Want to see which cookies are downloaded to your computer when you browse to https://www.cnn.com? Use the following command to save them to cnncookies.txt. You can then use cat command to view the file.

Curl Store Website Cookies

13. Send Website Cookies

You can use the cookies retrieved in the last tip in subsequent requests to the same site.

14. Modify Name Resolution

If you’re a web developer and want to test a local version of yourdomain.com before pushing it live, you can make curl resolve http://www.yourdomain.com to your localhost like so:

Thus, the query to http://www.yourdomain.com will tell curl to request the site from localhost instead of using DNS or the /etc/hosts file.

15. Limit Download Rate

To prevent curl from hosing your bandwidth, you can limit the download rate to 100 KB/s as follows.

Summary

In this article we have shared a brief history of the origins of curl and explained how to use it through 15 practical examples.

Do you know of any other curl commands that we may have missed in this article? Feel free to share them with our community in the comments! Also, if you have questions feel free to let us know. We look forward to hearing from you!

If You Appreciate What We Do Here On TecMint, You Should Consider:

TecMint is the fastest growing and most trusted community site for any kind of Linux Articles, Guides and Books on the web. Millions of people visit TecMint! to search or browse the thousands of published articles available FREELY to all.

If you like what you are reading, please consider buying us a coffee ( or 2 ) as a token of appreciation.

We are thankful for your never ending support.

Источник

curl command in Linux with Examples

curl is a command line tool to transfer data to or from a server, using any of the supported protocols (HTTP, FTP, IMAP, POP3, SCP, SFTP, SMTP, TFTP, TELNET, LDAP or FILE). curl is powered by Libcurl. This tool is preferred for automation, since it is designed to work without user interaction. curl can transfer multiple file at once.

Syntax:

URL : The most basic uses of curl is typing the command followed by the URL.

This should display the content of the URL on the terminal. The URL syntax is protocol dependent and multiple URLs can be written as sets like:

URLs with numeric sequence series can be written as:

Progress Meter: curl displays a progress meter during use to indicate the transfer rate, amount of data transferred, time left etc.

If you like a progress bar instead of meter, you can use the -# option as in the example above, or –silent if you want to disable it completely.

Example:

Options:

-o : saves the downloaded file on the local machine with the name provided in the parameters.

Syntax:

Example:

Output:

The above example downloads the file from FTP server and saves it with the name hello.zip.
-O : This option downloads the file and saves it with the same name as in the URL.

Syntax:

Example:

Output:

-C – : This option resumes download which has been stopped due to some reason. This is useful when downloading large files and was interrupted.

Syntax:

Example:

–limit-rate : This option limits the upper bound of the rate of data transfer and keeps it around the given value in bytes.

Syntax:

Example:

Output:

The command limits the download to 1000K bytes.
-u : curl also provides options to download files from user authenticated FTP servers.

Syntax:

Example:

Example:

-T : This option helps to upload a file to the FTP server.

Syntax:

If you want to append a already existing FTP file you can use the -a or –append option.
–libcurl :This option is very useful from a developers perspective. If this option is appended to any cURL command, it outputs the C source code that uses libcurl for the specified option. It is the code similar to the command line implementation.

Syntax:

Example:

Output:

The above example downloads the HTML and saves it into log.html and the code in code.c file. The next command shows the first 30 lines of the code.
-x, –proxy : curl also lets us use a proxy to access the URL.

Syntax:

If the proxy requires authentication, it can be used with the command:
Sending mail : As curl can transfer data over different protocols, including SMTP, we can use curl to send mails.

Syntax:

curl –url [SMTP URL] –mail-from [sender_mail] –mail-rcpt [receiver_mail] -n –ssl-reqd -u : -T [Mail text file]

Syntax:

Example:

Output:

Note: There are a number of other options provided by cURL which can be checked on the man page. The Libcurl library has been ported into various programming languages. It’s advisable to visit the individual project site for documentation.

Источник

How to Use cURL Command in Linux

No matter what you use your computer for, there has probably been at least one occasion where you needed to download something, and opening a browser felt like overkill. This would be a great use case for cURL.

As the name suggests, cURL is a command-line tool for transferring data with URLs. One of the simplest uses is to download a file via the command line. This is deceptive, however, as cURL is an incredibly powerful tool depending on how you use it. Even if you’re somewhat familiar with the command, you probably aren’t using it to its full potential.

Basic cURL Functionality

One of the most basic things you can do with cURL is download a webpage or file. To do this you just use the curl command followed by a URL. Here’s an example:

Most of the time, using the command this way will get you a terminal full of raw HTML at best and a wash of unrecognizable characters at worst. If you would rather save it to a file, you can use standard Unix-style redirects to do so.

Follow HTTP Headers

Your browser often fixes this for you, but the Internet is very specific. When you type in a URL, there’s a good chance that you may be redirected one or more times before you reach your destination.

Say for example that you’re trying to reach the Make Tech Easier website. Typing the following will simply get you a redirect notice:

You can follow these HTTP location headers by using the -L flag like so:

This won’t look great in your terminal, but it’s a good option to know.

Save cURL Results to a File

There are a few ways to save a URL’s contents to a file. The -o option lets you decide the filename, while the -O option uses the filename in the URL to save. To choose your own file, use the following option:

More often than not, you’ll want to save a file to the same name it uses on the server. For that, use the -O option.

Download Multiple Files at Once

If you need to download a few files at the same time, cURL makes that easy to do. You’ll typically want to use this with the -O option.

When you download this way, cURL will try to re-use the connection instead of making new connections each time.

Continue a Stopped Download

It’s never fun when a download stops halfway through. Fortunately, cURL makes it easy to resume a download without having to start over. The syntax is somewhat strange, as you need to add -C — to your command.

Say you started a download with the following:

Then you stopped it by pressing Ctrl + C . You can resume it with the following command:

Use Basic HTTP Authentication

This won’t work for everything that requires a username and password, but if a server uses basic HTTP authentication, cURL can work with it. To download a file with username/password authentication, you can use the following:

This also works with FTP servers, as cURL supports a ton of different protocols.

Conclusion

There’s just a whole lot you can do with cURL. Sometimes all this functionality might feel like too much. If cURL feels too feature-rich and arcane to you, there is a simpler alternative: GNU wget.

While cURL gives you all the options you could possibly want, wget aims to provide good default options for you. Not sure if this is what you’re looking for? Don’t worry, we’ve got a detailed comparison of cURL and wget that should help you figure out which is the right one for you.

Kris Wouk is a writer, musician, and whatever it’s called when someone makes videos for the web.

Источник