fir3net
PPS-Firenetbanner-780.5x190-30-03-17

Useful Curl Examples

Curl is a tool to transfer data from or to a server, using one of the supported protocols (HTTP, HTTPS, FTP, FTPS, TFTP, DICT,  TELNET, LDAP or FILE) and is designed to work without user interaction.

Below shows some useful examples:

Total Retrieval Time

[root@webserver1 ~]# curl -s -w "%{time_total}\n" -o /dev/null www.bbc.com
0.196

Host Header

[root@webserver1 ~]$ curl -IL -H "Host: fir3net.com" http://79.170.40.160
HTTP/1.1 200 OK
Date: Mon, 19 Sep 2011 19:12:38 GMT
Server: Apache/2
X-Powered-By: PHP/5.2.17
P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"
Expires: Mon, 1 Jan 2001 00:00:00 GMT
Cache-Control: post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: 76b8dc15e4f021ca24944dc631ff9=u1hgqoktq2ibq78adt6jrmack6; path=/
Last-Modified: Mon, 19 Sep 2011 19:12:38 GMT
Content-Type: text/html; charset=utf-8

Download Speed

[rick@webserver1 ~]$ echo "scale=2; `curl  --progress-bar -w "%{speed_download}" http://speedtest.wdc01.softlayer.com/downloads/test10.zip -o test.zip` / 131072" | bc | xargs -I {} echo {}Mb\/s
######################################################################## 100.0%
17.63Mb/s

Display Headers

[root@webserver1 ~]# curl -I -L http://www.bbc.com
HTTP/1.1 302 Found
Server: Apache
Content-Type: text/html; charset=iso-8859-1
Date: Tue, 26 Jul 2011 20:06:51 GMT
Location: http://www.bbc.co.uk
Transfer-Encoding: chunked
Connection: Keep-Alive
Set-Cookie: BBC-UID=249e925f01ee552b893f64c770301672967b38bd20d0b05cb299737f99e21c3f0curl%2f7%2e15%2e5%20%28i686%2dredhat%2dlinux%2dgnu%29%20libcurl%2f7%2e15%2e5%20OpenSSL%2f0%2e9%2e8b%20zlib%2f1%2e2%2e3%20libidn%2f0%2e6%2e5; expires=Sat, 25-Jul-15 20:06:51 GMT; path=/; domain=bbc.co.uk;

HTTP/1.1 200 OK
Server: Apache
Vary: X-Ip-is-advertise-combined
Cache-Control: max-age=60, private
Content-Type: text/html
Date: Tue, 26 Jul 2011 20:06:51 GMT
Transfer-Encoding: chunked
Etag: "1311710811"
X-Lb-Nocache: true
Connection: Keep-Alive
Set-Cookie: BBC-UID=a4beb26f01bec55b46058b8640b02ef80e704841100090dc0299e3afa9820c8f0curl%2f7%2e15%2e5%20%28i686%2dredhat%2dlinux%2dgnu%29%20libcurl%2f7%2e15%2e5%20OpenSSL%2f0%2e9%2e8b%20zlib%2f1%2e2%2e3%20libidn%2f0%2e6%2e5; expires=Sat, 25-Jul-15 20:06:51 GMT; path=/; domain=bbc.co.uk;
Age: 0

Cookies (save/send)

[root@webserver1 ~]$ curl -IL -c cookies.txt http://fir3net.com
HTTP/1.1 200 OK
Date: Tue, 04 Sep 2012 09:40:37 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.3.8
Set-Cookie: f71b6edf06ff1b3592582aad6e51abf7=2o7bssltlpli8grvva9a9qk0e2; path=/
P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"
Status: 200 OK
Expires: Mon, 1 Jan 2001 00:00:00 GMT
Last-Modified: Tue, 04 Sep 2012 09:40:38 GMT
Cache-Control: post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8

[root@webserver1 ~]$ curl -IL -b cookies.txt http://fir3net.com
HTTP/1.1 200 OK
Date: Tue, 04 Sep 2012 09:40:40 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.3.8
P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"
Status: 200 OK
Expires: Mon, 1 Jan 2001 00:00:00 GMT
Last-Modified: Tue, 04 Sep 2012 09:40:40 GMT
Cache-Control: post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8

TEST HTTP COMPRESSION

[root@webserver1 ~]$ curl  -s --write-out "size_download=%{size_download}\n" --output /dev/null http://www.fir3net.com
size_download=19753
[root@webserver1 ~]$ curl  --compress -s --write-out "size_download=%{size_download}\n" --output /dev/null http://www.fir3net.com
size_download=6054

* Note : The --compress option instructs the client to send the accept-encoding header with the HTTP request.

Tags: Curl

About the Author

RDonato

R Donato

Rick Donato is the Founder and Chief Editor of Fir3net.com. He currently works as a Principal Network Security Engineer and has a keen interest in automation and the cloud.

You can find Rick on Twitter @f3lix001