38

I am looking for a tool to test a website from a Linux command line.

From the output, I need to know the http response (status codes) but also benchmark the time it takes to download the different elements of the site.

Thank you in advance.

2
  • What have you found, looked at, or are considering (suitable / not suitable) so far?
    – mctylr
    Mar 22, 2010 at 16:04
  • None of these answers sound very good to me. You still have to wait ages for a negative
    – geotheory
    Nov 5, 2015 at 23:27

10 Answers 10

46

You can try wget with -p option:

wget -p http://site.com

It will tell you how long it takes to download each element and the return codes for each request.

4
  • 6
    wget -p URL -O /dev/null Mar 30, 2016 at 10:43
  • From manual: p, --page-requisites: get all images, etc. needed to display HTML page
    – Pablo A
    Feb 10, 2023 at 21:20
  • 1
    Use the command from @Fedir RYKHTIK in order to not pollute your current dir Apr 5, 2023 at 15:00
  • You can get just the Total wall clock time by appending stderr to stdout 2>&1 and grepping it: wget --page-requisites --output-document /dev/null URL 2>&1 | grep Total
    – Pound Hash
    Sep 3, 2023 at 23:26
9

Please see Apache Benchmark:

Apache Benchmark

This should give you an overview of your page's performance.

4
  • Apache benckmark cannot be use for this since it is an external source, in fact it is youtube video playlist. We are monitoring the access to this ressource. TY
    – Embreau
    Mar 22, 2010 at 14:42
  • I don't see why you can't use ab; like wget in the next answer, it will work as long as the URL of your site is accessible from the machine where you're runinng the benchmarking tool. Mar 22, 2010 at 14:46
  • Apache benchmark is not restricted to local resources, it's meant to be a full performance measuring tool (including network latency, i/o, etc).
    – andre
    Mar 22, 2010 at 14:52
  • Good to know, I will evaluate this option, thank you.
    – Embreau
    Mar 22, 2010 at 16:31
8

You may want to look at the following options of curl:

  • --write-out - displays any of several time-related variables
  • --trace-time - Prepends a time stamp to each trace or verbose line
  • --verbose
  • --include - (HTTP) Include the HTTP-header in the output.
  • --trace-ascii <file> - Enables a full trace dump of all incoming and outgoing data, including descriptive information

And the following option of wget:

  • --timestamping - Turn on time-stamping
1
  • curl --trace-ascii <file> --trace-time -o /dev/null <url> was exactly what I needed. Thanks!
    – dsummersl
    May 17, 2013 at 14:21
3

Selenium and Curl are good options depending on what your goal is. Also, a utility that I've come to like quite a bit is twill. More information is available at http://twill.idyll.org/.

It's nice as it has it's own little specialized language for filling out forms, validating links, and checking response codes. Since it's just Python code, you can easily import the libraries and automate your tests yourself if you'd like to do something different.

2

Use curl to get the header for the page, and time the process:

time curl -I http://yourpage.com | grep HTTP

wrap that in a while loop and you're good to go. The same way you can check for all elements if you know the URL.

1

try a commandline tool called 'siege' as instructed here

1
  • 6
    That's not what he's asking for. and you're also 3 years late =)
    – mveroone
    Sep 3, 2013 at 12:15
1

What tool you choose depends on what you want to measure and the complexity of the site.

If the behaviour of the site is dependent on cookies (e.g. user needs to login) then ab / curl / wget (described in other answers) will not suffice. One solution is to use http::recorder / www::mechanize.

All the data you are asking for is in your webserver logs - and a simple awk script will return it in a more readable form.

benchmark the time it takes to download the different elements of the site.

This is a very poor indicator of performance (although it is useful for monitoring the health of a production system). With the exception of large/slow resources such as bulky reports, iso images, multimedia files the perception of performance has got very little to do with the time taken to process a single request - and its really difficult to measure this accurately (simply adding %D to your apache log appears to solve the problem but ignores TCP handshakes, SSL negotiation, caching effects, DNS lookup times).

A better solution is to use something like Boomerang - but that runs in a Javascript capable browser. While this gives a better indicator of perceived performance than tracking individual HTTP requests, it relies on browser events to derive a value for the performance - but peceived performance is all about the time taken for the viewport to render (again there are tools for this - have a look at the filmstrip tools in WebPageTest).

There is also the argument about measuring the performance actually delivered to users of the site (RUM) vs synthetic testing.

0

If you are going to need something bigger then curl and/or wget, there is also selenium

0

I think for running performance test you can try JMeter. You can record your test using the built in proxy. It also runs in text mode, local or distributed. You can save your results in csv or xml format. If using xml format you can also store the content of the page.

0

For checking headers, I like httpie (docs).

Installation

pip install httpie --user

Usage

$ http -h http://serverfault.com/q/124952/113899
HTTP/1.1 302 Found
Accept-Ranges: bytes
Age: 0
Cache-Control: private
Connection: keep-alive
Content-Length: 198
Content-Type: text/html; charset=utf-8
Date: Fri, 06 Jan 2017 10:01:06 GMT
Location: http://serverfault.com/questions/124952/testing-a-website-from-linux-command-line
Set-Cookie: prov=392298d9-103e-7dfc-5b55-8738be46cf3b; domain=.serverfault.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly
Via: 1.1 varnish
X-Cache: MISS
X-Cache-Hits: 0
X-DNS-Prefetch-Control: off
X-Frame-Options: SAMEORIGIN
X-Request-Guid: 07a25de4-6e16-4aa8-acfc-12b1bbbc6180
X-Served-By: cache-hhn1543-HHN
X-Timer: S1483696865.976259,VS0,VE187

Not the answer you're looking for? Browse other questions tagged .