26

I have just migrated to a UNIX workstation. My memory of Bash shell scripts has faded since school and I find the syntax to be highly confusing. I am wondering what other scripting languages are used to automate tasks. The two most popular ones that I have heard of are Perl and Python.

  1. Which scripting language is most widely used by real world shops ?
  2. Which scripting language most closely resembles C/C++ syntax ?
  3. Is there another scripting language that I am not aware of ?
3
  • 2
    This should probably be a community wiki.
    – EBGreen
    Sep 4, 2009 at 15:38
  • 2
    I actually use PHP now for most system tasks. I'll put together a proper answer with this soon. It has lots of libraries built in, which is handy.
    – Kyle
    Sep 4, 2009 at 16:15
  • Shell-intended languages do something that other languages might not; Which is manage and keep in check all children processes recursively ever spawned. The blog advertising Phusion Passenger explains why in detail: phusion.github.io/baseimage-docker In short, your scripts written in lang-x probably do not handle shutdown routines properly. Which is perfectly okay so long as you run them as children of a real shell process. Disclaimer, I am no expert. This information is only to the best of my understanding which may be flawed. Nov 12, 2014 at 23:39

15 Answers 15

15

To some degree it's a matter of personal preference. Perl and Python are definitely up there in terms of popularity.

To answer two of your three questions, though:

  1. I would guess that bash is most widely used, probably followed by perl. Just guessing though, dunno if anyone has done a survey. :)
  2. There are bazillions of scripting languages out there. :)

My recommendation is Python though. It's easy to read and write, immensely powerful, and there are tons of useful resources on the web for learning it, not to mention code that you can copy and re-purpose.

3
  • 2
    I second that. I find Python easy to learn and able to do anything I throw at it as it's been in good old OS/2 days with REXX.
    – slovon
    Sep 4, 2009 at 15:47
  • 4
    +1 for python...the power of C in a scripting language. You can make your scripts as simple or as complex as you want ;)
    – KFro
    Sep 4, 2009 at 23:01
  • 2
    on non NT systems the python module sh makes using python as a shell scripting language accessible; However if you run into edge cases (like multiprocessing) you'll have to learn the subprocess module which I to this day find too hard to just use. Nov 12, 2014 at 23:20
7

Perl and Python are the obviously answers; but each of them is used to complete different tasks:

  1. Bash - Hands down the easiest to learn. With Bash scripts you run commands and manipulate the output.
  2. Python - Second easiest to learn. Python is way more useful than Bash since you can use Python to write full programs, and it's well supported by the community. Python is also extremely readable and easy to edit months later.
  3. Perl - The best of the three (for scripting), but easily the steepest learning curve. Has very good regular expression support (which is the main reason it's still in use today), and you'll probably run into a lot of Perl scripts over time. The main problems with Perl is code readability -- months after you write code you won't be able to figure out what it does. So you'd better comment heavily.
  4. Awk/Sed - Both very useful, but both very hard to learn. I'd suggest tackling these last, unless you're good at patterns.

Concerning which one to learn right now (since you presumably know none of them), I'd learn Python. It's the simplest and most useful to know.

And as for the answers to your questions:

Which scripting language is most widely used by real world shops ?

All of them, but I'd imagine that Perl has a little bit of an advantage here (being that it's been in use in system administration for much longer).

Which scripting language most closely resembles C/C++ syntax ?

Perl

Is there another scripting language that I am not aware of ?

Ruby, Tcsh/Csh, http://en.wikipedia.org/wiki/Interpreted_language

6
  • 3
    If you put some time into perl, you can write perfectly readable perl. People who write spaghetti code are too quick to blame Perl for their failures.
    – Cian
    Sep 4, 2009 at 16:10
  • 2
    I'm only speaking based on experiences as a sysadmin; I've had to maintain code other people have written that has been damn near impossible to read. Usually I end up just rewriting the code in Python. Sep 4, 2009 at 16:25
  • i personally find Perl 10x easier to learn than Python. so i guess it's a matter of taste..
    – olivierg
    Oct 11, 2020 at 16:20
  • 3
    Bash - Hands down the easiest to learn. With Bash scripts you run commands and manipulate the output. I've been scripting with Bash for 20+ years and I'm still horrified at code I wrote 6 months ago. If you think Bash is easy, you don't know Bash. This is the most error-prone thing you could pick to execute a deployment.
    – jcayzac
    Jul 13, 2021 at 5:02
  • Bash - Hands down the easiest to learn. I fell off my chair reading that. Literally any language I ever came into contact with is easier than bash. For anyone who thinks bash is easy - this SO thread will cure you forever stackoverflow.com/q/2953646/1097451 Jun 8, 2022 at 17:35
3

awk for small jobs, python for more-than-oneliners.

Python (if you don't use specific APIs and other platform dependent things) has a rich "standard library" which is available on all platforms where you get the python version in use. This is very nice as you can re-use your scripts easily and means your knowledge is a good investment even if you change your platform.

In our company we use python for all scripting, monitoring, ... and java for The application, and we're well off in heterogeneous environment.

2

Sh (bourne shell), grep, awk and sed is always a good thing to get to grips again. Perl is excellent if you wan't a lot of text regexing and maintainability is not that important (not saying that it is not possible). Python; if readability is important to you. My preference is Python but all these tools are good for most thing you throw at it. You might want to reconsider you c/c++ syntax requirement, keeping stuck on it won't help you in the long run.

2
  • Thanks Martin. I've noticed (from samples on the web), that Perl bears a remarkable resemblance to PHP (although I'm sure it was Perl before PHP, historically). Is this a fair statement ? Sep 4, 2009 at 15:55
  • PHP 1.0 was a perl script, so it's not coincidence :)
    – Bill Weiss
    Sep 4, 2009 at 21:43
2
  • Ruby
  • Python
  • Go - a cross-platform language, scripting is mentioned as one of Go's strengths.
  • Node.js - see Grunt.js, Gulp.js. Many projects are now using Node.js as a shell automation tool.
  • Perl
2

To add yet another interesting option to the pile check out Closh

https://github.com/dundalek/closh

It's a bash replacement that runs full Clojure(Script). You can use all the concise and powerful tools of Clojure on your command line. It's still a young project but quite an interesting and appealing alternative to the ..unkindness that is bash syntax.

0
1

I would say that perl and python are probably the two best alternatives. You can do a lot with sed and awk as well.

1

If you are looking to learn something from scratch, it's probably best to (re-)learn bash.

Beyond that, perl and python are popular these days; I'm partial to perl, but my perl scripting style hasn't changed much since Perl 4 was out.

I'm not aware of any "c-syntax-like" scripting languages.

1

Perl, hands down, if only for the incredible resource of CPAN (http://search.cpan.org/). The benefit of using a scripting language with such a massive contributing community is that you are unlikely to ever need to invent something yourself.

Perl also has the benefit of being as complex as you need for the current situation. You can treat it as a simple scripting language for quick administration or single purpose scripts, but you can use object oriented concepts or extensive meta object systems when you want to create larger, easy to maintain applications (like daemons, servers, clients).

The differences among the scripting languages alone aren't severe enough to pick a clear winner, but you should primarily look at how easy it will be for you to find information on topics you care about.

Perl syntax can also be strikingly close to C, as long as you can get around the lack of typing and the sigils; advanced perl use would come naturally as you eventually learn the strengths of the language.

And for naysayers on the readability front: you can write hard to read code in any language. If you look back at your code from 4 months ago and can't understand it, you're doing something wrong.

p.s. This post was filled with links to various administration friendly modules on CPAN, but apparently new users can only post one link, so use that CPAN link to search for things like 'CVS' 'SVN' 'Cron' and 'Moose' (an extensive object system)

1
  1. most used: Perl, mostly for historical reasons. Python is widely available in all Linux and BSD distros (it's even installed on Mac OSX), and is far nicer to learn/program. Of course, once you get the syntax, Bash is simpler for almost anything that you could do at the command line. After all, it is the command line.

    • C syntax: there's csh, but you can easily install TCC and use real C as scripting language, compiled from source in less time than other languages startup.

    • other scripting languages: lots! Lua, Ruby, JavaScript...

1

You should be using a combination of all of those where it is appropriate. I really dislike perl ( readability ), but it's good for a lot of things. Python on the other hand was something new to me in terms of system administration, until i had to manage ( and still have to ) a few KVM machines. Starting, stoping, saving, loading, migrating all those machines via script became reality after 1 hour of messing around with libvirt's python bindings.

Last but not least i wouldn't use perl or python if i have to , let's say, dump all databases from a mysql server every night ( each database in it's own dump file ), tar them and rsync that to some other server. Why ? Because it's faster and easier to write it in bash :)

In the end, you should use whatever language fits your current task best and don't use only a given scripting language just because you like it most.

1

[for general scripting advice, see the other answers]

This answer is about use of command-line programs and also large chains of them.

For running chains of CLI programs, and perhaps many chains in sequence, bash is the easiest. Perl can do the same but the language is much harder to learn and is far more complex, and as many have pointed out, it's easy to write unintelligible Perl.

If you are iterating over files and parameters, bash is easiest to get going because you can prototype in the shell, then copy into a script and add some arrays and loops with minimal overhead.

Examples of what I mean:

$ cat input.dat | numerical_model source_data ${SRC} grid_size ${GRID_SIZE} param1 ${P1} param2 ${P2} | tee ${OUTPUT} | plotter title ${TITLE} ylabel ${YLABEL} xlabel ${XLABEL} > ${OUTPUT_FIG}
$ cat ${OUTPUT} | stats_cmd bin_size ${BINSIZE} dist_type ${DIST_TYPE} | tee ${OUTPUT_STATS} | plotter plot_type ${PLOT_TYPE} title ${TITLE_STATS} > ${OUTPUT_FIG_STATS}

In bash, it is easy to wrap that in a few loops over some of those parameters. However, the code becomes hard to read once the script is more than a few tens of lines. ---imagine 30 sets of commands (a few lines each) for thirty different data plots---

Perl and Python could be better for large scripts because they have cleaner syntax for looping and variables, but the bash commands must be generated as strings, run as subprocesses, and stdin,stdout captured. It can be done, but it cannot fully replace the native shell environment.

If shells had a better language this would all be avoided.

0

I like ruby. It's simple and easy to program with. There's an excellent book on system administration using ruby.

Ruby is used in several configuration management systems (capistrano, chef, puppet) and last version of metasploit.

You can use rush, a replacement for the Unix shell with ruby syntax and do things like this:

local = Rush::Box.new('localhost') 
remote = Rush::Box.new('my.remote.server.com')     
local_dir = local['/Users/adam/myproj/']
remote_dir = remote['/home/myproj/app/']

local_dir.copy_to remote_dir
remote_dir['**/.svn/'].each { |d| d.destroy }
3
  • Thanks chmeee. I had no idea you could use Ruby for sysadmin - I always thought it was web-centric. I'm looking for something in use at most organizations. Is it fair to say that Ruby for sysadmin is still too bleeding edge ? Sep 4, 2009 at 15:58
  • 1
    Shell/Perl are the only two languages that you can reliably find installed everywhere. Python's probably the next most common, with ruby running a very distant fourth (actually, probably fifth, behind awk).
    – Cian
    Sep 4, 2009 at 16:12
  • @ScottDavies PHP and ASP.net are the only web-centric languages I know of; even Javascript designed to run exclusively on the web has matured past being the lackie for form validation. However you can dispatch html with any language, even bash apparently. Nov 12, 2014 at 23:24
0

Ruby one-liners example:

> echo 'Shell is the best!' | ruby -pe 'gsub("Shell", "Ruby")' 

Ruby is the best!

> printf "foo\nbar\nbaz\n" |  ruby -ne 'BEGIN { i = 1 }; puts "#{i} #{$_}"; i += 1'

1 foo
2 bar
3 baz

I'd use ruby.

Compare to sh, it is modern and powerful.

Compare to python, it is consistency, one edition, one package manager. And I don't like indentation syntax.

Compare to Perl, a little bit of old-school and the syntax is too complex for me.

0

To throw another language into the ring, I'd highly suggest Raku. Raku (originally Perl 6) was meant to be the next version of the Perl language, but ended up splitting off due to some heavy redesigns intended to make it more modern.

A few changes:

  • It uses . instead of ->
  • The language has some of the stranger whitespace layouts removed
  • Several implementation tweaks to make the language saner

https://raku.org

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .