I can run a script locally, but cannot do "ssh HOSTNAME /path/to/script.sh"

Question

I have a linux server, and a linux desktop.

I have written the following simple script to dump the database of a django web application:

#! /bin/bash
set -o errexit

cd $(dirname $0)

. virtualenv/bin/activate

cd mysite

export DJANGO_SETTINGS_MODULE="settings.my_hostname"
django-admin.py dumpdata --settings=$DJANGO_SETTINGS_MODULE > database.json

The programme django-admin.py requires the DJANGO_SETTINGS_MODULE environmental variable to work properly.

If I ssh into the machine, ssh HOSTNAME, and then run the script /var/www/example.com/dumper.sh from the bash terminal on the remote host, and everything works fine. I get no output (as expected), and the file database.json is there and has the right data.

However (on my linux desktop), I cannot run this command: "ssh HOSTNAME /var/www/example.com/dumper.sh`` and I get the following error:

Traceback (most recent call last):
  File "/var/www/example.com/virtualenv/bin/django-admin.py", line 5, in <module>
    management.execute_from_command_line()
  File "/var/www/example.com/virtualenv/lib/python2.6/site-packages/django/core/management/__init__.py", line 429, in execute_from_command_line
    utility.execute()
  File "/var/www/example.com/virtualenv/lib/python2.6/site-packages/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/var/www/example.com/virtualenv/lib/python2.6/site-packages/django/core/management/__init__.py", line 261, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "/var/www/example.com/virtualenv/lib/python2.6/site-packages/django/core/management/__init__.py", line 67, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File "/var/www/example.com/virtualenv/lib/python2.6/site-packages/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/var/www/example.com/virtualenv/lib/python2.6/site-packages/django/core/management/commands/dumpdata.py", line 4, in <module>
    from django.db import connections, router, DEFAULT_DB_ALIAS
  File "/var/www/example.com/virtualenv/lib/python2.6/site-packages/django/db/__init__.py", line 14, in <module>
    if not settings.DATABASES:
  File "/var/www/example.com/virtualenv/lib/python2.6/site-packages/django/utils/functional.py", line 276, in __getattr__
    self._setup()
  File "/var/www/example.com/virtualenv/lib/python2.6/site-packages/django/conf/__init__.py", line 42, in _setup
    self._wrapped = Settings(settings_module)
  File "/var/www/example.com/virtualenv/lib/python2.6/site-packages/django/conf/__init__.py", line 89, in __init__
    raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'settings.my_hostname' (Is it on sys.path?): No module named settings.my_hostname

It's as if the export command isn't being run, or isn't taking effect.

Why doesn't this work? (Or alternatively should it work? Am I mistaken in thinking that this should work?)

Update №1

On @faker's suggestion, I put echo $PATH in the script before the call to django-admin.py. The . virtualenv/bin/activate changes the shell path. I get a path of /var/www/example.com/virtualenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games when I run it via ssh HOSTNAME /path/to/script.sh, and after logging in I get /var/www/example.com/virtualenv/bin:/home/rory/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games. The only difference is the ~/bin part, but I did a which django-admin.py aswell, and in both instances it's using /var/www/example.com/virtualenv/bin/django-admin.py so in both instances it's using the same programme for the django-admin.py command.

Update №2

On @Andrew Schulman's advice of the same, but for $PYTHONPATH. PYTHONPATH was empty in both cases. However I added python -c 'import sys; print sys.path' instead of just echo $PYTHONPATH, and got different results.

When I ssh into HOSTNAME and run script manually (the one that works):

['', '/var/www/example.com/virtualenv/lib/python2.6/site-packages/distribute-0.6.10-py2.6.egg', '/var/www/example.com/virtualenv/lib/python2.6/site-packages/pip-1.0.2-py2.6.egg', '/var/www/example.com/mysite', '/home/rory/code/python/lib', '/var/www/example.com/virtualenv/lib/python2.6', '/var/www/example.com/virtualenv/lib/python2.6/plat-linux2', '/var/www/example.com/virtualenv/lib/python2.6/lib-tk', '/var/www/example.com/virtualenv/lib/python2.6/lib-old', '/var/www/example.com/virtualenv/lib/python2.6/lib-dynload', '/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '/var/www/example.com/virtualenv/lib/python2.6/site-packages']

When I invoke the script over ssh (this doesn't work):

['', '/var/www/example.com/virtualenv/lib/python2.6/site-packages/distribute-0.6.10-py2.6.egg', '/var/www/example.com/virtualenv/lib/python2.6/site-packages/pip-1.0.2-py2.6.egg', '/var/www/example.com/virtualenv/lib/python2.6', '/var/www/example.com/virtualenv/lib/python2.6/plat-linux2', '/var/www/example.com/virtualenv/lib/python2.6/lib-tk', '/var/www/example.com/virtualenv/lib/python2.6/lib-old', '/var/www/example.com/virtualenv/lib/python2.6/lib-dynload', '/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '/var/www/example.com/virtualenv/lib/python2.6/site-packages']

The working version has a /var/www/example.com/mysite. The settings file is located in /var/www/example.com/mysite/settings/my_hostname.py. This makes sense since the working invokcation can load the file.

Why isn't the 'remotely invoked' version getting that on it's python path?

Answer 1

Change your script and put -l after the shebang. so #!/bin/bash -l

Alternatively change your ssh command to ssh HOSTNAME bash -l /var/www/example.com/dumper.sh

Rationale: When you run ssh HOSTNAME command instead of just ssh HOSTNAME, the shell used to launch command is not a 'login' shell, and so different scripts get called (See the INVOCATION section of man bash) which results in your environment being setup differently.

Answer 2

I have discovered that you need to quote the bash line for it to recognize the -l properly, like this:

ssh HOSTNAME "bash -l /path/to/script.sh"

Answer 3

When running cmd over ssh in one line, the .profile is not read. To test, try this:

ssh host env

Use this instead to fix this issue (quotes are compulsory):

ssh host '. ~/.profile; cmd'

eg:

ssh HOSTNAME '. ~/.bashrc; /var/www/example.com/dumper.sh'