3

While going through the initial installation and set up of Graphite on CentOS 6.4 using Apache mod_wsgi via the stock graphite-web rpm, I'm getting the following "DatabaseError: database is locked" message:

mod_wsgi (pid=9009): Target WSGI script '/usr/share/graphite/graphite-web.wsgi' cannot be loaded as Python module.
mod_wsgi (pid=9009): Exception occurred processing WSGI script '/usr/share/graphite/graphite-web.wsgi'.
Traceback (most recent call last):
  File "/usr/share/graphite/graphite-web.wsgi", line 16, in <module>
    import graphite.metrics.search
  File "/usr/lib/python2.6/site-packages/graphite/metrics/search.py", line 6, in <module>
    from graphite.storage import is_pattern, match_entries
  File "/usr/lib/python2.6/site-packages/graphite/storage.py", line 7, in <module>
    from graphite.remote_storage import RemoteStore
  File "/usr/lib/python2.6/site-packages/graphite/remote_storage.py", line 8, in <module>
    from graphite.util import unpickle
  File "/usr/lib/python2.6/site-packages/graphite/util.py", line 82, in <module>
    defaultProfile.save()
  File "/usr/lib/python2.6/site-packages/django/db/models/base.py", line 460, in save
    self.save_base(using=using, force_insert=force_insert, force_update=force_update)
  File "/usr/lib/python2.6/site-packages/django/db/models/base.py", line 553, in save_base
    result = manager._insert(values, return_id=update_pk, using=using)
  File "/usr/lib/python2.6/site-packages/django/db/models/manager.py", line 195, in _insert
    return insert_query(self.model, values, **kwargs)
  File "/usr/lib/python2.6/site-packages/django/db/models/query.py", line 1436, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 791, in execute_sql
    cursor = super(SQLInsertCompiler, self).execute_sql(None)
  File "/usr/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
    cursor.execute(sql, params)
  File "/usr/lib/python2.6/site-packages/django/db/backends/util.py", line 34, in execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python2.6/site-packages/django/db/backends/sqlite3/base.py", line 234, in execute
    return Database.Cursor.execute(self, query, params)
DatabaseError: database is locked

I've verified that the DB file ("/var/lib/graphite-web/graphite.db") is accessible by the apache user which owns the httpd process.

Also, I've tried restarting both the httpd and the carbon-cache processes as mentioned in this thread on github.

A list of lsof shows the following:

# lsof | grep graphite.db
httpd      9006    apache   17u      REG              253,2    69632     526186 /var/lib/graphite-web/graphite.db
httpd      9007    apache   17u      REG              253,2    69632     526186 /var/lib/graphite-web/graphite.db
httpd      9008    apache   17u      REG              253,2    69632     526186 /var/lib/graphite-web/graphite.db
httpd      9008    apache   22u      REG              253,2    69632     526186 /var/lib/graphite-web/graphite.db
httpd      9009    apache   17u      REG              253,2    69632     526186 /var/lib/graphite-web/graphite.db
httpd      9009    apache   22u      REG              253,2    69632     526186 /var/lib/graphite-web/graphite.db
httpd      9010    apache   17ur     REG              253,2    69632     526186 /var/lib/graphite-web/graphite.db
httpd      9010    apache   18u      REG              253,2      512     526174 /var/lib/graphite-web/graphite.db-journal
httpd      9010    apache   24ur     REG              253,2    69632     526186 /var/lib/graphite-web/graphite.db
httpd      9011    apache   17u      REG              253,2    69632     526186 /var/lib/graphite-web/graphite.db
httpd      9012    apache   17u      REG              253,2    69632     526186 /var/lib/graphite-web/graphite.db
httpd      9013    apache   17u      REG              253,2    69632     526186 /var/lib/graphite-web/graphite.db

In my mind it's got to be related to httpd, but I'm not getting anywhere with it.

1 Answer 1

0

I kept at it and found that restarting httpd magically worked.

However, it is still unclear why my previous dozen attempts to fix the error with httpd restarts failed. I thought it might have been a race condition between starting up apache+mod_wsgi and hitting the graphite-web page too soon, but on subsequent restarts I've not been able to reproduce the lock. Further attempts in various states (apache running, apache down, with the db owned by root, etc.) have not reproduced the error. Indeed, at this stage I can delete the graphite.db, reinit with "syncdb", and view the webpage successfully all without restarting httpd. The graphite-web page runs fine in all cases.

My suggestion if you're stuck on this:

  1. Shut down the httpd or apache process.
  2. Delete the "graphite.db" file.
  3. Re-run the syncdb process as the "apache" user (or user that owns the httpd process).
  4. Start up the httpd process.
  5. Wait 60+ seconds before hitting the web page, then try it.

This might be the equivalent of "waving a dead chicken", but is how I got out of the mess. Feel free to post your experience here.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .