Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm looking for an open source headless browser to run on a FreeBSD shared Web server, which can do reasonably accurate rendering of current HTML/CSS/JS.

As I'm not very skillfull with Unix/servers and don't have root access, this should have no dependencies that would not be installed on a typical Apache shared Web hosting server (ruling out Java I believe), and should ideally even be pre-compiled for FreeBSD.

I guess I need a command-line and/or PHP-accessible interface (I'm not too clear on how any of that will work).

UPDATE:

From the answers I'm getting this seems to still be an obscure subject. Here's the sort of thing being done with headless browsers:

http://googlewebmastercentral.blogspot.com/2009/10/proposal-for-making-ajax-crawlable.html

But rather than turning this question into a general discussion of the subject, can we keep it focused to answers that have specific software recommendations for me -- if any (I suspect what I'm looking for isn't available yet). Thanks

share|improve this question
    
I think you could some terms messed up here, are you really looking for a browser like Firefox, but for the terminal? If yes, why would you need a browser on a headless system which can even do JavaScript? –  Bobby Jan 18 '11 at 12:37
    
    
This is an extremely obscure request. There's only a dozen companies around who would want/need functionality like this. No 'typical sysadmin' would need it. –  Chris S Jan 18 '11 at 13:33

2 Answers 2

Look at twill, which gives you a command-line interface. It doesn't support Javascript, but it does support cookies and forms.

The Mozilla project has a more complex offering, XULrunner, which is supposed to support the whole XUL runtime, but I don't know how well this works in practice. My gut feeling is that the semantics of javascript are hard to model satisfactorily with a browser-in-the-middle.

share|improve this answer
    
"...a more complex offering..." — I think that's generally going to be the problem. I need to stick a ready-to-go app file on my server, read a quick tutorial, and then be able to write a simple PHP script that says "hey headless browser — go render this page that's somewhere on the Web and take a virtual screenshot of the page and stick it in this directory for me", or "give me what JavaScript is reporting as the rendered height of that page". Not be a "certified XUL programmer" with 3 annual conferences and a stack of books on the desk... in order to get the thing installed and configured :) –  Patricia Jan 18 '11 at 12:51
    
+1 XULRunner is the only thing I know of. It's definitely not install & use though; it's basically a library and framework for developing a browser or integrating browser technology into another product (like a webpage thumbnail generator program). –  Chris S Jan 18 '11 at 13:32
    
@Patricia: I'm guessing you won't find anything simple. An HTML page with Javascript is a running program, especially if it is Ajax; you need to decide under what conditions you will count it as loaded and what resources your headless browser needs (Flash? H264?). It's not much like telnetting to port 80 and typing a few commands. –  Charles Stewart Jan 18 '11 at 13:36

Why?

Although there are text-mode browsers about, even some which support javascript and CSS, there's a huge gap between what they render and what you see in conventional browsers.

It would have been helpful to know why you wanted to put yourself through this pain.

Is there a reason you don't run Firefox over X?

I need a command-line and/or PHP-accessible interface

No such thing. And I'm now even more confused trying to guess what problem you are trying to solve. Have you looked at cURL?

share|improve this answer
    
For example to make dynamic thumbnails of Web pages. Or the entire new concept of pre-rendering AJAX pages for Google's new(ish) AJAX site indexing guidelines. This is not a novel concept I'm talking about -- headless browsers are kind of the big new thing that's not big yet. See HtmlUnit (but this is Java which I think I can't run) htmlunit.sourceforge.net. –  Patricia Jan 18 '11 at 12:45

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.