I’ve spent the last 4 days working on the problem of how to make a screenshot of a URL on a Linux server with no display. Furthermore, I need it to be programmatic, so I could call it from a web app when needed. Additionally, I wanted some architecture that could conceivably lend itself to threading in the future, so that we could scale the thing.
Essentially, all the solutions I found did this:
- Create a headless X server with something like Xvfb.
- Open some sort of browser on that in-memory X window.
- Screenshot the contents of that browser.
- Render that screenshot content out to some image format.
That’s when I found the work of Roland Tapken. His script and explanation were the solution I needed. It made nice screenshots, had the configuration options I needed (screen size, scaling image, destruction of X server, etc.). I thank him deeply for his work and his sharing it with the world.
The only problem off the bat was that it requires at least 2 packages that were not installed for me on Ubuntu 7.10 (Gutsy). In fact, they aren’t available in the versions I needed until the most recent Ubuntu 8.10 (Intrepid). Those packages are libqt4-webkit and python-qt4. Even on Intrepid (at least on my slicehost slice), I needed to install these and their dependencies. You’ll need to make sure to get libqt4-core as well. The libqt4-webkit didn’t seem to require it according to aptitude.
I had a small problem with the script as far as passing arguments into the Xvfb server. For that reason, I’m sharing my version of the code. However, all kudos go to Roland. His blog post on this subject provides a nice overview of the thought process and rationale behind this code. If you’d like a little more on the how and why, check it out. He also provides a simpler version if you’d just like to test out the Qt WebKit stuff and see how it works.
In summary, to use this code:
- Read Roland’s blog post.
- Install the necessary libraries: libqt4-core, libqt4-webkit, python-qt4, xvfb.
- Grab my code here. Change the extension to py.
- To run, use a command like this: python webkit2png.py -x -o cnn.png –debug http://www.cnn.com
If you have questions, comments, or code to add, please post in the comments or pingback this post. I will try to update once I have a good threading model built out around this functionality.
I’m joining a coworker for the Out of the Darkness Community Walk on Nov. 1 here in Nashville. All proceeds go to support the American Foundation for Suicide Prevention. We’ll be getting sponsorships for the walking we are doing. This is a cause which is deeply personal to me and a great way to support a coworker.
If you’d like to sponsor me, please click the link below:
Hopefully, we can make a difference for at least one person.
No, I have not signed up again for that damned Vonage plan. However, after 49 comments in the last two years (pretty much unheard of for my little blog) with the last one coming as recently as today, it seems like many of you still have issues canceling Vonage.
A quick Google search for “cancel vonage” shows that my blog post is one of hundreds, if not thousands, of blog posts, forum posts, email list posts, and all other forms of web communication which detail the drudgery that is attempting to cancel Vonage.
Why would a company go to so much trouble to try to keep customers who clearly do not want their service? Is their cash flow so bad, that they simply must have the revenue? I was amazed about a year and a half ago when I read the news that Vonage couldn’t sign up new customers. Even more amazing was the fact that not a day goes by that their damned commercial is on my TV or a week goes by that I don’t get some form of direct mail piece advertising their product. Apparently, somehow the news never made it to Vonage’s offices. Not one aspect of their business model or customer service strategy seems to have changed since it became clear that they were fleecing customers.
The money must be great, because they still are around and seemingly thriving. With the ubiquitous nature of the cell phone and with cell plans getting cheaper every day, what does Vonage offer that’s not available via a service like GrandCentral? How can we more effectively spread that word that Vonage is a Faustian bargain?
I’d like to write more about SQLAlchemy and Django, and I will, but for now, just this little note. (As a teaser, we use Django and SQLAlchemy [along with Elixir] on a daily basis, and it’s a great combination.)
As far as open-source support goes, we all know that projects vary greatly in how well they support the folks using their code. The incentive for them to do so is almost entirely altruistic, unless one considers having a popular open-source product as a certain kind of fame.
Anyway, in that environment, Michael Bayer, who wrote most of, maintains, and supports SQLAlchemy, single-handedly answers nearly every question on the mailing list. His answers inform, but also point the question asker to places where he or she can learn even more past the answer to the initial question.
I use SQLAlchemy and I really like it, but past that, it’s great to see Michael be such a great resource to the community surrounding the product.
When I was a high school teacher, our whole school read Buzz Bissinger’s book Friday Night Lights. It’s a good book, and was very familiar territory for me as a guy who grew up playing high school football in the South.
It’s a little odd then that I am only coming around to watching the TV series, on Hulu, based on that book. These are some of my first impressions having only seen the first two episodes of the first season.
- Peter Berg’s hand is directly evident in the direction of the show and that’s a good thing. I like the closeups, interesting camera moves and natural tone to the lighting and color paletter.
- There seems to be a pervasive sadness in all the characters. I don’t remember this from the book, but I read it a long time ago. Perhaps, this sadness comes from the way a small town always feels like it teeters on the edge of oblivion.
- Kyle Chandler’s portrayal of Coach Taylor is perfectly balanced: his concern for his players always at odds with his need and desire to win. I would be very interested to see Chandler get a hold of some meaty film roles.
- The possible demonizing of the townspeople is, like in the book, done with nothing more than what is very close to reportage. It’s not mean-spirited, but is presented by reality. As one of the team boosters says, “Football is all this town’s got.” That’s the reality of many small southern towns whose dreams are pinned on the occasional exceptional athletes or really young people of any kind that are products of these towns. There’s an irony there that’s worth exploring.
- While I tend to disagree with the football as a metaphor for life, it’s hard to ignore the parallels any sort of competition or group endeavor gives a person to the challenges we face internally.
Recently, two places of which I am an alumnus have started making me very angry.
As best I can tell, both the high school from which I graduated and the fraternity of which I was a member in college have recently contracted out their alumni data gathering to a company called Publishing Concepts. Took a look around the web and you’ll see what sort of insanity this company does in it’s quest for your information.
In PCI’s effort to contact me and “verify my information,” I have received 3 postcards about my high school and 4 about my fraternity, numerous emails, and now, multiple phone calls per day. When I answer the phone, I hear nothing but a dial tone, then the call disconnects. This phone tag happens at least 3 times a day.
The real gipper here is that I have already pledged both of these organizations money and am making ongoing contributions. This is exactly the sort of treatment that makes one begin to question his/her continuing support.
I’ve now emailed the school and the fraternity to let them know exactly the treatment they’ve paid for their alumni to receive.
A long time ago, longer than I’d like to admit, I started on a freelance project. I can’t say too much about what it does, but I would like share a bit about the development process I went through.
When I started the project, I was beginning to get back into web development after a couple of years away from the work during which I was teaching. So, I had some catching up to do and rusty skills needed to be dusted off and honed.
The project is somewhat ambitious with lots of GIS work and a little bit of social networking thrown in. It is a big project and means a lot to me, both financially and as a way to learn and really sharpen up my skills. It’s been a pleasure to work on as it is finally coming around to fruition.
All that said, I jumped in with both feet and starting coding in PHP, with some bad habits such as HTML all over my PHP. I know enough CSS to be trouble, so there was lots of that, too. I was building with some eye toward a previous incarnation of the app, so there were some design decisions made then that we have long since abandoned.
Many months late, I produced a passable version. It didn’t look like much, but it worked. When the fellow I was doing the work for asked me how scalable and maintainable it was, I had to do a lot of soul searching. I had become friends with this client and I knew deep inside that what I had built would work up to a point, but that it wasn’t that extensible and I was unsure how some of the GIS bits would scale.
I owned up to some of what I saw as problems with what I had built. I was building something to be a an alpha/proof-of-concept when what the client wanted was a solid beta product. I had misunderstood and probably gotten in over my head.
It was then that my client and I had a heart-to-heart and decided we would start from scratch with a platform that was extensible, maintainable, and offered some niceties that no homegrown PHP app could easily do. That platform was Python/Django/PostgreSQL.
With PostgreSQL, we got the PostGIS libraries which overjoy me day in and day out. With Python, we got clean, maintainable code. With Django, we got a robust framework that makes it easy to add new features without massive rewrites. Django also gave us fantastic code/design separation and it was just plain fun to learn.
These great visions in mind, I set about to not only learn the Django framework, but to also learn Python. Luckily, my new full-time job was giving me plenty of PostgreSQL experience, though no PostGIS (yet!), as well as, giving me the ability to work with Django and Python on the new features we are building at work. So, I wasn’t totally in over my head.
Every piece of the app that I write now is a pleasure to code. When I’m “done,” I can hand over these neat, clean HTML templates to a designer to spruce up with little fear of confusion. The database access stuff, especially with GeoDjango in tow, has become a breeze. Everything I used to hate about web development has been made a tiny little bit of the work. The vast majority of my time is spent thinking about scalability and how the feature actually works rather than building another script to handle form post input. It has simply been a joy all the way around.
We are nearing the end of the application work on the new Django version of the app. There’s lots more to do and lots of ideas for the next version. The difference between this Django version of the app and the PHP version is night and day. It might look similar from a UI perspective, but it is vastly fewer lines of code (not including Django itself), another developer could easily pick it up and improve on it or add to it, bugs are much, much easier to track down, and it was actually fun.
When all is said and done, I will, of course, point you to the app and will maybe do some more detailed comparisons between the two with some code. One thing to keep in mind is that the difference is not just between a PHP and Django app, but between the developer I was and the developer I am becoming. Sure, I’m no rock star, but I’m getting better at this every day.
I’d like to give a big thanks to my client for his faith and reassurance as I tackled this daunting task. A great client and an open, honest relationship go a long way. I’ve so far past any deadline we ever set on this project, but my client sees the vision and understands why we’re doing what we’re doing.
I’ve decided to come back to WordPress.com [http://Wordpress.com] after branching out to some Rails-based software for a while on my own domain. In short, a blog at WordPress.com [http://Wordpress.com] is just so much easier than running through the update dance on a self-hosted WordPress system. Maybe I’ll actually write a little more here now.
I am moving my blog to here:
It runs SimpleLog which itself is a Rails application. I will be building out the root site soon, but have moved the blog for now.
Once again, I find myself looking for work. I guess that’s the problem when you love new challenges. Sometimes, they don’t work out. If anyone knows of web development work in Nashville, TN, let me know.