• Vote on Oct 14!

  

• Search for:
• 
  This work is licensed under a Creative Commons Attribution 2.5 Canada License.

• Similar Posts

  • blog repatriation
  • 30,135
  • Drupal 4.7 RC3
  • Airport Extreme on Telus DSL?
  • on flash on the iPhone / iPod Touch

• Pages

  • About
    • Bookshelf
    • Colophon
    • Contact
    • Music
    • My (not so) Secret Identities
    • Presentations
  • Archives
  • Asides
  • Photos
  • WordPress
    • Post Revision Display

• My Other Places

  • Mindfulseeing.com (photoblog)
  • Flickr/dnorman
  • Teaching & Learning Centre
  • The University of Calgary

• Recent Comments

  • Alec Couros on giving thanks
  • Joshua Archer on on open mindedness and the institution
  • Joshua Archer on but universities ARE open…
  • Joshua Archer on on the three parts of open education
  • dnorman on RIP Structured Blogging and Microformats?

• Meta

  • Log in
  • Entries RSS
  • Comments RSS
  • WordPress.org

If you can read this, the move she is done. I’ll write more on that later, but hopefully the performance problems this site has been having for almost 2 years will be a thing of the past *touch ethernet*

After realizing that the sympal_scripts were silently failing to properly call cron.php on sites served from subdirectories on a shared Drupal multisite instance, I rolled up my sleeves to build a script that actually worked. What I’ve come up with works, but is likely not the cleanest or most efficient way of doing things. But it works. Which is better than the solution I had earlier today.

I also took the chance to get more familiar with Ruby. I could have come up with a shell script solution, but I wanted the flexibility to more easily extend the script as needed. And I wanted the chance to play with Ruby in a non-Hello-World scenario.

Here’s the code:

#!/usr/local/bin/ruby

# Drupal multisite hosting auto cron.php runner
# Initial draft version by D'Arcy Norman dnorman@darcynorman.net
# URL goes here
# Idea and some code from a handy page by (some unidentified guy) at http://whytheluckystiff.net/articles/wearingRubySlippersToWork.html

require 'net/http'

# this script assumes that $base_url has been properly set in each site's settings.php file.
# further, it assumes that it is at the START of a line, with spacing as follows:
# $base_url = 'http://mywonderfuldrupalserver.com/site';
# also further, it assumes there is no comment before nor after the content of that line.

# customize this variable to point to your Drupal directory
drupalsitesdir = '/usr/www/drupal' # no trailing slash

Dir[drupalsitesdir + '/sites/**/*.php'].each do |path|
  File.open(path) do |f|
    f.grep( /^\$base_url = / ) do |line|
      line = line.strip();
      baseurl = line.gsub('$base_url = \'', '')
      baseurl = baseurl.gsub('\';', '')
      baseurl = baseurl.gsub('  // NO trailing slash!', '')

      if !baseurl.empty?
        cronurl = baseurl + "/cron.php"
        puts cronurl

        if !cronurl.empty?
          url = URI.parse(cronurl)
          req = Net::HTTP::Get.new(url.path)
          res = Net::HTTP.start(url.host, url.port) {|http|http.request(req)}
          puts res.body
        end
      end
    end
  end
end

No warranty, no guarantee. It works on my servers, and on my PowerBook.

Some caveats:

• It requires a version of Ruby more recent than what ships on MacOSX 10.3 server. Easy enough to update, following the Ruby on Rails installation instructions.
• It requires $base_url to be set in the settings.php file for each site you want to run cron.php on automatically.
• It requires one trivial edit to the script, telling it where Drupal lives on your machine. I might take a look at parameterizing this so it could be run more flexibily.
• It requires cron (or something similar) to trigger the script on a regular basis.

I’m running a couple of servers full of Drupal sites hosted in a multisite configuration (one copy of Drupal used to host dozens of sites, each with their own sites/sitename directory. I’d been using sympal_scripts to automatically run Drupal’s cron.php script for each site in order to keep search indexes up to date and run other routine maintenance functions as expected. It’s easy enough to drop a curl http://server/site/curl.php into a crontab, but as you start adding sites to the server, it becomes unwieldy to maintain a current crontab of sites to cron.

Sympal_scripts attempts to read through the scripts directory, poking through each site and loading Drupal for each one in order to fire off the appropriate cron.php. It’s been adding records to the Drupal watchdog table, so I expected it to be working just fine. Except it hasn’t actually been running cron.php - it’s been failing silently.

Looks like there’s something funky in the way Drupal refers to the $base_url variable for the site. It’s set in each settings.php file, so it should be as simple as returning the content of a string variable. But it’s borking, and returning the name of the directory containing the site’s settings.php file.

Say I’ve got a server, myserver.com, with a bunch of sites all configured to be served as subdirectories of that server’s main website, such as myserver.com/site1 and myserver.com/site2

Each site has a respective directory within the Drupal installation’s sites directory, such as myserver.com.site1 and myserver.com.site2 (the / are converted to . for use in the directory name because / would be invalid in a directory or filename).

When Drupal is initialized by sympal_scripts/cron.php, it’s getting $base_url values of http://myserver.com.site1 and http://myserver.com.site2.

So, when it goes to fire off the cron task, it’s using urls like: http://myserver.com.site1/cron.php

It works fine on sites configured to run on their own domain, as the domain matches the site directory.

WTF? The http:// shows that it’s reading the value within each settings.php file (or does it?), but why is it retaining the .site1 rather than /site1?

Failing that, is there a better way to reliably run cron.php on a bunch of hosted sites? I’m thinking of writing a script that crawls the sites directory and pulls out the $base_url values for each site and then fires off a curl base_url on the lot of them.

It’d be really cool if Drupal’s own cron.php had a command-line version, capable of operating on any (or all) configured sites. Any ideas?

I’ve been trying to move domain registration and DNS hosting for darcynorman.net from GoDaddy to Dreamhost for a couple of months. It’s been a long and frustrating process, involving faxing my driver’s license to Arizona to somehow prove I am who I say I am.

I just logged into my Dreamhost account to check on the status (still hasn’t finalized - they sure did set it up in a hurry, but it takes a looooong time to switch off of GoDaddy). On a lark, I tried adding registration for darcynorman.com. But Dreamhost’s registration utility complained that the domain was already taken.

Mwaaaah? Another D’Arcy Norman out there? Lemme check that out. A quick whois darcynorman.com turned up this:

   Domain Name: DARCYNORMAN.COM   Registrar: GO DADDY SOFTWARE, INC.   Whois Server: whois.godaddy.com   Referral URL: http://registrar.godaddy.com   Name Server: CNS1.CANADIANWEBHOSTING.COM   Name Server: CNS2.CANADIANWEBHOSTING.COM   Status: REGISTRAR-LOCK   Updated Date: 16-mar-2006   Creation Date: 16-mar-2006   Expiration Date: 16-mar-2007

Oh, wait. No. It’s a domain squatter. Sitting on my name, assumedly hoping for a portion of the mad cash this blog generates. Mad cash, I tell you. Some lame squatter leech decided to register my name in the hopes I’d pay a ransom to get it back. At least the squatter is using a Canadian service provider to park the DNS for the domain. I guess that’s better than having it offshored to Moscow or something.

The combination of cheap domain registrations and “secure/private” registrations where you can hide behind a proxy make this practice possible. When I register domains, I need to go through CIRA verification, accept agreements about usage, etc… But these roaches can register other people’s names and park them for ransom. Rules (like locks) are for the honest people.

Screw you, squatter. I just went and registered darcynorman.ca - the only other variant of the domain I’d care about. Go ahead and squat on the rest, you rat bastage.

Woah. Dreamhost is celebrating their 9th birthday, and decided to party by increasing limits on accounts. Account holders now get 200 GB (200 gigabytes - a fifth of a terabyte) of disk space. And 2 TB (2 terabytes) of bandwidth per month.

That’s insane. Three things must have happened, in order for them to be able to offer this at $7.95/month.

1. bandwidth costs have come waaaay down over the years
2. the cost of hard drive space has come waaaay down over the years
3. almost nobody comes even close to using their full allotment of either

It’s awesome that Dreamhost is doing this. It’s pretty cool knowing I’ve got 200GB backing my account, and that I’ll never have to worry about bandwidth. Now, if only the performance of the MySQL server would get a boost…

So… Why hasn’t the decreasing cost of bandwidth affected my home DSL connection at all?

It’s a total non-issue for me, but Dreamhost (the cool company that’s hosting my blog) is going through some rough times in their data centre at the moment. Apparently the heat wave in California is wreaking havoc on their power situation, causing a power outtage. The generators kicked in, but there was a short. And a fire. Hell broke loose. (the mention of the fire has disappeared from their Dreamhoststatus.com blog, so maybe it wasn’t that bad…) So, my blog was down for awhile. Really no big deal. If you can read this, it’s back up. I’m guessing there may be periodic outtages while it’s sorted out.

David noticed and emailed me within minutes of the blackout - well before I would have. Actually, he seems to notice every outtage or hiccough on my server well before I do…

btw, Dreamhost is so unbelievably cool as a hosting company. I accidentally discovered that they have installed Appleshare services on my server (perhaps it’s standard on all of their servers?) - I can have my hosted directory mounted on my desktop, and take advantage of Finder-y goodness rather than resorting to FTP or shell connections for everything. Nice. So, I have a 20GB (that’s GigaBytes) volume, accessible anywhere, via FTP, AFP, or shell connections. They also offer WebDAV for directories (which I don’t use), and Subversion, and one-click installs of every web app I could ever want. And the shell account is fully enabled, with access to emacs, cron, rsync, lynx, etc… not like the silly locked down accounts some providers offer (what? we had no idea you’d want to edit files. we have to enable emacs for you… mysql command line access? really? why would you want that? etc…)

I’m babbling. Dreamhost is an insanely cool hosting company. I’m extremely happy with the service they offer, and this minor downtime is trivial (and unavoidable, given the fragility of the North American power grid - does this scare the crap out of anyone else? A fuse can blow in southern Ontario, and drop the entire Eastern seaboard into darkness. Prime targets. yikes.)

My various online bits are now living at Dreamhost. It took only a few minutes to install my stuff, copy over the files, and get up and running. It’s taken a bit longer to have DNS changes propagate, but I think that process is pretty much over now. Wordpress seems pretty happy there, and I’ve installed copies of Drupal, Mediawiki and Lace (the cool ajax chat app), as well as a Quicktime streaming server and Jabber server. The last two were autoinstalls, so I just flicked them on to see what they did. Actually, everything but Lace could have been automatically installed, with subdomains and databases created automatically, but I opted to do the manual install because I already have copies of the apps configured.

So far, things seem to be working pretty well. They give an insane amount of disk space (20 GB to use as I need) and monthly bandwidth (1 TB/month, plus an extra 8 GB added each week), as well as SSH and FTP access.

I’m looking forward to playing around with Rails a bit, and have a place to host it.

The Dreamhost support team is also pretty darned responsive. I’ve had to contact them twice (once when their account creation form barfed on the apostrophe in my name, and once when the stats weren’t being displayed). Very helpful, those Dreamhost folks. They also have 2 blogs that they use to communicate about status and other stuff. The off-site dreamhoststatus.com blog is a good way to keep up on pending changes or outtages. And their general blog is just plain entertaining - with a category dedicated to rants! It’s good to see a decade-old hosting company that’s grown rather large still be able to have a sense of humour.

Update: One of the things I’m loving about hosting at Dreamhost is that backups are trivial. I have a script in my hosted account that I can trigger via SSH, and it will rsync my entire home directory (including all files needed to host my domain and any subdomains) to my desktop box on campus for backup. And, because rsync is so cool, it only takes a few seconds, since only changed files are sent. So, I can make sure all of my 300+MB of files are backed up in about 10 seconds. Add a scripted mysqldump into the mix, and all data backups are up to date as well.

I’ve hopefully finished shuffling around the bits that run this blog. Sorry for the extra noise in the RSS feed. This will hopefully be the last move for D’Arcy Norman dot net for some time. It’s now living on a Dreamhost server, and DNS should be propagating over the weekend.

The performance of my shared server at GoDaddy leaves a LOT to be desired. Pages render out of the database in several seconds, when they should be easily generated in under a second. Their tech support response was to blame images and javascript, when the actual database-based page generation itself is waaaay too slow. Even when pages are rendered OK, they may be spit out somehow triggering a file download rather than content in the browser. (you may have seen a “Download file: index.php” dialog box - I get it all the freaking time)
Comments on the Wordpress support forum pointed to perhaps the server being overwhelmed and reverting to a “safe” download behaviour.

Either way, I think it’s time to look at the options. I’m currently looking at DreamHost. Their services seem really good (use a wiki for support info, have SSH access and a bunch of other nicenesses that GoDaddy doesn’t). Are there better options? Should I be concerned about Patriot Act and DMCA implications with an American hosting provider?

I’m also wondering about what the migration process is. GoDaddy hosts the DNS as well, so how smooth is it to move that to another provider?

Patrick just came by to ask me about my experience with GoDaddy, so I was telling him about the great deal - 25 GB of bandwidth per month and 500MB of disk space. Patrick looked at me quizzically and said “No, that’s not right… It’s 250GB and 5GB.”

Wha? So, I check my GoDaddy account, and they’ve increased the hosting account limits! It is now 250GB of bandwidth per month, and 5GB of disk space. For $5CDN/month.

GoDaddy ROCKS! I just hope they can stay in business at these rates. Now, if they hosted Ruby, I’d have a nice testbed for playing with Rails…

Next Page →