An Analysis of Market Demand for Web Programming Languages ​​– TechCrunch

Editor’s note: Marc Gayle is a Rails developer and founder of 5KMVP, where he builds minimum viable products for just $5,000. Follow him on Twitter.

A few months ago I had the idea that one way to get leads for freelance gigs remotely was to browse Craigslist. So, after doing the manual labor of “crawling” at least 100 job postings by hand, I wrote a Ruby script to do the heavy lifting and filtering for me.

Once I started browsing through the data, some interesting things started jumping out at me. Even though I don’t actually live in the Valley (I live in Jamaica), I consume a lot of news, blogs and articles from the Valley. Suffice it to say that I am affected by the “Echo Chamber of the Valley”. A side effect of this is an obsession with Ruby and Ruby on Rails as my development stack and a general expectation that the rest of the world will have woken up to its beauty and elegance.

Alas, to my surprise, this is not the case.

Before diving into the data, let me explain exactly what this script does. Throughout Craigslist, there are two URL subpaths that tend to have the majority of freelance web development gigs: /cpg/ and /web/. So the script creates a list of all the cities on Craigslist (because CL doesn’t provide a clean, RESTful API that lets you easily get this information), then just appends /cpg/ and /web/ to the end of this URL. .

Then, on each link, it checks if the current link actually contains concerts published in that city. The reason is that whenever there is no concert displayed in the current city, CL displays the concerts of “nearby cities”. To avoid duplication, the script automatically checks for this and weeds out cities that don’t have uniquely posted gigs. However, that doesn’t rule out a concert that has the exact text and is shown in two different cities – because, well, I hadn’t gotten there yet.

Once the script has a list of valid cities with published gigs, it starts analyzing each of the links on the front page of those cities (i.e. up to 100 links in each city – CL does the pagination by the 100 links) for the keywords which I specified. The advantage of only using the last 100 links in each city is that they are the most recent. The downside is that in active cities, the last 100 links are not always a good sample of the entire population.

For the Rails results, I have the following keywords: rails, (ruby on rails), (ruby on rails 3), (rails 3), (rails 2).

For Ruby results, I have the following keywords: ruby, (ruby 1.8.7), (ruby 1.9.2), (ruby 1.9.3), ruby187, ruby192, ruby193.

Searches are case-insensitive, so any link containing Ruby, rUbY, or RUBY will be found and included. I’m trying to capture all the permutations someone would use “Rails” in a web development sense.

The downside to this basic approach is that for technologies that share similar keywords – for example Ruby on Rails (the framework) and Ruby (the language) – there will be overlap. So in this case, the Ruby results contain a ton of Rails links, i.e. Rails is essentially a subset of Ruby.

I did my best to narrow down as many obviously spammy CL messages in the results to really get to the legitimate messages.

So it’s fair to say, I think, that these results give us a relative approximation of what the indie programming gig market is actually looking for.

Without further ado, here is the data and analysis.

General statistics
  • Cities analysed: 720
  • Total gigs found: 11,992 to 12,076 (scripts were run multiple times – for accuracy – and results returned were within this range)
  • Script execution time: 16 minutes to 1.25 hours
Languages ​​searched

Server-side languages ​​and frameworks

  • C# (C sharp)
  • CodeIgniter (PHP Framework)
  • Django (Python Framework)
  • net point
  • Java
  • Lisp
  • pearl
  • PHP
  • Python
  • Rails (Ruby Framework)
  • Ruby

Client-side languages ​​and MVC frameworks

Lot 1 – Languages

  • HTML
  • CSS
  • JavaScript
  • shine

Batch 2 – MVC JavaScript Frameworks

  • Backbone.js
  • Closing
  • Ember.js
  • Knockout.js
  • Node.js

On the server side, you can see that PHP wins by far, with almost Bolt-like performance knocking everyone out of the water. In fact, Ruby comes in a measly fifth place, Java comes in second, and Dot-Net and C# come in third and fourth respectively.

The most surprising result of the above is that Flash is still in demand, even with all the “Flash is dead” rhetoric circulating in the tech press and blogs. It’s almost as much in demand as JavaScript! Who would think? !

When it comes to JavaScript frameworks, a silent battle is going on with the multitude of existing and regularly released JavaScript frameworks. It is here that the unscientific aspect and the imprecision of this little exercise reappear.

At first glance at the results, the script indicates that Ember produced 14 gigs. However, since the searched keyword is “ember”, Ruby finds any string with a substring of “ember”. So there were 14 links with “Member” and “Membership” in the link title. Not one with Ember.js or the Ember we were looking for. So after manually reviewing the links, 0 results were returned. So the only two client-side MVC frameworks that the script found that are requested are Backbone and Node. Both, barely.

That being said, please take this with a grain of salt. Here is an alternative data point for you. One of the founders of GroupTalent told me a few months ago that the biggest customer demand they see is actually for client-side JavaScript frameworks. Even more so than server-side frameworks.

Below is the combined data:


This message is not intended to start a heated war between the different camps. This is simply an unscientific analysis of what the general market (using Craigslist as a proxy for that market) is looking for in web development talent.

If you’re considering learning one of these languages ​​or frameworks, using what the market demands is a factor in your decision-making process. I wouldn’t necessarily encourage that, though. I certainly didn’t.

I chose Ruby and Ruby on Rails and loved every minute of it. I encourage you to try out different languages ​​and see which one you feel most comfortable with, as the vast majority of time you’ll spend in the language (assuming you really want to improve) will be non-billable.

In the web applications I create for 5KMVP clients, I use Ruby and Ruby on Rails because that’s what I like. Customers have been satisfied and seem to like it too.

If you want me to do an analysis of anything else – for example Mobile vs Server languages ​​or anything else, let me know in the comments or drop me a line at Twitter.

If you found this interesting, you might find an article I wrote on Dropbox, or a guide to understanding cash flow versus profit, equally interesting.

You can find the Ruby script I created on GitHubas well as some sample output files containing a list of all the gigs generated for this article.

Look around and if you submit enhancement requests for any enhancements you may have, the Karma Fairy will multiply your lineage tenfold and your seed will outnumber the celestial bodies.