How To Build the Next Mega Open Source Project
8 lessons learned from a 3-dimensional framework for understanding how to turn your open-source project into the next WordPress or Linux.
Tufayal Hossin Emon
- The Riskiest Thing Is To Take No Risks.
- Writen article 12
- View Articles
How To Build the Next Mega Open Source Project
Here's what we've been up to recently.
“Open Source projects exhibit natural increasing returns to scale. That’s because most developers are interested in using and participating in the largest projects, and the projects with the most developers are more likely to quickly fix bugs, add features and work reliably across the largest number of platforms. So, tracking the projects with the highest developer velocity can help illuminate promising areas in which to get involved, and what are likely to be the successful platforms over the next several years”
Open-source has a very limited history, 20 years or so, and yet, some of the most successful open-source projects have been around just as long! Linux, WordPress, to name just a few.
So what makes them successful if 98% of open-source projects end up in the waste bin? I believe the answer lies in a three-dimensional framework. The answer is that at different stages of growth, the project needs something very different, both in terms of growth and in terms of monetization. And projects, or COSS companies sponsoring projects, might not catch that wind.
Let’s explore them all in detail, starting with a little history of the WordPress CMS system, which isn’t just a CMS anymore. I chose WordPress because it’s an excellent example of walking through all three dimensions, having a great understanding of what it means to change to drive adoption in each stage, and how it is possible to build a flourishing business that changes in each dimension.
The three dimensions are:
Dimension-1: extending the core project (the usual OS stuff)
Dimension-2: letting the project extend as intended (using extensions, modules, plugins, themes,…)
Dimension-3: getting surprising extensions! The phase of unguided extensions, where the project evolves along never imagined paths.
WordPress’s 3-Dimensional Journey
WordPress’s 3-Dimensional Journey
WordPress started out as a fork of b2/cafelog in 2003 by Matt Mullenweg and Mike Little. They started off making it a great blogging system by working on the permalinks, searchability, comment moderation, and so on. This is the time spent in dimension-1, extending the core project.
The adoption started to surge at the end of 2004, half a year after WordPress started to incorporate the first dimension-2 feature: plugins. Plugins allowed others to work on their thing themselves, and then connect it to WordPress in a predefined way.
You’ll notice you’re starting to go from dimension-1 to dimension-2, once there’s an explosion of repositories around you that are connected. And yes, it wasn’t until 6 months after the introduction, that the adoption kicked in.
In 2005 more dimension-2 features like an advanced theme system, and a library system for plugins were added.
In 2003 & the beginning of 2004, the WordPress focus was on making using WordPress as easy as possible with the “5-minute installation” and making development really easy by focussing on web standards, etc. From the end of 2004–2005, the focus widened to make it really easy to extend WordPress both with plugins and with themes. The functions.php e.g. was added to the theme system as well as customizable headers and the likes.
The WordPress theme directory, obviously another dimension-2 feature, was launched in 2008, together with the plugin installer library. Notice how WordPress focused 2008 on extensibility? Additionally, WordPress enabled an ecosystem around itself, it enabled businesses to sell both plugins as well as themes, thus really bringing other companies into the mix, not just sole developers.
Till the end of 2010, WordPress continued in these three tracks, it extended its dimension-1, going from a blogging system to a CMS for static websites, it worked on its dimension-2 by vastly improving the developer experience by having an improved code editor for plugins and even auto-upgraded for plugins.
End of 2010, WordPress already powered roughly 10–12% of the internet. So in 2011, it apparently was time for a change: dimension-3. Unguided extension. 2011 saw the release of multiple e-Commerce plugins and small systems around these plugins. WordPress has finally made the jump to becoming a framework itself for building stuff online, not just blogs or static websites, but any type of content.
By today, you can use WordPress to build anything, not just content but even mobile apps with WordPress, which displays the extension in dimension-3. Oh, and WordPress is on its way to power half the internet. That’s the success we’re talking about here.
The three-dimensional Model
The three-dimensional model is really simple, it’s characterized through two simple questions:
- Is your project expanding in different directions than your main direction?
- Did you intend this expansion or not?
But through this simple classification, a vastly different set of spaces opens up, with a lot of different incentives, growth models, business models, and risks.
Apache Kafka is a great example of an open-source project that is still very centered in dimension-1. Kafka is trying to move into dimension-2 with Kafka Connect, even offering paid extensions there, but the scope of extension is still only focused on “sources & sinks”, just components of the whole Kafka system.
The company Dbt Labs is currently moving into dimension-2 with their project dbt and their offering of “dbt packages”. In comparison to Kafka, this is happening fast. It will move dbt away from the “risk of service wrapping”. (See below) But more extensions will be needed to truly move into dimension-2.
Superset started to focus on dimension-2 roughly 6 years after its first commit at AirBnB:
“For engineers and hackers, we’ve made Superset much more modular, extensible **and integratable**. We’re now exposing the building blocks of Superset for engineers to extend or use in other projects. It’s now easier than ever to create new visualization plugins for Superset and to share those plugins back with the community. ”(Superset v 1.0 launch announcement)
And no, just having an API that does “everything the tool does” does not qualify as dimension-2. Dimension-2 is about progressively extending & building a community around it, and Superset is a great example. They realized it’s not just about having an API but about exposing the building blocks and making it easy to share stuff back to the community.
All the big successful projects, WordPress, Linux, Firefox are great examples of projects that made the successful jump into dimension-3, all containing a large number of big forks and whole companies that spin around using it as a framework for creating something new.
There are also open-source projects that expand as a “main direction”. A tool like Meltano or RudderStack has an extension at its core, unlike e.g. Kafka. I consider these tools to be dimension-1.
8 Lessons from our Framework
Frameworks alone are pretty useless unless we can draw some lessons from them. Here is my small list of lessons from the 3-dimensional open-source framework.
1. Moving through the three dimensions to expand the market!
WordPress extended its core market, from blogging into static sites. But hosting both blogs as well as static websites is today just a very small part of Automattic's revenue stream. Both are simple extensions in the 1st dimension.
Automattic's plugin Akismet is another story, it extended the market into a whole new world where Automattic could also generate money from the creation of cloud-hosted plugins.
Finally, a big share of Automattic's revenue today is from premium/enterprise-level customized hosting & support for dynamic websites like the NYTimes and others. These are most possibly through the extension of WordPress into dimension-3.
So in that sense, WordPress/Automattic extended the market & thus the business scope by 100x or more, by extension in all three dimensions, and basically growing the pie for everyone involved.
2. Successful COSS business models are iterative
But it’s not just WordPress or GitLab, a lot of COSS companies explain how they iterate through business models. If you listen to Justin Borgman on opensourceunderdogs, you’ll notice how the company Starburst, even though it’s quite young, already stumbled through a bunch of business models.
They went through both, trying to offer support & consulting in the early days of the presto project. Of course, they likely noticed that this is a fine business model, clashes with the incentives you actually want to set. Support & consulting benefit from hard-to-set up projects, whereas in dimension-1 you want to make it as easy as possible to set up the project.
GitLab was on the same journey and afterward decided to focus on customized development. Does that sound right? Probably not, because to become successful, they realized they could just as well move their project into dimension-2 and make it extensible, thus eliminating most need for custom development.
This is the same moment in the journey where Automattic as a company launched by doing both, extending into dimension-2 by providing the “plugin store” as well as providing their own plugin.
3. There is a jump point
It might be obvious but it’s important to state, this model has two jump points! You have to have a certain level of adoption & maturity in dimension-1 before you can extend to dimension-2. The same goes for dimension-3. But once you make a jump, you’ll have to extend in 2 resp. 3 dimensions, not just the new one to keep progressing.
4. Competitive Contribution: Sacrifice short term $$$ for long term $$$
The Akismet story is a great example. Akismet was launched inside the plugin store, a major dimension-2 extension. Automattic could’ve just as well restricted the plugin store to prevent competition, but they did not. What they did was to launch a paid plugin as well as giving others the tools to compete. Indeed they continuously made it easier to build a competitive plugin by in general making it easy to provide & manage plugins.
That sounds very counterintuitive but it’s not, because of (1). Giving others the tools grew the complete pie by 100x or more, and as such gave space to many more paid plugins, many of which could’ve also been provided by Automattic.
5. Dimension-3 business models are diversified
Both the growing of the pie (1) and the iterative business model search (2) produce very sustainable business models, diversified business models. None of these companies completely abandon their previous business models once a new one enters their vision. Automattic today generated revenue from a variety of sources, to name just a few:
- WordPress.com standard hosting (of which I am a customer)
- Akismet is a paid plugin
- Customized and personalized enterprise hosting (e.g. for the NYTimes.com)
- WooCommerce, e-commerce websites based on WordPress which come with extended support & consulting. (a part of Automattic)
6. Dimension-3 business models thrive on knowledge
Given that dimension-3 COSS companies face competition in all of their revenue streams because after all, that’s the point, to grow adoption and grow competitive contribution it’s interesting to see how they manage to turn a profit.
One key point is of course the sales & marketing they draw from their open-source project. Another key point I found is that they thrive on the knowledge they’ve built. The COSS accompanying an open-source project is usually undoubtedly THE expert in the field and as such the first point of contact for, say, buying a proper anti-spam plugin or high-level hosting.
Thirdly, these companies usually build a culture around the open-source project. This in turn allows them to realign their business again and again, kicking other competitors out of the water. Other companies might rely on custom development for an open-source project, while the COSS company that strives to be a dimension-3 project has no problem implementing a plugin-store which destroys the business models of some companies.
7. Only dimension-1 is truly at risk of “service wrapping”
Example: “Service Wrapping” is… if someone takes your “service” and wraps it into his own product. AWS Kinesis is built on top of Kafka, but the name does not appear anywhere. AWS Kinesis seems to be mostly independently developed. Worse, it’s a competitor to Kafka and thus cuts into Confluent’s, the company behind Kafka, revenue stream.
That in a nutshell is why COSS companies are worried about “service wrapping”. The lighter form is simply a managed version of the open-source system, which still is competition and the reason why a BSL license focuses on excluding just that.
My claim is, only projects that live mostly in dimension-1 are at risk of true service wrapping.
Why? Because the value of your project is derived from dimension-2, it is not possible to “wrap it away”. That doesn’t mean service wrapping does not happen, in fact, what will happen is that services appear that are e.g. “PostgreSQL compatible” leveraging both the project and the modules in this case.
But that’s a good thing! It means that both the adoption & development power of the project grows.
Example: Most of the top open-source projects are great examples. Take Firefox for instance, how many extensions does the average Firefox user have running? Maybe an individual theme, lots of customizations! If you “wrap firefox” you will have to put a big stamp on it “firefox extension compatible” which essentially still will fire up the adoption of firefox extensions and thus fire up firefox.
In dimension-2 you will still be at risk of getting hosting competition, but that is it. If you’re curious how to navigate around that, read my other post about COSS product development and pricing.
8. When To switch dimensions
As you can see, successful projects don’t switch dimensions. They level up, go from one dimensional to two dimensional, but they return to expand on the first dimension when needed. That seems very much like a judgment call to make, the only key being that you should never forget to see where you’re headed in the end, something big three-dimensional.
Question for you to ponder
Return to the quote in the beginning and ask yourself, where do you see your specific market segment in 10–15 years? If the answer is not “my project: №1, everything else: somewhere else” then you’re in trouble — at least if you’re looking to build something that lasts… Ask yourself:
- Where am I in this model?
- What are the next steps to expand in every dimension I’m already in?
- Where do I see my project in 10–15 years in all three dimensions?
- Can I already jump to another dimension?