Okito.net

On SproutCore 2.0

As you might have heard, Apple released the icloud.com service to developers yesterday.  Some people violated their NDA if you are interested.

First of all, a big “congratulations!” is due to the people that worked on this project.  It’s truly fantastic.  I am proud to have worked with you in the past.  (And prouder still that I get to work with some you at Strobe!)

Secondly, this seems like a good time to explain a bit more about what’s happening with SproutCore and our move from the 1.6 branch to 2.0.  There has been some confusion in the community about what the new 2.0 release means and why we are making such radical changes to the old code.

I think iCloud demonstrates perfectly why 2.0 needs to exist.

Some Background

SproutCore has always been about creating “native” style web applications in the web browser.  Thanks to apps like Twitter and Gmail, and iCloud, the trend towards these types of apps seems pretty obvious today.  Five years ago - when I wrote the first version - it wasn’t so clear.

Lucky for me, though, a major company was willing to pay me and a small team to spend our full time just thinking about how you could push browsers to their limits in creating large scale, fluid, native-style web applications.  You can find most of the tech we built to make that possible in SproutCore 1.6 today.  As you can see, in the hands of someone who understands the entire system, you can do truly amazing things with it.  

Over the years, some very smart and brave pioneers have been willing to dig in and learn the code necessary to master SproutCore.  Those people have been able to do great things with it; many even contributed back awesome new improvements of their own.  

I am very grateful for this group of people.  But, what about everyone else?  It shouldn’t take a few days of training and lots of stamina to make something amazing with SproutCore.  That is where 2.0 comes in.

If SproutCore 1.0 was about figuring out how to get the most out of browsers so a select few could build amazing native-style apps, SproutCore 2.0 is about bringing that power to everyone else.

SproutCore 2.0 is a completely refactored edition of SproutCore.  While the old version was made up of four frameworks and tightly coupled build tools, 2.0 is made up of nearly a dozen libraries and an independent tool chain.  The result is a system that is much easier to document, easier to learn, and easier for community members to get involved with it.

Let me give you a couple of examples:

Whitelisting vs String Loading

SproutCore 1.6 is a huge framework.  I think it can top out around 1MB when you include everything.  To make matters worse, although the framework is made up of four sub-frameworks, poor management overtime has meant that they all share dependencies making it impossible to turn most of them off.

The way you solve this in 1.6 is with a poorly documented feature called “whitelisting”.  (There is also a “blacklisting” option, equally poorly documented.)  These two features allow you to selectivly omit or include specific JS files from SproutCore during your build.  This makes your build much smaller but essentially requires you to understand all the dependencies to make it work.

SproutCore 2.0 is much simpler.  For one thing, there are now nearly a dozen frameworks (and growing) with carefully managed dependencies, making it much easier for you opt into only the parts you care about.  More importantly, the new build tools use a new ‘string loading’ technique which alleviates most of the pain around loading a lot of JavaScript - especially on mobile - making this issue far less important to begin with.

More importantly, the process of managing these dependencies is a part of the `bpm` - the package manager - which is currently being finished but will include full man pages explaining the process.  String loading is a bit more esoteric, but it is also completely automatic so most developers will get the benefit of this feature without even having to know it exists.

RunLoops vs Invisible RunLoops

Another very important feature of SproutCore is the RunLoop.  RunLoops allow SproutCore applications to defer expensive operations to the end of a cycle.  For example, most rendering in SproutCore views is deferred in this mannor, meaning that expensive DOM operations only happen once - dramatically improving performance on large applications.

The problem is that RunLoop’s are a classic ‘leaky’ abstraction.  It’s very frustrating to write code and then have it seemingly execute out of order.  As a result, almost everyon SproutCore 1.x developer has to learn in-depth at some point about RunLoops and how they work.  1.x code is frequently peppered with calls to `invokeLater()` and `setTimeout()` to try to set these timing issues right.  (Though almost always this actually makes things worse.)

In SproutCore 2.0 RunLoops still exist, but they are far more invisible.  We spent a huge amount of time rewriting the View layer to carefully manage which parts of your code we will defer in order to minimize any surprises when writing code.

Personally, I still think most SproutCore developers will need to learn what a RunLoop is at some point in time, but for the early people trying SproutCore 2.0 it is far less of an issue.  You definitely don’t see invokeLater() or setTimeout() peppered about like before, which is a huge improvement.

Ad Infinitum

There are many more examples.  But perhaps the most important point about this is that SproutCore 2.0 is essentially an effort to take the great technology we assembled in SproutCore 1.6 and go through it piece by piece - carefully testing and documenting each component, working through rough patches that seem to trip up or confuse people, until we feel confident that most people will be able to get it.

And this is where we get to the part that confuses most people.  This task is like organizing a messy house; you can’t do it all at once.  The only way to give each thing the attention it deserves is to go room by room, working on each section until you feel it is done well it enough to move on.

During this process, some features you can find in SproutCore 1.6 today will be missing.  Eventually, most of them (a few things in 1.6 are truly bad ideas) will come back.  Until then, our recommendation is simple:

1. If you want the full power of everything in SproutCore today and you cannot wait and you are willing to dig into the code, use SproutCore 1.6.  A lot of people still are.  Strobe’s forthcoming app even uses 1.6.
2. If you are building a simpler app or a “hybrid” native-web style app along the lines of Twitter, SproutCore 2.0 - when it goes final - is for you.  It’s leaner, easier to learn, faster out of the box, but also missing the rich desktop controls of 1.6.

And what if you want the desktop controls in 1.6 but the ease of use of 2.0?  

Wait. Or better yet: contribute!

SproutCore 2.0 is far easier for people to learn and use.  And now all of the core functionality is basically done.  Views work, bindings work, the datastore works.  Mostly what we need now are the rich desktop-like controls - an effort we plan to capture in the sproutcore-ui and related packages.

Thanks to our more flexible design in SC 2.0 it’s way easier to get involved now too.  These are separate packages, meaning you don’t have to get anyone’s permission to start writing your own and sharing them with others.  

So hopefully this helps to clear some things up.  SproutCore, I believe, is still the best framework out their for building fluid native-style web apps.  With the transition from 1.6 to 2.0, we’re opening it up to the public too.  

We are making it far easier to anyone to learn how to use it and contribute themselves.

If you want to check out the new SproutCore 2.0 the best place to learn more about it today at at the Github page.  We are building out an updated website and docs for SproutCore and the build tools we are using (called bpm) for the final release - so stay tuned for more information there.

In the meantime, if you have thoughts on this transition, I would love to hear from you. Drop a comment here or follow me on twitter @okito.

SproutCore/WWDC Meetup 2010

The SproutCore Team is getting together tomorrow night in San Francisco for dinner and drinks following the last WWDC session.  This is really informal meeting this year, but if you want to see the core team and chat, signup here:

SproutCore Meetup Eventbrite

We will confirm a location tonight based on the number of RSVPs at 6pm Pacific Today.  Hope to see you there!