Steering Group #2

These are the notes for the second meeting of the Arachne steering group, which occurred on Friday, May 6.

The attendees this week were:

  • Tim Ewald
  • Jay Martin
  • Russ Olsen
  • Stuart Sierra
  • Luke VanderHart

This was a smaller, shorter meeting than the last one. Because Luke's efforts the last couple weeks were spent on public-facing and Kickstarter-related materials, there was no new technical content to discuss in depth. We did, however, cover a number of other topics.

Also of note, this was our first meeting to include Jay Martin, who backed the Kickstarter campaign at the $10k level and will be joining the steering group going forward. He has a background in IT and finance, and the Arachne project resonated with him after getting back into web development with Python and Clojure. He is enthusiastic about the project and eager to be involved in development, educational materials, and ensuring that the project remains easy and accessible to beginners.

Topic: Engaging better with the Clojure community

Although public reception of the project has been largely positive, there have also been some negative reactions. We spent some time discussing the community dynamics and how we can ensure we remain friendly, inclusive and supportive of the Clojure web-development community in its entirety. Some suggestions were:

  • Stay positive
  • Focus on the novel features and vision of Arachne, not any perceived deficiencies in alternatives.
  • Answer questions and stay engaged, but put the primary focus on releasing software that will speak for itself. Show, don't tell.
  • Focus on users who already feel the need for something different. Those who are happy with the status quo will remain happy; Arachne should focus on helping those who want a more framework-heavy approach and growing the webdev community, and is not in any way a contradiction or threat to the existing ecosystem.

Topic: Engaging better with the non-Clojure web development community

So far, we haven't managed to engage at all with the non-Clojure web development community. We discussed why this is; the consensus was that there's no reason anyone should care, given that nothing really exists yet. Once Arachne is released, we can reach out to grow our user base by telling a compelling story and providing some demos of Arachne's capabilities. We can't expect to grow outside the Clojure community until we have this.

We also discussed how people evaluate frameworks based both on "hygiene factors" (doing basic things in a easy and clean way), and "selling points" (unique capabilities that bring people to the framework.)

Some ideas:

  • The Rails-esque "write a blog in 15 minutes" is now table stakes for a modern web framework; we need something comparable.
  • Create demos showcasing the "hygiene factors" - ease,
  • Create demos showcasing the "selling points." Some examples for Arachne could be:
    • Modularity: live-code swapping out major components of an app.
    • Introspectability: demo a dashboard exploring the config of an application.
  • Demo a one-button deploy of the system.
  • Tell a good story about security, baked in from the ground up.

Topic: brainstorming asset pipelines

Asset pipelining is an extremely important and somewhat difficult functionality for any web framework.

Some questions:

  • Do we build something from scratch in Clojure, or offload to an existing Ruby or Node tool (the approach taken by Phoenix, which offloads to Brunch)
  • Do all the tools need to run in the JVM or are we OK shelling out to external processes? What are the implications of that for setup complexity?
  • What are the actual "must have" tools? Can we get away with not supporting tools that only have an implementation in one language?
  • What are existing Clojure tools in this space? (Boot tasks, twixt, stefon, etc.)
  • What strategy should we use for cache busting?
  • For development workflows, should we rebuild assets using filesystem watchers, or build on-demand when a request for an asset is received (or some hybrid approach?)

It would be nice if the asset pipeline were complete and robust enough that a project using only the asset pipeline component was a complete, usable static site generator.

Ultimately we came out with lots more questions and few or no firm answers, but it was good to have the discussion.

Topic: abstract data layer

At this point in the conversation, we were getting pressed for time, so we had little opportunity to do more than bring up the topic for future thought.

In general, the idea is to have a means of specifying persist-able domain entities in a way that is agnostic to any particular database. This means defining a common schema format and persistence API that will work with JDBC, Datomic, various NoSQL databases, etc.

This has three primary use cases.

  • Modules sometimes need to persist data, but they should not be strongly coupled to any particular choice of storage, for greater modularity. Such modules need to be able to program exclusively against an abstract data storage API.
  • Modules should be able to re-purpose an entity definition for uses other than persistence. For example, a module might take an entity definition schema and expose it as a RESTful resource, or generate a form to edit it.
  • Users should not hesitate to exercise the full power of their chosen database, rather than the greatest common denominator of database features, which will be the most the abstract layer can do (by necessity). However, the most common 70% of database interactions should still be extremely easy via the abstract API.

Several points were raised:

  • To make this problem tractable, we need to be aggressive about limiting the scope of the abstract data layer. It simply can't do everything well and still be manageable.
  • We need to have a good story about schema generation, but also about connecting to existing databases with pre-existing arbitrary schemas.
  • We need to be able to handle sets of columns/attributes that are constructed at query time as well as concrete tables; there is not a necessarily a 1:1 mapping between your application's concept of an "entity" and the data store's concept (e.g, a Datomic entity or a SQL table)

At this point, we used up our allotted hour and adjourned until the next meeting.