Community Update

Picture: Arachne demo of web handler Picture: Arachne demo of web handler

Approximate Reading Time: 5 minutes

Hello Arachne Enthusiasts!

Arachne is taking form. What does its future hold? After sharing some news and updating you on our progress, this post peers into Arachne's future. Our last post was in early July, so Luke has been feeling torn between writing a community update and writing Arachne itself. That's why I'm writing you instead.

My name is Jay Martin. I'm a fledgling web programmer with a sweet tooth for Clojure and Datomic. I'll be helping Luke focus on building Arachne by serving as a community liaison.

So, what does a community liaison do? Mostly listen.

Arachne's community liaisons will listen to feedback from the Community and relay that feedback to Luke in digestible form, striving to honor the feedback's spirit and substance.

Also, we want to learn as much as we can about Luke's mental model for building software systems with Arachne. We intend to share that mental model with you, clearly and concisely.

Luke is passionate about fostering a culture of learning and knowledge sharing, as is the Steering Group.

You can expect updates on our progress about once every two weeks.

Also, keep an eye out for our first technical screencasts and Wiki articles covering Arachne fundamentals. We'll post links to new resources here on this site as they become available.

Many people have stepped forward to offer Luke their help. Luke is deeply inspired by this outpouring of support. The Steering Group sees a risk to the project's success by opening up the development process too early, while Luke is simply getting ideas out of his head and into an initial project structure. The opposite risk for Arachne is that of isolating the development process from the Community, alienating the very people who will help Arachne reach its full potential – you, and every single supporter, code contributor, Kickstarter, developer, designer, author, blogger, speaker and tire kicker out there.

We aim to strike a pragmatic balance between these competing risks and we'll rely on you to let us know when we're off track. If you have ideas about how to improve Arachne, please open a GitHub issue. If you have ideas about how to make our Community Experience the very best in this quadrant of the galaxy, please send me a tweet @webappzero.

We'll open up more lines of communication, as needed to best support Arachne's community with the care and attention it deserves. We'll include Community Developers and Designers in the creative process once Arachne's fundamental pieces are more concrete. The idea being that it's easier to talk en masse about improving tangible code than intangible concepts. Presently, that's the role of the Steering Group. Luke shares his ideas and progress and then listens to our feedback. Gradually, the Community as a whole will replace the Steering Group in that role.

Speaking of Arachne's core pieces, here's a list of things that Luke has gotten done, adding to his original work, since officially starting work on Arachne on July 18th:

  • implemented continuous delivery via CircleCI
  • fully Speced the core module
  • implemented an ontology for config schemas
  • reified config & runtime entities in the config database
  • wrote utilities for error messages
  • factored apart the abstract HTTP & Pedestal modules
  • added provenance info to config & init scripts
  • designed Spec-based config & runtime validation
  • wrote 12 Architectural Decision Records (ADRs)

…and finally, saving the best for last:

  • created three working demos of Arachne's core functionality!

Yes! We've got demos. They reveal the inner-workings of the low-level Arachne system. These are largely undocumented so only the adventurous should expect to receive a learning outcome by attempting their execution. Also, keep in mind that these primitives are very low level and much friendlier DSLs (Domain Specific Languages) can and will be built on top of what you'll see there now.

My first impression of Arachne, based on seeing the working demos, is that of a system which vanquishes mystery and embraces accountability. Arachne's validated, acyclic, runtime system graph ignited atop a queryable, validated, schema compliant config value feels like a new level of control and awareness when initializing a software system. And while Arachne is marketed as a web framework, at its core its a framework of frameworks, a novel way of organizing a software application with the potential to influence the way we write server, desktop and even mobile software.

This level of control comes with a price: learning a new way of doing things that is, as yet, unproven in the real world of shipping software. Existing Clojure Libraries and software will need to be wrapped in an interface to comply with the Arachne Module protocol in order to reap the benefits described in the ADR on abstract modules.

When I saw my first Arachne demo I felt a little overwhelmed by all the new words like 'ontology', 'config schema', 'component' and how they all fit together. At that time, I'd never even used a Stuart Sierra Component, so it was a lot to take in.

For some, this price will appear too steep. Fortunately for Arachne, our Industry is awash in change. Driving us toward this new future is the mere possibility that the inherent frustrations of building and managing software systems could be significantly mitigated. Many have already opted for a future that includes Arachne by voting for its existence with their time, talent and treasure. For that we cannot thank all of you enough!

Arachne's immediate future will see Luke writing the Database Abstraction Layer (DAL). The DAL will include the common set of database primitives and commands shared by today's prevalent database systems: entities, queries, etc. I'll be working on the Static Site Generator which will allow us to dogfood Arachne by using it to host this site in the near future.

Concluding with a bit of philosophical conjecture about Arachne's future:

I recently learned about solving problems using recursion from a web page by Carnegie Mellon University:

To solve a problem recursively means that you have to first redefine the problem in terms of a smaller subproblem of the same type as the original problem. […] This step is often called a recursive leap of faith. Before using a recursive call, you must be convinced that the recursive call will do what it is supposed to do. You do not need to think how recursive calls work, just assume that it returns the correct result.

Arachne isn't pulled from the ether. It benefits from a host of innovations, built by people that answered countless challenges and shared their life's work with the world. It has been refined by many generous and thought provoking conversations within the Clojure and programming Communities. Something else will come after Arachne and It will be better than It could have been because of what Arachne is and what it is not. Ultimately, Arachne's success will depend on our collective willingness to convince ourselves that the pleasure it promises will outstrip the pain it hides and move ourselves to that self-evident and demonstrable leap of faith we call – action!


P.S. Tweet me with any ideas you have about Arachne or our Community. Or just to say hi!