Summit/WebsiteIntegration

Motivation

Right now, the *.debconf.org (*.dc.o) websites are based on Perl page templates rendering HTML files, giving a consistent layout, while CSS is used in those templates for a consistent style. www.dc.o appears in a neutral style (and uses its own templates), while each edition of dcX.dc.o provides a new set of templates (duplication), along with the specific style files.

Recently, Summit was added and integrated with the website, using the same style sheet. However, the integration is based on what I will call "stitching", meaning that the static website and Summit both have to have the same set of tabs defined, linking to each other — put your mouse over the links on http://debconf14.debconf.org to see this. If a tab is added/removed/renamed, the change has to be done in Git, as well as in Summit.

While there is some redundancy here, it does sort of work (although it might send shivers up your spine). [wendar notes: The difference between the two systems does present a barrier to volunteer maintainers. In DC14, the HTML header copied into summit was a bit broken (strange floating header images), and the header menu didn't get updated with new links/info when the main site changed. I should have asked around, found out who was maintaining summit, and asked them to fix it, or asked them how to check out the code, set up a test server, etc. so I could do it myself. But, I didn't. It was just one more thing I didn't have time for, and it was "good enough". This is nothing against summit, which I've set up now, and is quite easy to work with. Just a comment on the psychological barrier of two radically different systems for website maintainence.]

The actual web site content (e.g. visa information) is provided using HTML files, and it is unlikely that HTML was ever designed for human authoring or consumption, certainly not anymore in the 21st century. Sure, it's possible to do, but there's also a greater barrier of entry resulting in sluggishly maintained websites and us over-using the wiki for a lot of things that should be on the main conference website.

The solution proposed herein would solve both these shortcomings. It would get rid of human-facing HTML files and have all content served from a single, uniform location, namely Django (which runs Summit). The content itself would remain in Git, but be replaced by a high-level mark-up language, similar to this Wiki.

To clarify: the solution would apply to future websites (possibly including DC15). Past conference sites would continue being served as they are.

Details

Types of data

Let's distinguish types of data:

static data

templates, styles, images used for all pages

content

human-consumable per-page content (meat)

dynamic data

data dependent on context and/or database

Django router

Django's central piece of logic is called the "router" and it matches patterns against a URL. As soon as there is a match, it passes control to a handler. As this is a wonderfully simple approach, I hope that the following table is enough to convey the essence of the idea:

URL pattern	Handler	Description
/static/*	Apache/lighttpd alias	Git clone containing DebConf global static data (see below).
/dcX/conf	Summit	Single Summit instance initialised with defaults for dcX
/dcX/static	Apache/lighttpd alias	Subdir /static of the Git clone for dcX web data (see below)
/dcX/*	Git-Markup-Cache	Pure content served out of dcX Git repo, see link
/*	Git-Markup-Cache	Pure content served out of www Git repo, see link

[wendar notes: Why not use the traditional www.dc.o, dcX.dc.o, and summit.dc.o? This gives us more flexibility to make the transition gradually now, launching a new version of each domain as we consider it ready. And, preserves the easy ability to make transitions in the future, as we continue to evolve.]

[madduck: I'd say, we don't need summit.dc.o, but having dcX.dc.o serve a conference and www.dc.o just be a landing page does allow us to change things in the future without affecting the past. So for dc1[45].dc.o, dcX.dc.o actually routes within Django to Summit, and all the while dc42.dc.o can be routed to something entirely different without having us to juggle. And similarly, pre-dc15.dc.o could just continue their lives as they are. Good idea.]

I hope most of this is straight-forward. The only thing that's not plain is Git-Markup-Cache, so without further ado:

Git-Markup-Cache

This stems right out of #Motivation. It's not rocket science either:

1. A given Git repository holds files written using some sort of mark-up language, e.g. MarkDown or reStructuredText
2. When Django receives a request for e.g. /dcX/visa/info, it
   1. fetches visa/info.raw from Git (possibly via the filesystem)
   2. pre-processes the data (see below)
   3. renders them using MarkDown or reStructuredText (or whatever)
   4. post-processes the result (see below)
   5. caches the result (fixed time, or actual filesystem modification check)
   6. renders it in the context of the applicable page template (which provides styling information)

Pre- and post-processing

Not much needs to be done here, but it might be nice to (warning: braindump follows…):

• Provide macros that allow indexing into summit or the Django router
  • Pages linking here?
  • Tags
  • Link to {Summit:event:189} etc., or somesuch, to prevent having to hardcode HTTP URIs
• Correct header levels and crop to body text (processors can mostly do this)

In general, we should not aim to do too much here, as dynamic data should probably be handled by specific Django controllers…

Template/style inheritance

CSS and Django templates can use inheritance, allowing for a consistent presentation.

Since style of presentation will vary from year to year, it'll be important to design this such that the base template provides enough flexibility (slots) and can be extended and elements moved around by the author of each dcX's design.

CSS is much simpler in this regard as the dcX CSS style sheet can just inherit from the global default and override at will.

Regarding the main webpage at www.dc.o, I believe this page always appear in the layout/style of the next DebConf, and so its template gets manually updated at the time of the switch to refer to the appropriate /dcX/static files instead of the templates and styles in /static.

[wendar notes: I'd prefer www.dc.o to have its own very simple style. It mostly needs to serve as a splashy landing screen pointing to the latest debconf and past debconfs, so something like this https://za.pycon.org/ would be better than the current style or a copy of the style-of-the-year.]

Notes and pointers

Implementation

• Django docs on the built-in filesystem storage class and how to write custom storage systems.
• Does it make sense to implement this as a custom storage class?
• The suggested alternative is to just open() in the view, set ETag to the commit ID and leave caching to NGinx: https://groups.google.com/forum/#!searchin/django-users/Git/django-users/CScLXdd7nYM/6P0F0ok-ojgJ

Similar approaches

• git-cms already implements this basic idea, but it remains to be seen how well it fits.
  • Does it allow for caching?
  • It does look like it could be used for menus, header, sponsor logo list, etc., which makes the whole site more accessible for updates.
  • If this is not a reasonable implementation for our purposes, at least it demonstrates the feasibility of the concept and may be a source of inspiration for features and implementation.
• https://bitbucket.org/bors_ltd/django-gitstorage
• https://github.com/mgaitan/waliki

FAQ

What would change for me?

Instead of editing HTML files and ensuring syntax, maintaining them in Git, you get to concentrate on content and maintain it in Git. So unless you love writing HTML more than you love writing e-mails, you should have less to worry about.

Can Django handle the load

Yes. We would be a tiny Django site