View source for Summit/WebsiteIntegration

= Motivation =

Right now, the <tt>*.debconf.org</tt> (<tt>*.dc.o</tt>) websites are based on Perl page templates rendering HTML files, giving a consistent layout, while CSS is used in those templates for a consistent style. <tt>www.dc.o</tt> appears in a neutral style (and uses its own templates), while each edition of <tt>dcX.dc.o</tt> provides a new set of templates (duplication), along with the specific style files.

Recently, Summit was added and integrated with the website, using the same style sheet. However, the integration is based on what I will call "stitching", meaning that the static website and Summit both have to have the same set of tabs defined, linking to each other — put your mouse over the links on http://debconf14.debconf.org to see this. If a tab is added/removed/renamed, the change has to be done in Git, as well as in Summit.

While there is some redundancy here, it does sort of work (although it might send shivers up your spine). [wendar notes: The difference between the two systems does present a barrier to volunteer maintainers. In DC14, the HTML header copied into summit was a bit broken (strange floating header images), and the header menu didn't get updated with new links/info when the main site changed. I '''should''' have asked around, found out who was maintaining summit, and asked them to fix it, or asked them how to check out the code, set up a test server, etc. so I could do it myself. But, I didn't. It was just one more thing I didn't have time for, and it was "good enough". This is nothing against summit, which I've set up now, and is quite easy to work with. Just a comment on the psychological barrier of two radically different systems for website maintainence.]

The actual web site content (e.g. visa information) is provided using HTML files, and it is unlikely that HTML was ever designed for human authoring or consumption, certainly not anymore in the 21st century. Sure, it's possible to do, but there's also a greater barrier of entry resulting in sluggishly maintained websites and us over-using the wiki for a lot of things that should be on the main conference website.

The solution proposed herein would solve both these shortcomings. It would get rid of human-facing HTML files and have all content served from a single, uniform location, namely Django (which runs Summit). The content itself would remain in Git, but be replaced by a high-level mark-up language, similar to this Wiki.

To clarify: the solution would apply to future websites (possibly including DC15). Past conference sites would continue being served as they are.

= Details =

== Types of data ==

Let's distinguish types of data:

; static data
: templates, styles, images used for all pages

; content
: human-consumable per-page content (meat)

; dynamic data
: data dependent on context and/or database

== Django router ==

Django's central piece of logic is called the "router" and it matches patterns against a URL. As soon as there is a match, it passes control to a handler. As this is a wonderfully simple approach, I hope that the following table is enough to convey the essence of the idea:

{|class="wikitable"
! URL pattern
! Handler
! Description
|-
| <tt>/static/*</tt>
| Apache/lighttpd alias
| Git clone containing DebConf global static data (see [[#Template.2Fstyle_inheritance|below]]).
|-
| <tt>/dcX/conf</tt>
| Summit
| Single Summit instance initialised with defaults for dcX
|-
| <tt>/dcX/static</tt>
| Apache/lighttpd alias
| Subdir /static of the Git clone for dcX web data (see [[#Template.2Fstyle_inheritance|below]])
|-
| <tt>/dcX/*</tt>
| [[#Git-Markup-Cache|Git-Markup-Cache]]
| Pure content served out of dcX Git repo, see link
|-
| <tt>/*</tt>
| [[#Git-Markup-Cache|Git-Markup-Cache]]
| Pure content served out of www Git repo, see link
|}

[wendar notes: Why not use the traditional www.dc.o, dcX.dc.o, and summit.dc.o? This gives us more flexibility to make the transition gradually now, launching a new version of each domain as we consider it ready. And, preserves the easy ability to make transitions in the future, as we continue to evolve.]

[madduck: I'd say, we don't need summit.dc.o, but having dcX.dc.o serve a conference and www.dc.o just be a landing page does allow us to change things in the future without affecting the past. So for dc1[45].dc.o, dcX.dc.o actually routes within Django to Summit, and all the while dc42.dc.o can be routed to something entirely different without having us to juggle. And similarly, pre-dc15.dc.o could just continue their lives as they are. Good idea.]

I hope most of this is straight-forward. The only thing that's not plain is [[#Git-Markup-Cache|Git-Markup-Cache]], so without further ado:

== Git-Markup-Cache ==

This stems right out of [[#Motivation]]. It's not rocket science either:

# A given Git repository holds files written using some sort of mark-up language, e.g. [http://daringfireball.net/projects/markdown/ MarkDown] or [http://luispedro.org/software/git-cms reStructuredText]
# When Django receives a request for e.g. <tt>/dcX/visa/info</tt>, it
## fetches <tt>visa/info.raw</tt> from Git (possibly via the filesystem)
## pre-processes the data (see [[#Pre-_and_post-processing|below]])
## renders them using MarkDown or reStructuredText (or whatever)
## post-processes the result (see [[#Pre-_and_post-processing|below]])
## caches the result (fixed time, or actual filesystem modification check)
## renders it in the context of the applicable page template (which provides styling information)

=== Pre- and post-processing ===

Not much needs to be done here, but it might be nice to (warning: braindump follows…):

* Provide macros that allow indexing into summit or the Django router
** Pages linking here?
** Tags
** Link to <tt>{Summit:event:189}</tt> etc., or somesuch, to prevent having to hardcode HTTP URIs
* Correct header levels and crop to body text (processors can mostly do this)

In general, we should not aim to do too much here, as dynamic data should probably be handled by specific Django controllers…

=== Template/style inheritance ===

CSS and Django templates can use inheritance, allowing for a consistent presentation.

Since style of presentation will vary from year to year, it'll be important to design this such that the base template provides enough flexibility (slots) and can be extended and elements moved around by the author of each dcX's design.

CSS is much simpler in this regard as the dcX CSS style sheet can just inherit from the global default and override at will.

Regarding the main webpage at <tt>www.dc.o</tt>, I believe this page always appear in the layout/style of the next DebConf, and so its template gets manually updated at the time of the switch to refer to the appropriate <tt>/dcX/static</tt> files instead of the templates and styles in <tt>/static</tt>.

[wendar notes: I'd prefer www.dc.o to have its own very simple style. It mostly needs to serve as a splashy landing screen pointing to the latest debconf and past debconfs, so something like this https://za.pycon.org/ would be better than the current style or a copy of the style-of-the-year.]

=== Notes and pointers ===

==== Implementation ====

* Django docs on [https://docs.djangoproject.com/en/dev/topics/files/#file-storage the built-in filesystem storage class] and how to [https://docs.djangoproject.com/en/dev/howto/custom-file-storage/ write custom storage systems].
* Does it make sense to implement this as a custom storage class? 
* The suggested alternative is to just <tt>open()</tt> in the view, set ETag to the commit ID and leave caching to NGinx: https://groups.google.com/forum/#!searchin/django-users/Git/django-users/CScLXdd7nYM/6P0F0ok-ojgJ

==== Similar approaches ====

* [http://luispedro.org/software/git-cms git-cms] already implements this basic idea, but it remains to be seen how well it fits.
** Does it allow for caching?
** It does look like it could be used for menus, header, sponsor logo list, etc., which makes the whole site more accessible for updates.
** If this is not a reasonable implementation for our purposes, at least it demonstrates the feasibility of the concept and may be a source of inspiration for features and implementation.
* https://bitbucket.org/bors_ltd/django-gitstorage
* https://github.com/mgaitan/waliki

= FAQ =

; What would change for me?
: Instead of editing HTML files and ensuring syntax, maintaining them in Git, you get to concentrate on content and maintain it in Git. So unless you love writing HTML more than you love writing e-mails, you should have less to worry about.

; Can Django handle the load
: Yes. We would be a tiny Django site

[[Category:DebConf Manual/Summit]]