Monday, December 26, 2016

On Documenting Software Engineering Projects

I started working through my own fork of the Open Eats recipe site software.  To date, I've only made a few minor changes, such as placing (almost) the entire repository in an openeats module, so that the manage.py file can run correctly without further user setup.  I've also started documenting a lot of what I'm finding here.

One of the first things I questioned was where my documentation should live.  I've essentially got access to this here blog, the above wiki, the issue tracker attached to Bitbucket, and the code itself.  I'm sure my preferences will change over time, but here are the current rules for documentation usage:

  • Code - Documentation - much to the surprise of a younger version of myself - should be minimal here.  If the 'how' of the code is unclear, the code should be refactored.  If there's a specific 'why' behind an algorithmic design, that's fair game, though I could argue that documentation belongs in the wiki.  Architectural/project design should be left to the wiki.  So, the code should really only behave as an actor, not as an explainer.
  • Issue Tracker - Document immediate issues that are either enhancements (which I'm also treating as stories) or bugs.  I'm not using any other types.  If they existed in this simple bug tracker, I'd potentially use epics to group bugs and enhancements, but they don't, so I won't.  Things here should be immediate calls to action with very specific exit criteria for fixing.  There may be some overlap with the wiki on information, but that's OK.  We don't refactor issues or wikis, so duplicate information is OK.  Although, we should be very diligent in editing the wiki to ensure that information is correct.  Issue tracking requires less diligent editing, as issues are expected to be ephemeral and out of date at some not too far distant point.  Of course, anyone who's ever managed a backlog knows this isn't always true, but the principle still holds.
  • Wiki - This is the canonical documentation and requires the most care.  Wait, what!? Don't you mean the code requires the most care, since it runs the site?  Well, maybe.  In most cases, you'll be reading a lot more code than you'll be writing.  If you're not, even as part of greenfield development, you're doing it wrong, because you're copying and pasting from stackoverflow without understanding what's going on.  Second, the code is to some extent, self-correcting.  If you add a feature and it doesn't work correctly, you'll eventually find out.  If your documentation is wrong, you'll only wind up frustrating people in the long run who expect it to impart correct information.  The wiki can be used for just about anything, and I'd say that shading toward over-information is preferable to less information, as long as you're following along with a decent narrative (so, don't just barf up some documentation from somewhere else and expect that it will be sufficient for your wiki).  People are going to come to your wiki to understand just about everything, so you should give it the appropriate care.  What's more likely than not to show up in a wiki:
    • Architectural overviews and decisions.
    • Notes that are expected to last a long time and show why the decisions that are in place are in place.  
    • Notes describing particularly inscrutable or idiomatic code choices.  This reduces the text volume overhead in the code itself.
    • Discussions for future enhancements or extensive bug work.
  • Blog - I've debated a bit about whether or not both a blog and a wiki are necessary for documentation in this case.  If I had to jettison one, I'd lose the blog, but I think that a blog does a good job of complementing the wiki.  A blog generally shades toward thought process, while the wiki shows the output of that thought (though a wiki can also show thought process).  A blog is good for discussing meta topics (like this one about choosing documentation standards).  A blog is time based, where a wiki is more generally a static snapshot.  So, I can write a lengthy post about documentation standards and then completely contradict myself next week without making edits to the wiki documentation.  If I were promoting the recipe website, a blog serves as another separate medium for marketing, which serves a useful purpose.  Finally, a blog allows me to go off-script with various posts (say, if I want to review Rogue One tomorrow) without seeming too out of place.
A few final notes on documentation - 

Given the comprehensive abilities of markdown files, I'm fine with the argument that a good README.md with children could supplant a traditional wiki.  In fact, now, as I write this, that may be a better option, as then the repository and documentation is entirely self contained.  For the moment, Bitbucket makes it easy enough to integrate issue tracking, code, and a wiki that the argument for markdown is a little academic.

The most important part of this post (talk about burying the lede), is that documentation is difficult.  It requires a lot of writing and editing.  It's extremely time intensive, and can be frustrating when you put in a lot of effort, but people either don't get your point or don't read it.  However, those aren't excuses not to document.  Code without documentation is essentially dead code.  Code is part of a living ecosystem and needs support - it never lives in isolation.

This holds more true in the commercial space than in the hobbyist space, but, even though you may know the code like the back of your hand, eventually you'll quit or move on to something else and someone will need to support what you've built.

Saturday, December 17, 2016

A New Hope

Yes, of course that's a crappy Rogue One joke, because a new Star Wars movie came out.  More to the point, though, it's sorta true.  I've been working in earnest on my own recipe website for the last few months.  Originally, it was Python on Google App Engine (no wait, it was Angular!), then Python, then Python on a VPS solution.  I've learned a lot - especially around standing up infrastructure and basic security principles.  Enough so, that I've spent more time on the operations and infrastructure than I have on the actual feature code itself.  However, to move forward a little faster, I decided to see if there was another open source solution available.  Lo and behold, there was.  Before I get into that, though, let's go through my overall motivations for this project:

Motivations

  • I'm a software engineering manager now, not a software engineer.  In order to keep current on some skills, and learn some new ones, I wanted to get involved in a project.
  • I wanted to work on a project that has some nominal value for me.  Because I cook a lot, and because recipe sites are rife with ads that make browsing unbearably slow, I decided to write a recipe site.
  • Yes, I know I could do a ton of other things like install ad blocking software, utilize Google Drive, or probably just buy software, but I'm cheap and, also, see bullet point one.
  • I also want to learn as much as possible to run a small website.  This isn't start up territory where I lose money when I don't ship features.  This is a labor of love and software engineering.  Here's a list of things I want to explore and improve: back end feature development, front end feature development and UX design, mobile development, DB administration, scalability, reliability, monitoring, infrastructure as code, data science and documentation.  So, basically everything I can get my hands on (I'm sure there will be more).  However, I realize I may never get to all of those things, but it should be fun to try.
  • I do actually want to use the site though, and, if I concentrate on all of those things, I'll learn a lot (which is ok in the case), but won't actually have a product.
I realized after doing this, though, that one thing I liked about being a software engineer was being a code detective.  It annoys me to no end when people throw out old, working code to start over in search of a 'better' system rather than spend a little effort to improve what's there already.  In order to meet my goals above and adhere to my software engineering principles, I decided to find an open source recipe system I could work from.  In five minutes I found Open Eats by Googling for 'open source recipe software'.  Here's my copy of the repository.  Wish me luck!