The Nuages of Django

I've finally finished my upgrade to Django 1.10, and, even though I've ranted a bit about the choice between maintenance programming vs. greenfield programming, it's worth taking a final look at the effort it entailed.

• Here's the issue I used to track the work.  It took me about a month total to perform the work, though the vast majority of that was in relatively wide blocks of weekend time.  I'd estimate that the total effort required about 20-40 hours.  I did this in about 70 commits.  They aren't all independent, but they're as close to being side effect free between each other as reasonably made sense.
• Using the issue to track my progress gave me a lot of advantages: it made me define clear (though imperfect) criteria for finishing the issue; it kept me disciplined in my tracking through short, relevant comments (although across 107 of them at this juncture, I'm sure there's repetition); it gave me a place to track articles and musings without having to worry about whether or not they were too trivial to put into something more 'official' like the wiki or this blog; and the aforementioned discipline helped me decide when other ideas should require additional issues.
• I didn't hit every topic here during the upgrade, but I hit a lot of them.  Because a lot of the code was already available in the OpenEats project, much of the documentation served to be more tangential than it would be had I needed to learn things from scratch.  I think of this style of learning akin to learning the patterns of a foreign language via common phrases first before learning the rules of the grammar.  Greenfield development, to me, draws a common thread with attempting to learn how to conjugate all of the verbs in a given language before attempting to speak it.  In the former example, the constructs of grammar are important, but the ability to communicate, however imperfectly, outweighs the benefits of knowing all of the tenses of a limited domain.
• The existing code gave me a strong launching ground for writing my own views.  OpenEats, as it stands today, favors a model where recipes are public by default and expected to be shared.  Because I'm more interested in simply having a site that's ad free and that allows me to view collected recipes and don't want to step on anyone's intellectual property rights, it's important that I keep the recipes private.  I was able to take an existing view and generate one that fit my needs, even though they were vastly different from the site from which they were derived.  
• The other thing that an existing site does (even still somewhat within the context of the Django upgrade I started talking about), is provide straw men to help me decide my site strategy.  There's a lot that's part of the site that I want to keep, but there are some things that aren't as useful, and some that I found when they weren't there, provided a strong signal for a future idea.

There's probably a number of other things that I found valuable during this Django journey, but I'm happy leaving those musings for another post.

The Software Detective - Django Authentication

As I move through the Django upgrade of my OpenEats fork, I also continue to double down on my philosophy that it's better to build from a previous foundation than it is to just write new code.  In today's argument to that effect, I'll use the site's authentication system as an example.

The current OpenEats site uses a combination of django registration redux and the framework's authentication module.  It also places login and sign up logic on the same page as separate tabs and makes calls to those individual functions via ajax.  I don't find anything wrong with this approach, but it seems a bit complicated in its design, so I decided to separate the two functions out into separate pages without ajax calls.

Now, if I were to do this as a green field project, I likely would have looked up the framework's authentication module and got a lot of the login/logout/password reset goodness that comes with it, but I wouldn't have found the additional registration functionality that comes with the redux app.  I may have stumbled on to redux if I were researching alternative authentication modules, but it would have been one of many.  The fact that I may have chosen something else isn't bad, but I probably would have spent too much time looking at the pros and cons of each module than simply selecting something, testing it, and moving on.  Because redux was already there, it reduced the decision space I had to wander through.  Also, because the redux functionality is pretty comprehensive, but the documentation hasn't yet caught up, I needed to read through the redux source code (which is pretty easy to follow, btw).  The end of the README also led me to a possible replacement when I want to move over to OAuth2.

Finally, because the OpenEats code references a lot of different snippets of authentication, and these made no sense to me at the beginning of the project, I needed to delve into the Django documentation to learn more about its authentication mechanisms.  Even more to the point, I needed to learn how the code in front of me worked, so I had a clear stopping point when researching documentation.  Again, if this were green field, I either would have copied and pasted code I don't quite understand, or attempted to read as much documentation as possible in order to ensure that I wasn't missing out on some valuable detail.

So, to recap, here's what I got out of not simply writing an authentication app from scratch:

• I was able to leverage someone else's code to skip writing functionality that's generally considered tedious.
• I was able to delve into other code on github to better understand my own project (and, as we've been told time and time again, it's good to read others' code and get familiar with github projects).
• I had a ready definition of requirements for this portion of my project.
• I was able to delve somewhat deep into the authentication documentation while making sure I didn't go too deep that I couldn't come up for air.
• I found a lead for a possible future feature on my site.

Not too shabby for something that's generally considered maintenance programming.

Everything New is Broken Again

I'm tackling the biggest programming task I've faced in quite some time as part of my fork of Open Eats.  The code is in good working order, but it's a few years old and relies on a lot of outdated libraries, most notably, Django.

It's at this stage that I could take a few paths - I could ignore what's there and completely write my own site; I could take the requirements and just rewrite everything from scratch with those requirements; I could leave everything in place and build off of the older foundation; or I could undertake the maintenance effort to upgrade the system.  I chose the last option.  Though I do generally opt for fixing what's there, it's not always straightforward, and I think this article does a good job of outlining some of the reasons why.

In a professional atmosphere, one of the big questions companies need to ask themselves is - do we move forward to get the latest feature set out there and risk incurring technical debt, or do we work on technical debt and risk falling behind on the feature curve?  There's no definitive answer, as it requires a balancing act.  Start ups, because they only have a toe hold in an industry, usually opt for features.  More established companies have the luxury of working on infrastructure projects (in fact, anyone at a start up who's advocating the use of microservices is probably an overzealous advocate of software engineering trends).

However, I feel that in both cases, small and large companies tend to do maintenance programming wrong.  Even large companies attempt rewrites of portions of their code base and through their 'best' (usually read most prolific or most pedantic) programmers at the problem.  This seems to be based on the human perception that if something's broken, starting anew will not only fix it, but improve it.  Unfortunately, when rewriting something, people often ignore the lessons of Dr. Heidegger and continue making the same mistakes over again while contributing new bugs to the rewrite effort.

The only time I could ever advocate rewriting something from scratch is given the following conditions - (a) the requirements are well defined, meaning the problem set the software solves is relatively self-contained (b) no one at the organization either understands the project's code or has the ability to delve into it and (c) everyone realizes that the new code will miss existing features and surface new bugs.

If those conditions aren't met, it's better to maintain what's there than it is launch a new greenfield project.  If you're making arguments like (a) it'll take too much time to maintain the existing project, or (b) it's too hard to rewrite because there's no safe way to rewrite the code (i.e. there's insufficient testing around the project), you're probably making the wrong argument.  For (a) you're ignoring the time you'll need to learn a new technology or process, and you're certainly ignoring the time to fix the new bugs and update the features you forgot about during the initial rewrite.  For (b), if you can't safely figure out what existing code can do, why do you think you can figure out something that has yet to exist?

This isn't to say that, via maintenance channels, you can't completely rewrite your entire stack.  You just go about it from a renovation point of view.  Don't bulldoze the house and blow up the foundation just to re-pour it.  If the floors are worn (or you need an ORM to replace your aging JDBC code), sand the floors and refinish them.  If the kitchen is outdated (or you need some semblance of an MVC framework to replace your hand-crafted version from 2000), install new counters and appliances.  If you need a new addition to the house (or need to get off Java 1.4 and have the option to use something newer, like Go), draw up the plans and do the work to make sure it fits well with the existing frame.

Sure, maintenance programming doesn't have the starry-eyed narrative of perfectly formed code springing forth from the head of a genius architect, but the ultimate result takes no less intellectual effort and is no less rewarding in the long run.