Matt Warren

Category: Software Engineering

Coding, Python, web development, architecture, and deployment

  • Creating More Apps

    One of the goals of Halotis is to create and launch a number of robust webservices for Mobile Apps.  However, before we are ready to launch those to the public for use we want to make sure they work at scale and prove that the ideas are useful.

    To that end we have set an ambitious goal of launching a large number of apps and games over the next few months in order to gain the scale we need to dog food our own services.

    A game we have been working on which will be coming out soon is a match-3 type game called Invader Crush:

     

    Another game which will be released shortly is our hilarious take on the Flappy Bird phenomenon.

     

    With the help of tools like SpriteBuilder we are producing a stream of new games as quickly as possible all of which will be used as a test platform for refining our backend services before they’re ready for public use.

  • Yeoman for Modern Web Design

    While I’ve been delving into Django and Ruby on Rails for the last few years the world of Node.js has been maturing.

    The set of command line tools that have developed around the Node.js platform has evolved tremendously.  They have escaped beyond the confines of Node.js Apps.

    Yeoman brings together an app starting templating engine (yo), Grunt – a framework for command line tools, and Bower for dependancy management.

    The biggest thing I have gotten from working with Yeoman is the exposure to new tools and new ways of thinking about web development.  SASS + Compass, with HTML processing.  you can tie together Jekyll for static page templating, and use bower to pull down javascript libraries without having to scour the internet for the latest versions.

    Yeoman brings together all the nice things we are used to in the python and ruby world.  It’s kind of interesting to work with management dependancy for javascript libraries.

  • Creating Feedback Loops

    A feedback loop for success is anything that looks at your past or current state and if you have fallen off the tracks does something to get you back on them.

    That’s kind of vague. So here are some examples of feedback loops:

    • For a weight lifter – keeping a log of past lifts encourages them to lift more, and work harder if they see loss of strength over time.
    • For this blog a nag email goes out after a week of not writing, that email encourages me to write, which stops the emails
    • The Jerry Seinfield calendar which encourages you to do something every day.
    • A regular financial audit – helps to identify areas that need focus hopefully before they become a problem.
    • An annual performance review helps employees develop strategies to improve.

    If you have a goal then having an effective feedback loop can help you reach it.

    The start of the year is a good time to do a review of your feedback loops (a feedback loop for feedback loops) to decide if any of them are ineffective and can be dropped, or if new ones are needed to match your goals. For a business it helps to be systematic.

    If the goal is to 10x your business in 2014 then to start with you would create a strategy that would achieve it (concrete things to do). For each of those concrete steps in the strategy you would evaluate each major pillar of your business to see if any feedback loops would be appropriate – Financial, HR, Sales, Marketing, Product development.  Is there anything you could measure? A good feedback loop will help identify when you need to make adjustments to strategy earlier.

    The worst feedback loop for a goal is to just put it in your calendar – “Dec 31 2014: review business to see if it’s 10x”.  A day or week after setting that goal, you’ll forget about it, get derailed, and be surprised at the end of the year when you get the calendar alert. You can do better.

    If part of your strategy to 10x your business is to re-design your website, then an effective feedback loop is one that actually forces you to do the work. Perhaps an alert on the website’s github repository to ensure there is active development on it every day. or a kanban board to organize the work and visualize progress. If you want an effective website and not just a different one, adding A/B testing and doing a weekly review of conversion rates is a great way to keep focused on continual improvement.

    The most effective feedback loops will DEMAND that you do the work to execute your strategy, and will MEASURE the effectiveness of the strategy to achieve your goals.

     

  • Buying Code

    As a developer I have strong opinions about what makes good software and there is a desire to rewrite code that I find from other people so that it is ‘better’. This makes it difficult  to justify purchasing code.  I could just write something similar/better myself.  Yet, buying code is probably the cheapest most effective way to get ahead.

    Software takes time to write. If I could sustain 500 lines of code per day, a 5000 line app would be 2 full-time weeks of effort.  That’s thousands of dollars worth of my time.

    I recently purchased roughly that much code. A 5000 line iOS game and it only cost about $200! That’s a massive savings over the time it would take to write myself. It puts me 2 weeks ahead in my development time.

    It’s easy to see the value in leveraging open source software, but even paying for code can sometimes provide a massive discount over custom crafted code.

  • Api First Web Development

    The web is still suffering through a page refresh disaster. HTTP started with simple pages where full page reloads when navigating wasn’t so bad. Modern sites have boatloads of images, css, js and html files on each page that need to be requested each time a page loads and checked to see if the file has changed to re-download it.

    It’s wasteful and slows down everyone’s experience with browsing.

    There are various approaches to overcoming these shortfalls: gzip everything, image atlas files, concat and uglify javascript etc. These lead to further technical complexity on the server side and nuances that make web development more difficult to get right.

    The rise of client side MVC frameworks is a breath of fresh air in web development.

    Ajax was just a stepping stone to this new web development paradigm.

    There are many competing frameworks right now: Ember.js, Backbone.js, Angular.js, and Meteor.js to name a few.  They provide several big wins for web developers: simplification on the server side (by removing the need for template engines, asset management, and a multitude of views)  This also allows for greater performance on the server side since template rendering is generally slow and rendering a full page is no longer required.  Web backends can be lean and optimized much more easily.  It lowers the bar to get other more performant languages like Go to compete with Django and Ruby on Rails.

    With an API first approach the front-end web development can be completely isolated.  All front end files can reside on high performance, highly available and cheap hosting like Amazon S3 and CDNed around the world.

    Done smartly the backend can be used against multiple front-end technologies: iOS, Android,  Windows Phone, and desktop apps can all use the same API.  It’s the best of both worlds. You get the superior feel of a native app while maintaining much of the code savings of consolidating business logic on the server.

    For the past few years we’ve been living in a web development world where there is a tug of war to decide what pages, or parts of pages should be rendered on the server side and what should be done with javascript and DOM manipulation.  It’s a design decision that has to be thought about consistently when working on complex web sites and generally creates a messy code base with javascript scattered around everywhere.  The future of API first web design makes it very clear. The server provides JSON, and javascript on the client side does the rest.

    From a testing perspective the backend becomes easier to unit test. CRUD interfaces and validations are simple to assert.

    The messiness of testing the UI now falls entirely on the javascript application where they can be safely written without needing a complete web stack of infrastructure. The MVC frameworks nudge you towards more organized and easier to test javascript code.

    For your next web development project consider taking this new approach to the web.  Simplify your backend by doing a RESTful JSON API only, and simplify your front-end by using one of the Javascript MVC frameworks.

  • Taking Rails Lessons to Django

    For the past couple of months I’ve been working on a number of Ruby on Rails projects with some talented and experienced developers.  After years of working in the Django world on various projects, Rails is an interesting contrast.

    My perspective on developing webapps has changed greatly since working with Rails. This is both due to technical differences as well as differences in the business goals of the projects I’ve worked on.

    The rails community has embraced speed of delivering finished apps to an unprecedented level. Using the Heroku stack and add-on services it’s easy to build a fairly complete application by selecting a series of services you need and installing some carefully selected ruby gems. That gets you 90% to a workable project in a lot of cases. The last 10% is adding in the business logic and custom models and views.  Seeing this app development though the selection and configuration of 3rd party services and software was eye opening.

    By embracing 3rd party services for things like logging, errors, email sending, and databases you are relieved of a significant amount of infrastructure and configuration setup time. Having a deployment process that ‘just works’ with a git push removes the need for writing custom scripts in Fabric, and Chef/Puppet recipes – so long as you’re willing to accept the higher costs for hosting.

    The Python community, even though it’s supported on Heroku seems to have missed the boat on this a little.

    I wanted to see how the speed of development could be tweaked for Django projects by making a couple changes especially on the early setup and bootstraping of new projects.

    The Python community actually has some tools that make creating projects from scratch even better than I expected.

    Cookie Cutter is an awesome tool for development teams that are creating new projects. You get to define a starting point that includes your favorite libraries and is organized the way you like. If you create lots of new projects from scratch then cookie cutter will save you a bunch of time. Rails bootstraps you with their preferred gems which gives developers a good starting point though it also includes things you might not like. Django developers need to invest in a bit of time to develop their own templates 

    Embracing class based views – For a long time I was a hold out. Functional views are easy to understand and require less looking up of documentation to see what things need to be called or overridden in order to do what you want.  Class based views however eliminate a lot of the common patterns you would be writing over and over again in a common app. Using the class based views in Django feels a lot more like rails to me, and once you grasp it they will save you development time.

    Use and love libraries. The variety of Ruby gems available is shocking. Even things like collecting in static files for something like jquery can save you some time. In Rails you can include jquery-rails gem and then include the js file in your template. two lines added to your project and everything is hooked up. The Django community hasn’t taken the same approach – adding jquery to a django project requires downloading the files from jquery.com yourself and putting them in the right folder before adding it to your template.  It may not seem like much but it makes a difference.

    Rails and Django are interesting to contrast. They’re similar enough and yet have taken different approaches to solve the same problems. Beyond the technical differences the community differences are fascinating.

  • New Site New Direction

    With the launch of the new design for this site we have also taken a new tack on the strategy going forward for future development of products and features.  The backend services that currently have been built for Halotis iOS Apps will be re-built so that others can benefit from the work done.  Where appropriate the source code will be open sourced and in other cases Halotis will launch and run some web services so that you don’t have to deal with the servers and infrastructure required to run them yourself.

    The new direction will aim us at creating services that enable real social integration for mobile games. Creating a social game requires more than just slapping a share my score button, it needs to be tightly integrated into the core experience.  This is achieved through team building, communication tools, interactions that players can have on each others’ game state. Halotis will be creating a series of simple to integrate services to power this kind of social gameplay.  We want all games to be better.

  • Building something bigger

    Halotis is in the process of a pivot. We’re re-writing all of the software developed for the backend of our mobile apps so that they can be opened up for anyone to use. We will be offering a host of BaaS style applications to make mobile development easier, more social and hopefully raise the bar for what individual developers can accomplish.

    Imagine individual developers able to launch in game chat features like that in Clash of Clans. Or provide social networks to tie players together.  We have tools for monitoring analytics, and providing game config files dynamically, for pushing ads and sending notifications.

    It will take some time, but progress is being made to launch a host of services.

    We’re excited for this change and to see where things go in the future.

  • Playing with Docker, Vagrant, Dokku for deployment

    I’m looking at launching a series of new web services in the future and that got me thinking about finding ways to improve my development and deployment workflow to make it super trivial to launch new services and APIs.

    Recent experience with Heroku’s PaaS got me looking in that direction. Pushing a git repository to kick off a server build is an eloquently simple approach that makes spinning up new services a snap.

    However Heroku costs can quickly spiral up and out of control. Running it myself on raw Amazon EC2 or a local dev server would be ideal.

    Thanks to several new projects it’s possible to create an easy to use mini-heroku on your own server.

    Docker is a brilliantly simple idea that has sparked a huge following in the last few months.  It is one of the most exciting developments in Open Source at the moment.  Basically Docker lets you run an extremely lightweight virtual machine on top of Linux.  You can spin up and down services in 1-2 seconds that are isolated from the host server which allows you to play with various server configurations quickly, easily, and without touching the core system.

    Docker allows you to specify your entire server configuration in a Dockerfile of which there are already thousands defined and shared and searchable at http://index.docker.io.  Need a WordPress server? Redis? Memcached? you can now get them up and running in a virtualized environment extremely easily.

    Dokku, builds on top of Docker with the ability to replicate the most impressive feature of Heroku: git push to deploy.  Once set up you can configure a git push remote and have Dokku automatically build and run a Docker for your app and host it on a subdomain.  Brilliant.

    Managing all these Docker services can be done with another awesome tool called shipyard. Bring your virtualized multi-host environment under control.  It even supports automatic restarts of crashed services.

    Getting it all up and running has been fairly straight forward. Once in place, it will make managing my own cloud of services much more robust.

  • Learning Ruby on Rails

    Rails has been the popular framework for generating web applications for a long time now, yet I’ve managed to avoid learning it until this month. However a new job gave me an opportunity to dig in and learn it.

    Having come from several years of Python/Django development learning Rails shouldn’t have been difficult. Ruby has a comparable feature set to Python and both Django and Rails frameworks perform the same roles. Yet the transition to Rails has proven to be more confusing than I expected.

    Perhaps the biggest thing having come from the Python world is that the belief in python is to make all code explicit and reduce the amount of “magic” that happens. Rails on the other hand is all about magic. declarations in models can affect how pages render and there are many.

    I have yet to find a good resource for discovering what is available in my namespace.

    The biggest thing I haven’t yet wrapped my head around is the declarative nature of the code. The problem with that is that it doesn’t leverage my existing knowledge of HTML/CSS/SQL and instead starts over. As a result the learning curve is steeper than I expected. The fast pace of development of the rails framework itself leaves documentation and online help confusing for new developers to follow.