Matt Warren

Author: Matt

  • Migrating to Amazon RDS

    I’ve been suffering from some scaling issues lately and since my system administration skills are fairly rudementary I decided to opt for Amazons RDS services for providing a MySQL server that is properly built with the appropriate block devices and automatically kept up to date by Amazon.

    Moving my existing database over to an Amazon RDS instance was relative straight forward.

    After launching an RDS instance from the Amazon AWS console.  I added my existing security groups to the security groups so they could connect to it.

    Once everything had a launched I was able to easily move my existing MySQL database over to the new server.

    mysqldump dbname -u root -p > dbname.sql
cat dbname.sql | mysql --host=hostname -u dbusername --password newDBname

    Once that had run I was able to update my django application to connect to the new server and all was good.

    Surprisingly painless. Thanks Amazon.

  • Steady Growth

    I’m finally seeing some steady progress on the iPhone/iPad app side of my business.  It’s been growing for the last few months rather steadily and with the latest updates I’ve further optimized the monetization and am now making a stable cash flow.

    The goal I had starting out was to make $200/day which would roughly be enough to pay all my living expenses.  It’s not there quite yet, but I’ve recently passed 10% of that goal.  Things are lining up now to quickly be able to double that income in the next few months.

    Getting  up to $50-$75 per day by Christmas is my goal for the next few months.  It should be realistically doable.

     

  • An Easy VPN on EC2

    I’ve become a little bit more paranoid over the last few weeks about computer security. I used to think I will be fine – why would anyone want to hack my computer and find my personal files. But after a bit of reading I’m more concerned about the far more likely situation of mass spying and infiltration programs either churning through my network traffic or infecting my computers with a botnet. In those cases people will collect everything they can and data mine it afterwards.

    Starting a VPN should help greatly with preserving my privacy.  It also gives me the added benefit of having an IP address in the USA which allows me to watch Hulu and other US only content sites.

    After several false starts trying to get OpenVPN working I decided to try something easier.

    Starting with an existing AMI was easy.  I searched for and launched a new instance using ami-744c971d which is pre-configured for pptp with a simple username/password.  Create a new security group for the instance when configuring it.

    Once the server is launched and running you’ll need to open up some ports so that it is accessible.  PPTP uses TCP port 1723.

    In the Security Group settings open up TCP port 1723 and TCP port 22 (SSH).

    Ssh to the server using the key pair you selected when starting the instance and the username ‘ubuntu’ to edit the settings file.  Edit the file /etc/ppp/chap-secrets  change the username ‘seamus’ and password ‘wiles’ to something else.

    Restart pptp: ‘sudo service pptpd restart’

    You should be able to connect then to the new VPN server.

     

  • First Experience Scaling Web Apps on EC2

    Last week I had a bit of a crisis situation with my apps.  After several mornings of waking up to find the server not responsive and 700+ emails waiting in my inbox something had to be done.

    When starting out I leaned towards keeping the costs as low as possible.  That meant putting several virtual hosts (apache) and MySQL on the same server.  But as the traffic for the site grew I started getting more and more crash situations.  It took a while to diagnose but evenutally I figured out that HTTP requests would come in and eventually use up all the memory on the server at which point MySQL would crash and restart.  In the time it took MySQL to restart I would end up with several hundred error emails from failed requests.  Fun.

    After some struggling I decided to launch another instance to host my most taxing web application.  Thus isolating the memory requirements of apache from MySQL.

    Setting the new server up behind a load balancer will make future scaling much easier.

    Doing something similar without the flexibility of Amazon’s AWS would have been a much bigger nightmare.

    Luckily now that the apps are making decent money paying for the extra server usage is easy to justify.

  • Writing another (short) book

    I have a book available in for the kindle that I wrote several years ago. It’s been doing ok for sales considering that I have never really done anything to promote it.

    This past week I have been following someone who is selling a course on kindle publishing and I learned a few things from watching his examples. It’s got me a bit charged up to get another short book written up and published on kindle.

    Over the last couple days I hammered out the first 4500 words of my goal of 10,000 words about brewing beer. Hopefully I can pull this together quickly and get it up for sale by the end of September.

    After this one I already have an idea of what I want to write next.

    It’s strange that I never considered myself a good student in English class at school and yet I’ve managed to keep this blog going for nearly a decade, have one book published on Amazon.com, a second one now 50% completed and a third book in the research stage.

  • Using tmux and Vim

    Working on the same projects from multiple computers can be difficult.  There are lots of solutions for this problem such as using Dropbox to sync files between computers. but that leaves your workspace to start from scratch ie when working from the office last I had these 10 files open, and a couple of things running, then when I get home if the files sync over I have to reopen those files locally and configure that computer for anything else that needs to run to continue debugging.

    Enter Tmux and Vim.

    Lately I’ve been working 3 days at home, 2 days at the office each week.  As a result I have re-evaluated my workflow to find something a bit more efficient.  Vim in the terminal is brilliant once you’ve got it set up and have mastered it.  It has been my main editor for about 3 years now.  There are a ton of great resources out there to learn Vim for those of you still trying to find the way.

    Tmux is new to me though.  It’s basically a better version of screen.  If your not familar it basically provides a way to do 2 main things.   First it allows you to create a persistent session that you can connect to and leave running.  Secondly in this session you can open up and layout many different consoles into tabs or windows. It’s sort of a command line version of VNC.

    I have tmux installed on my Linux servers on AWS so that I can work efficiently on those machines, and I have it installed on my workstation at the office.  I can log into the environment with SSH and continue right where I left off even if I’m switching between my laptop and a friends computer.

    With Vim I have the ability to keep my settings and plugins saved and configured the way I like them with a configurations version controlled on github.  Getting a new Linux or Mac machine up and running takes very little time to do with this setup.

    I’ll give my best tips and tricks for Vim in another post.

  • Write a lot of Code

    I write a lot of code.  I love it.  The challenge and creativity of solving problems along with the hope that the right program could save either myself or many other people their valuable time or provide them something better to do with their time is a great reward.

    There are about 5 software projects that I’m in the process of working on.  All of them have me excited.

    One interesting thing I’ve noticed is just how much more difficult it is to code when working on a project in a team.  On my own personal projects I have no problems with adding or removing 500 lines of code in a day because I have a pretty good understanding of how everything works.  With a team project there are more moving pieces and therefore every change requires second guessing, searching for usages and being vigilant to validate arguments and document more thoroughly.

    Skipping the boiler plate code and coding as much as possible as quickly as possible gets me to the real challenges quicker, forces me to learn things at a more rapid pace, and arrive at a functional (but perhaps more buggy) product sooner.

    Learning new things and always trying to practice to gain mastery in my craft is a never ending journey.  Putting fingers to keyboard to solve problems is an addiction I can’t shake.

    Tomorrows Goal: write more code.

  • App Control Demo

    App Control is a Django App that I have been evolving over the last year for a backend service to support my (and my friends) iOS apps.

    It allows you to update them, push new content to them, and provide some central storage for building things like a commenting engine and to tie everyone who uses your apps together within a community.

    It also works to pull in marketing information, ad revenue, sales, updates, reviews etc and add them together to compute some higher level business numbers such as RPI (revenue per install).

    Next features I plan on adding include a generic currency/wallet system and store for dynamically providing in game content and allow price changes without submitting an update to Apple.

    I’m wondering if this sort of app would be of interest to anyone else out there who has iPhone Apps.

  • The singularity

    In the last few days I read two books. I don’t read very many fiction books but these got me thinking.

    The singularity is a term taken from physics. In the world of physics a singularity is an object or event in space where the laws of normal physics break and thing behave in wild and unpredictable ways. The term has been taken over to the world of technology to be the point in time when computers become intelligent enough to improve themselves. Once there is a computer AI as smart as a human it will not take long for it to start coding on itself and eventually becoming 2x, 5x, 100x smarter than anyone on the planet.

    When this happens (not if) the world will change dramatically very quickly. But whether or not it will be good or bad is unknown. Any futurist cannot be certain of anything afterwards.

    Will people have jobs if a computer is capable of everything we can do? With mass unemployment as factories become completely automated, followed by AIs redesigning the products and taking over business management roles will it result in a wealth of free time or crises as people have no money.

    Will removing people from the supply chain bring the prices of goods to zero? Probably not since there will be at least a few thing that will remain scarce such as property, and time. How that will be handled if no one makes any money is anyone’s guess.

    Will an army of automated robots be able to re-forest the Sahara desert and save the planet from climate change?

    It is a fascinating scenario to think about because literally every industry will change. There are so many variables and uncertainties that a thorough discussion would probably fill a 1000 page book.

    The role of science fiction is sometimes to explore these ideas and present them to a wider audience so that culturally we can be somewhat more prepared for what the future may bring. The singularity is something far more likely to happen in the next 30 years than the invention of faster than light travel or transporter pads and yet it has been off the radar for most people.

  • iTunes Download Stats

    Recently I’ve been building out my iPhone App server to provide a business dashboard with all the relevant services and numbers that I care about available at a glance. It avoids me having to sign in and out of many different sites to get the information and makes it easier to push things together – for example charting both Admob and iAd data on the same graph.

    Thankfully the web is becoming more programmable every week and these things are becoming easier to put together quickly.

    This is a chart I built last night to display the downloads and updates across all my apps for the past 31 days:

    You can see the jumps in downloads that correspond to when I released updates to iTunes.

    With these sorts of things I’m finding that there is a tipping point. If the custom page I have created is only 90% as good as going to the original source then I’ll just opt to login there but once it becomes as good as or better than that you’ll quickly forget about the 10 different logins you needed to get all those numbers.

    Being in charge of it is even better. I use iAd and Admob for advertising and can pull those numbers in and compare them appropriately. On the same page I display data from Apple, Google, Linkshare, as well as numbers I collect myself such as traffic, link clicks and ad impressions. I only have to login to iTunes Connect to release new Apps.

    I will continue open sourcing the components for this system over the next few weeks.